These are the release notes for Pelias, the open source project. It includes news on the latest code changes as well as new features. There are also release notes specifically for the hosted instance of Pelias run by Mapzen, Mapzen Search.
13 March 2017
- We've added postalcodes to the Who's on First import process and enabled the postalcode-only query type, so users can now find postalcodes directly!
- This update required additions to the Pelias schema as well as query building logic.
- The download script in pelias-whosonfirst importer has also been updated to pull down postalcode data by default, so be aware when using that download command and when running the importer.
- This also means Who's on First imports will now take roughly 3 times as long, if importing the whole world... Yeah, there are a LOT of postalcodes in the world!
Pelias <3 Postalcodes
Code level changes
- Refactored pelias-wof-admin-lookup and updated to this latest version in all importers.
- This is in preparation for the upcoming fixes to our coarse reverse lookups.
- Next step will be to direct the API to this new Point-in-Polygon service when available to get more accurate reverse geocoding results.
- Geonames importer interface has finally been updated to match our other importers.
- Dependencies should now have the proper alpha3 ISO codes of their own in the country abbreviation (
country_a) properties, instead of alpha2 of the parent country as it did previously. See San Juan, PR for example.
- Washington DC wasn't getting a region abbreviation at all, but that's water under the Arlington Memorial Bridge now!
1 February 2017
- We're working on some fancy new things in dev so our only new feature released this week is retrying timed-out elasticsearch requests. By default, Pelias will retry a timed-out request up to 3 times before failing for good, though this number can be overridden by specifying
api.requestRetriesin your Pelias configuration.
27 January 2017
- Big news! 🐯 we have soft launched our new street interpolation service which includes TIGER data. This allows us to return more address results than before. For more info see https://github.com/pelias/interpolation
- We've fixed a bug where structured queries would always return 'fallback' as the 'match_type'.
17 January 2017
/v1/structuredendpoint now supports the
venueparameter, which allows for searching for venues with a particular name.
- We've improved result balance when using
focus.pointin the autocomplete endpoint. In particular, searching for cities far away from the focus point should work much better. More improvements to
focus.pointare planned for the near future.
- We've finished adding Pelias config validation (for example here in the Who's on First importer) to all of our repositories. It should be much harder for simple config errors such as typos to go undetected!
- The checks run before dropping data using pelias-schema have been improved. Accidentally deleting all your production data should now be even more difficult than before!
5 January 2017
Code level changes
- We are now considerably more strict about read errors when loading Who's on First data. Any read errors, whether due to missing files or invalid JSON will cause the entire importer to fail immediately. This prevents builds from reporting succesful completion when some data is actually missing.
- The Elasticsearch
index_concurrencysetting has been removed. In theory this setting allowed tuning imports for better performance, but in practice it had little effect. It's going away in Elasticsearch 5, so this will make the transition that much easier.
28 December 2016
- Searches for
St Louis, MOand
Saint Louis, MOnow return the same thing (the same goes for towns starting with
Code level changes
- Structured geocoding no longer fails horribly when the
addressparameter consists of only a house number
05 December 2016
- We've released what was previously referred to as component geocoding in the new structured geocoding endpoint! It lives at
- We fixed a bug where specifying the same parameter twice (eg
/v1/search?text=paris&sources=geonames&sources=gn) would cause a 500 error. It now returns a helpful 400 error message that includes which parameter is duplicated, so that the request can be fixed.
- Other errors that should have been 500 errors were being returned with status code 400. Fixing this will allow us to more quickly catch any 500 errors that happen in the future.
18 November 2016
- We've just released beta support for component geocoding so instead of passing in a single input to the
/v1/searchendpoint, the parts of an address can be sent to
/v1/beta/component! An example of this is
address=201+Spear+St&locality=San+Francisco®ion=CA. We haven't officially named this geocoding type yet, so if you have a naming suggestion, please weigh in here! Our basic design doc for using this new beta feature is here, please check it out. We're still working out the final implementation (why it's currently deployed to our
/v1/betatest bed) so check it out and don't hesitate to raise any issues you might encounter. Check out the acceptance tests for some more examples.
Code level changes
- We're enabling support for more response scenarios from libpostal! This release we're adding support for city+country, so requests for Paris, France and Reykjavík, Iceland are a lot cleaner.
- Speaking of Reykjavík, Iceland, support for inputs containing diacritics has improved. Now whether the input is Reykjavík, Iceland or Reykjavik, Iceland, results should be the same.
- Whether your input contains a 2- or 3-character ISO country code (
FR), we'll find it!
24 October 2016
New features and fixes
/v1/autocompleteendpoint now supports boundary.rect just like
- Labels for administrative areas should be improved in a few cases
Code level changes
- All Pelias code now requires Node.js 4.0 or newer! This means we can start using ES6 features.
letus have a party to celebrate! :tada:
- Label generation code has been extracted into pelias-labels
- Elasticsearch settings have been moved out of a dark corner and into pelias/config
- The whosonfirst importer has a better method for calculating hierarchy information. It's simpler and should be more accurate and faster!
10 October 2016
- libpostal, the super-sophisticated address parser, has been integrated for more accurate analysis of inputs at
- Street names containing post-directionals (e.g. -
186 Tuskegee St SE Atlanta GA->
186 Tuskegee St SouthEast Atlanta GA) are now treated the same as their pre-directional brethren.
- 10/10, would release again - geocoding fallback rules that favor traditional geocoding behavior instead of search engine behavior
19 September 2016
Another data-only release. Stay tuned for next week!
12 September 2016
- Get excited for the addition of ✨ STREETS ✨! That's right, with this release Mapzen Search gets a brand new
streetlayer, which contains OSM street centroids. With this addition, if we can't find the exact address you're looking for we'll return the street record. Stay tuned for an in-depth blog post in the next few days. 👏
7 September 2016
- Crikey! We noticed we weren't handling Australian province abbreviations, so we added support for them in our labels.
- Geonames ADM3 records now are correctly listed as localadmins, not venues.
- Our wonderful, now departed intern made sure Germanic street names are consistently handled (previously, some would end in -strasse while others ended in the abbreviation -str).
- Records with a Who's on First dependency now have that dependency listed in API responses.
22 August 2016
No changes in functionality at all, just the freshest data! We did clean up some tests and do other work only visible to developers and those who run their own Pelias instance, but nothing major.
Stay tuned for next week's release where we already have some nice changes queued up.
18 August 2016
- After much feedback we've added the
boundary.countryparameter for autocomplete! It works just like the one on the search endpoint.
- To help make Leaflet maps display results better, we now use use the
lbl:bboxproperty on Who's on First records. This is useful for places like San Francisco where the mathematical bounding box is bigger than people expect.
- The API was incorrectly warning against using the
boundary.circleparameter. Now it doesn't complain!
- We've added a new
/v1/nearbyendpoint that is currently in early alpha! There's no documentation, probably some bugs, and any part of the interface is still subject to change.
- Finally, we're now running Node.js 4 in production, rather than Node.js 0.12. For those running their own Pelias instance, be aware that we'll be dropping support for Node.js 0.12 in September. At first, things may work on Node.js 0.12, but we're very excited to finally start using ES2016, so that won't last too long.
8 August 2016
Incremental release resolving the final outstanding tasks in the Elasticsearch 2 upgrade.
We have registered a new website http://pelias.io which has information about the milstones we have planned for the current quarter.
- Elasticsearch 2+ does not support co-ordinate wrapping as it did prior to the 2 release. Some front-ends allow users to 'wrap' around the globe. Floats outside of the normal -90/+90 -180/+180 geographic coordinate ranges cause Elasticsearch to error. We added a function to the API which unwraps these coordinates; providing better compatibility with these tools.
- We added
boroughas a possible layer for Geonames
- Since the beginning of the project the Elasticsearch
_indexname has always been hard-coded as 'pelias', the index names configurable PRs allow this behaviour to be adjusted in your individual pelias config files.
- We removed the focus.viewport API which was undocumented and never used outside of test suites.
2 August 2016
Another bigger than usual release, we had some ops related challenges to resolve after the update to Elasticsearch 2, as well as some data issues, but we also have some great improvements in store!
- We use more of the population data in Who's on First, which really helps more relevant cities come up in searches.
- Searching for only records in certain layers in Geonames now works! We keep adding better handling of Geonames data but sometimes our API code doesn't keep up with those changes.
- Labels now include county names if there's no city (locality) info present. This helps with addresses that are outside the bounds of any city
- Capitalization across all OpenAddresess records is now more consistent. We've tried to properly capitalize all records that were either in all caps or all lowercase. This is better in general, although there are certainly exceptions, and we welcome bug reports for those cases.
- Geonames records for New York City boroughs like Manhattan and Brooklyn are now in the
borough, rather than
localitylayer. This makes them consistent with the records from Who's on First, which have been boroughs for some time.
- Addresses in the Czech Republic now show the street name before the house number, in keeping with local customs
- When using the
/v1/placeendpoint, the source name can either be the full name or the abbreviation (like the
sourcesparameter to the search and autocomplete endpoints). We love saving people some typing :)
- We've made lots of internal changes like reducing the size of our documents, using a cleaner method to construct layer filter queries, removing dependencies on packages we've deprecated, and allowing the Elasticsearch index name to be configured for both the API and schema packages.
- In related internal changes news, we've also worked to make sure that all our code works with Node.js version 6, which was recently released! Support for Node.js 0.10, which is quite old and near end-of-life, is also starting to be removed.
We also have two known issues in this build:
- Some OpenAddresses records for the statewide data in Massachusetts, USA are incorrect. This is because of an issue when changing data sources that will be resolved in the next OpenAddresses build
localadminrecords, like the City of New York will have extra components in the label (in this case, "Brooklyn, New York"). The fix for this is merged but was accidentally omitted from this build. Look forward to it next week!
07 July 2016
- Big news: We've finally upgraded to Elasticsearch 2.3! This brings improved performance and more importantly sets us up for lots of improvements from the new features of Elasticsearch 2. Elasticsearch 1.7 is no longer supported.
- As part of the Elasticsearch 2 upgrade we've also improved a few edge cases for searching for numeric values, and with single character tokens. You can read more in the Github issue for the upgrade.
- We've also fixed some lingering issues where a few places in Denmark were listed as being part of Sweden. This was due to the same data bug as mentioned in our recent blog post.
- The OpenAddresses importer now has better whitespace cleanup, so there won't be any extra spaces in street names.
- We recently added data to new layers in Geonames, but the API didn't know about it, and prevented you from searching for them. We fixed it.
13 June 2016
- Who's on First importer: records now use the label centroid if it's present. The previous behavior was to always use the center of the record's bounding box. In cases like San Francisco, this caused the record to not show up where people expect!
- Openstreetmap importer: A bug in config parameter handling that caused admin lookup to be disabled when it shouldn't was fixed. Thanks to @dylanFrese for helping us catch this tricky one.
26 May 2016
- We did it... we removed an Elasticsearch analyzer that was presumptuously assuming all queries were in English! The
k-stemminganalysis would do strange things like turn Daly into Dale, so finding "Daly City" was a challange. Well, no more! Word of warning, in
/searchwe are now less forgiving when someone uses a plural version of a word where the real name is singular.
23 May 2016
- All the extra 0's have been eradicated in addresses coming from OpenAddresses. You should not see any house numbers that reduce to 0 or any leading 0's in house numbers.
- Added the mysteriously missing
source_idproperty to response features. This property represents the original id at the source, if one existed, like in OSM and WOF. Where it didn't we made one up to help uniquely identify each record.
09 May 2016
- Cleaned up some invalid address data from our OpenAddresses import by removing anything with words like
- Improved error reporting in the API so users can decipher what went wrong much easier. More specifically, there are errors that Elasticsearch reports and we propogate up to the API response.
29 April 2016
- A big improvements to autocomplete results coming from numerous bug fixes and improvements! More details can be found in the pull requests: pelias/schema#127 and pelias/api#526. Some highlights include:
- Single digit housenumbers like
8 Main Stcan be found more easily
- Support for searching for the street name before the house number, as is common in many European countries, is improved.
- Searches that end in common words no longer return no results. These were being treated as stopwords internally in Elasticsearch. Now queries such as
Moscone Westwill work better
- Single digit housenumbers like
- Remove OpenAddresses records with 0 housenumbers in US/CA
18 April 2016
- Address parsing now works without spaces after commas. This was our bad. Feel free to leave those spaces out as long as you provide commas to delimit admin parts.
- Further streamlining of labels. You can expect the labels to a have more consistent and minimal feel. If the results are coming from New York, expect boroughs such as Manhattan, Brooklyn, Queens, etc. to be part of the label. You're welcome New Yorkers! :heart:
- Fixed a bug where specifying
layers=macrocountywould fail due to a typo in the API code. You can see how easy it is to mistype
macrocountyand not notice that
macrocountryis incorrect. #onlyhuman
08 April 2016
This release marks the official integration of the Mapzen
Who's on First data set into Mapzen Search. This data is replacing
Quattroshapes across the entire service. Any forward usage or references to
Quattroshapes will be replaced with
WhosOnFirst. This substitution allows us to fix long-standing encoding issues in administrative hierarchy place-names. We've also added a bounding box for individual features in the results, not only the all-encompassing bounding box at the top level of the geojson results. Also, the all-encompassing bounding box will extend to include the bounding boxes of all the features in the results, not only their centroids.
Another major improvement that many have been waiting for is the addition of more filters for the
/autocomplete endpoint. Users can now ask
/autocomplete to filter by
sources, as documented here.
See the detailed list of changes below for more specifics.
- Switched from
WhosOnFirstas the canonical source for administrative hierarchies and corresponding geometries.
- No longer importing
WhosOnFirstcontains all those records and more. Going forward, any use of
qsin queries will resolve to
bboxproperty has been added to individual results, for which geometry was available in the original source. This does not affect POI and address data.
- Drastic improvements have been made to the label generation logic.
gidformat has changed to make the ids more unique.
- New id format resolves previously outstanding bugs related to
geonamesids being invalid for lookup via the
- Additional place-types have been introduced, such as
gidvalues have been added for each parent in the admin hierarchies of results.
/autocompletenow allows filtering by
- Fixed a bug that allowed
/autocompleteto accept the
sizeparameter. The default and only size of
/autocompleteresults is now
10, as originally intended.