[index] Automatic location completion
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information. There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags. Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags. I think this is a better foundation for the discussion how we want to fill the missing location information. Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype I would be happy if you try it and make some proposals how it could be improved. Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used) Have fun! WanMil
Nice! Will test later. Just one question, can this handle relation boundaries? On Wed, Feb 23, 2011 at 6:34 AM, WanMil <wmgcnfg@web.de> wrote:
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------
Am 22.02.2011 23:34, schrieb WanMil:
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
This seems to be a good idea. And which location-autofill option has to be used with this ? Chris
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
This seems to be a good idea.
And which location-autofill option has to be used with this ?
Chris
At the moment it is independent from the location-autofill mechanism. As far as I understand the autofill mechanism all items that are assigned with proper information by the new algorithm must not be completed by the old autofill. In the end it should replace the autofill mechanism (if it's better...) WanMil
Hi, if you just want to complete addr:*-tag, it's good to use addr:* prefix. But I think this isn't useful for searching streets, because many streets in OSM haven't any correspondending addr:-tags. If you want to set add:* directly to the streets, I think another prefix would be better, just to keep it seperate. Henning
Hi, if you just want to complete addr:*-tag, it's good to use addr:* prefix.
I don't understand. Where should I use the addr:* prefix?
But I think this isn't useful for searching streets, because many streets in OSM haven't any correspondending addr:-tags.
If you want to set add:* directly to the streets, I think another prefix would be better, just to keep it seperate.
I have chosen the addr: tags consciously because they are already handled by mkgmap. I don't see a big advantage to invent a new tag for which new special handling must be implemented. Am I wrong? WanMil
Henning
Hi
if you just want to complete addr:*-tag, it's good to use addr:* prefix. I don't understand. Where should I use the addr:* prefix?
addr-tags are used to display the address of an object. If you just want to complete existing objects with tags from relations, then addr:* should be ok. But If you want to create new objects e.g. all streets with addr-tags, this would make it impossible to keep them separate from the original objects with addresses. I think using only existing objects, wont be useful, because there isn't a high density of address-objects to have enough points for linking with streets. So it would be better to use relations to put tags to all streets in a form like mkgmap:city= mkgmap:district= mkgmap:state= mkgmap:country= Also it would be useful to set an global default for each of these tags in mkgmap-call, because it could be possible, that there isn't any country relation in an extract. Henning
Hi
if you just want to complete addr:*-tag, it's good to use addr:* prefix. I don't understand. Where should I use the addr:* prefix?
addr-tags are used to display the address of an object. If you just want to complete existing objects with tags from relations, then addr:* should be ok. But If you want to create new objects e.g. all streets with addr-tags, this would make it impossible to keep them separate from the original objects with addresses.
I think using only existing objects, wont be useful, because there isn't a high density of address-objects to have enough points for linking with streets.
So it would be better to use relations to put tags to all streets in a form like mkgmap:city= mkgmap:district= mkgmap:state= mkgmap:country=
Also it would be useful to set an global default for each of these tags in mkgmap-call, because it could be possible, that there isn't any country relation in an extract.
Henning
That's a good reason to use a separate tagging scheme for the location completion information. I'l put it on my todo list. Thanks! WanMil
El 22/02/11 23:34, WanMil escribió:
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled I suppose you mean in case boundary polygons are missing. * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used) According to [1], there's a great variety in the matching of admin_level=* to places across countries, so it could be useful to have some options to choose what admin_level one want to assign to a given place=*. In a first try of your patch in Spain I see some streets are assigned suburbs while they should be (IMHO) assigned to cities. Searching for several streets and leaving city blank I get three (by now) patterns: street, city, region, country (the right one for me) street, suburb, province, country (street assigned to nearest place=suburb node) street, suburb, region, country (I don't know if street is assigned to boundary polygons or to nearest place=suburb node, because both exist and unpatched mkgmap gives the same result). I could give you ID's of involved nodes, streets and relations if you need them.
[1]http://wiki.openstreetmap.org/wiki/Key:admin_level#admin_level
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled I suppose you mean in case boundary polygons are missing.
Not really. I thought that I should not add any address information if the is_in tag is already set. But the is_in tag is very problematic. So I should delete this point.
* boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used) According to [1], there's a great variety in the matching of admin_level=* to places across countries, so it could be useful to have some options to choose what admin_level one want to assign to a given place=*. In a first try of your patch in Spain I see some streets are assigned suburbs while they should be (IMHO) assigned to cities. Searching for several streets and leaving city blank I get three (by now) patterns: street, city, region, country (the right one for me) street, suburb, province, country (street assigned to nearest place=suburb node) street, suburb, region, country (I don't know if street is assigned to boundary polygons or to nearest place=suburb node, because both exist and unpatched mkgmap gives the same result). I could give you ID's of involved nodes, streets and relations if you need them.
[1]http://wiki.openstreetmap.org/wiki/Key:admin_level#admin_level
Yes, I expect such configuration needs. It seems as if it will be necessary to have a config file for this information (which addr-level should be used for which information in which country). I'll put it on my list.
I have done some development and want you to share my current findings. 1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also. 2. In the patch the Locator is disabled as much as it was possible for me up to now. 3. I am using now separate tags (mkgmap:city, mkgmap:region etc.). I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works. WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
El 27/02/11 22:33, WanMil escribió:
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works. Some findings running your patch on Spain's map: 1-The list of regions (state/country field) is much better than the one obtained with trunk. All those included are actual regions (some with two different names, e.g. Castilla la Mancha & Castilla-la Mancha). Trunk includes many names that are not actual regions of Spain, but provinces, cities or even villages. 2-The list of countries has grown from ESP (España), España (ESP) to Es, ESP (España), España (ESP), Gribraltar / United Kingdom, Spain, Territorial water of Ibiza and Territorial waters of Mallorca. Could we have some mechanism to unify all forms of a country in a single one, i.e. Es, ESP, España, Spain and Territorial water...->España? Maybe LocatorConfig.xml could/should do that. 3-When the country field is filled in any of the search tabs, State/Province info is not taken into account. 4-When searching for a street, city, region and country information is missing in the results list. Searching for a street+city doesn't work; street+(State/Province | Country) does work.
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works. Some findings running your patch on Spain's map:
Thanks for having a look on it! It's still on heavy development and I am glad about ANY response!
1-The list of regions (state/country field) is much better than the one obtained with trunk. All those included are actual regions (some with two different names, e.g. Castilla la Mancha& Castilla-la Mancha). Trunk includes many names that are not actual regions of Spain, but provinces, cities or even villages.
That's fine! I don't understand why you get two different similar names. I think this is caused by addr: tags that don't use the same spelling like the boundary multipolygons. Do you know about any similar name detection algorithm? So something like a "sounds-like(String cityname)" function? This would be necessary to fix that.
2-The list of countries has grown from ESP (España), España (ESP) to Es, ESP (España), España (ESP), Gribraltar / United Kingdom, Spain, Territorial water of Ibiza and Territorial waters of Mallorca. Could we have some mechanism to unify all forms of a country in a single one, i.e. Es, ESP, España, Spain and Territorial water...->España? Maybe LocatorConfig.xml could/should do that.
Yes, fully agree. I have fixed that in my local code and it's working fine.
3-When the country field is filled in any of the search tabs, State/Province info is not taken into account.
Mmmh, could you also try that with the trunk? At the moment I have no idea why the autolocation could be the reason for this.
4-When searching for a street, city, region and country information is missing in the results list. Searching for a street+city doesn't work; street+(State/Province | Country) does work.
Aah, I tried to compile Spain and I got the same results. I have to look into it. Compiling of Germany was VERY fine. First result: some administrative boundaries seems to be not closed (e.g. http://www.openstreetmap.org/browse/relation/346488). Up to now the code works with closed boundaries only. This has to be improved. WanMil
1-The list of regions (state/country field) is much better than the one obtained with trunk. All those included are actual regions (some with two different names, e.g. Castilla la Mancha& Castilla-la Mancha). Trunk includes many names that are not actual regions of Spain, but provinces, cities or even villages. That's fine! I don't understand why you get two different similar names. I think this is caused by addr: tags that don't use the same spelling like the boundary multipolygons. Do you know about any similar name detection algorithm? So something like a "sounds-like(String cityname)" function? This would be necessary to fix that.
Look for the SOUNDEX algorithm. It is described at least at the german and english wikipedia. It was originally developed to find similar names in genealogy, but I think it could be well used in your situation, maybe with slight modifications.
Am 01.03.2011 19:38, schrieb Johann Gail:
1-The list of regions (state/country field) is much better than the one obtained with trunk. All those included are actual regions (some with two different names, e.g. Castilla la Mancha& Castilla-la Mancha). Trunk includes many names that are not actual regions of Spain, but provinces, cities or even villages. That's fine! I don't understand why you get two different similar names. I think this is caused by addr: tags that don't use the same spelling like the boundary multipolygons. Do you know about any similar name detection algorithm? So something like a "sounds-like(String cityname)" function? This would be necessary to fix that.
Look for the SOUNDEX algorithm. It is described at least at the german and english wikipedia. It was originally developed to find similar names in genealogy, but I think it could be well used in your situation, maybe with slight modifications.
I have searched a little more and found the metaphone algorithm as a successor of the soundex. For metaphone is a java class already available at http://commons.apache.org/codec/apidocs/org/apache/commons/codec/language/Me... I have never used this, but looks quite reasonable. Regards, Johann
El 01/03/11 19:24, WanMil escribió:
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
Some findings running your patch on Spain's map:
Thanks for having a look on it! It's still on heavy development and I am glad about ANY response!
1-The list of regions (state/country field) is much better than the one obtained with trunk. All those included are actual regions (some with two different names, e.g. Castilla la Mancha& Castilla-la Mancha). Trunk includes many names that are not actual regions of Spain, but provinces, cities or even villages.
That's fine! I don't understand why you get two different similar names. I think this is caused by addr: tags that don't use the same spelling like the boundary multipolygons. Do you know about any similar name detection algorithm? So something like a "sounds-like(String cityname)" function? This would be necessary to fix that.
No, I don't know such algorithm (I'm not a developer) and am not sure it would fix all cases, because some names are quite different as they are in different languages, e.g. Euskadi/País Vasco. But perhaps one of the names could be discarded or fixed in the data: I have three regions with two different names; for two of them, searching for one of the names finds all of the places and the other name finds nothing. For the third one, the second name only finds one place.
2-The list of countries has grown from ESP (España), España (ESP) to Es, ESP (España), España (ESP), Gribraltar / United Kingdom, Spain, Territorial water of Ibiza and Territorial waters of Mallorca. Could we have some mechanism to unify all forms of a country in a single one, i.e. Es, ESP, España, Spain and Territorial water...->España? Maybe LocatorConfig.xml could/should do that.
Yes, fully agree. I have fixed that in my local code and it's working fine.
Could you use it also for regions?
3-When the country field is filled in any of the search tabs, State/Province info is not taken into account.
Mmmh, could you also try that with the trunk? At the moment I have no idea why the autolocation could be the reason for this.
I tried, and if you give a State/Province only places in it are found.
4-When searching for a street, city, region and country information is missing in the results list. Searching for a street+city doesn't work; street+(State/Province | Country) does work.
Aah, I tried to compile Spain and I got the same results. I have to look into it. Compiling of Germany was VERY fine. First result: some administrative boundaries seems to be not closed (e.g. http://www.openstreetmap.org/browse/relation/346488). Up to now the code works with closed boundaries only. This has to be improved.
This boundary is open by the coast side, as many others (¿all?) in Spain. Something similar to the close-gaps in the sea generation could do the trick.
2-The list of countries has grown from ESP (España), España (ESP) to Es, ESP (España), España (ESP), Gribraltar / United Kingdom, Spain, Territorial water of Ibiza and Territorial waters of Mallorca. Could we have some mechanism to unify all forms of a country in a single one, i.e. Es, ESP, España, Spain and Territorial water...->España? Maybe LocatorConfig.xml could/should do that.
Yes, fully agree. I have fixed that in my local code and it's working fine.
Could you use it also for regions?
Of course it would be possible although you get the problem of maintaining this list.
3-When the country field is filled in any of the search tabs, State/Province info is not taken into account.
Mmmh, could you also try that with the trunk? At the moment I have no idea why the autolocation could be the reason for this.
I tried, and if you give a State/Province only places in it are found.
Ok, so it seems a problem caused not only by the autolocation.
4-When searching for a street, city, region and country information is missing in the results list. Searching for a street+city doesn't work; street+(State/Province | Country) does work.
Aah, I tried to compile Spain and I got the same results. I have to look into it. Compiling of Germany was VERY fine. First result: some administrative boundaries seems to be not closed (e.g. http://www.openstreetmap.org/browse/relation/346488). Up to now the code works with closed boundaries only. This has to be improved.
This boundary is open by the coast side, as many others (¿all?) in Spain. Something similar to the close-gaps in the sea generation could do the trick.
That's not a nice job to do... We might continue the boundaries along coastlines. This would be a job for a specialized BoundaryRelation class that inherits from MultiPolygonRelation. I have finished my checks with Spain and the result is perplexing: The splitted tiles do not contain any boundary tag. I have used splitter r164. The .pbf file does contain them. So it seems to be a splitter bug (or I have used it wrong). WanMil
This boundary is open by the coast side, as many others (¿all?) in Spain. Something similar to the close-gaps in the sea generation could do the trick.
That's not a nice job to do... We might continue the boundaries along coastlines. This would be a job for a specialized BoundaryRelation class that inherits from MultiPolygonRelation.
I have finished my checks with Spain and the result is perplexing: The splitted tiles do not contain any boundary tag. I have used splitter r164. The .pbf file does contain them. So it seems to be a splitter bug (or I have used it wrong).
Ways that cross tile boundaries may be incomplete. When generating a tile, the splitter includes in each tile all of the nodes in the tile, all of the nodes within OVERLAP of the tile, and then any ways that use any of those nodes and finally any relations that use those ways or relations. If a way denoting a boundary has widely spaced nodes, where none occurs in the overlap region, the way will have a gap between the the last node in the region and the border. At present, it doesn't go back and 'fill in' missing nodes in those ways/relations, and it has never had that functionality. Doing that would require additional passes, but I think it could be added in a few days of work. Scott
Am 02.03.2011 04:52, schrieb Scott Crosby:
This boundary is open by the coast side, as many others (¿all?) in Spain. Something similar to the close-gaps in the sea generation could do the trick.
That's not a nice job to do... We might continue the boundaries along coastlines. This would be a job for a specialized BoundaryRelation class that inherits from MultiPolygonRelation.
I have finished my checks with Spain and the result is perplexing: The splitted tiles do not contain any boundary tag. I have used splitter r164. The .pbf file does contain them. So it seems to be a splitter bug (or I have used it wrong).
Ways that cross tile boundaries may be incomplete. When generating a tile, the splitter includes in each tile all of the nodes in the tile, all of the nodes within OVERLAP of the tile, and then any ways that use any of those nodes and finally any relations that use those ways or relations. If a way denoting a boundary has widely spaced nodes, where none occurs in the overlap region, the way will have a gap between the the last node in the region and the border. At present, it doesn't go back and 'fill in' missing nodes in those ways/relations, and it has never had that functionality. Doing that would require additional passes, but I think it could be added in a few days of work.
Scott
I was too quick with my guess that the splitter removes all boundary tags. It was my wrong search statement. Sorry! The real reason is, that multipolyong boundary definitions in Spain are tagged with type=boundary instead of type=multipolygon. Therefore the multipolygon algorithm does not handle them yet. @list: Should mkgmap handle type=boundary in the same way type=multipolygon? I vote for that. @Scott: I would be happy if there is a splitter option which ensures that all ways/relations are completely contained in a tile. That would improve the options we have to deal with the boundary information and would fix some multipolygon problems. WanMil
I think this is caused by addr: tags that don't use the same spelling like the boundary multipolygons. Do you know about any similar name detection algorithm? So something like a "sounds-like(String cityname)" function? This would be necessary to fix that.
I've used the soundex algorithm for similar problems in the past relatively successfully, however it's really only targetted at the English language. Another option might be to calculate the Hamming distance between the names and use some threshold to decide if they're equivalent or not. Wikipedia also has a page with some alternatives that might be of some use: http://en.wikipedia.org/wiki/Phonetic_algorithm
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag. The patch now patches the trunk and no longer the index branch. I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't. WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
El 03/03/11 20:40, WanMil escribió:
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't. My findings with your v3 patch: 1-Streets are now correctly assigned to cities, instead of suburbs or nearest (wrong) cities. Great! 2-State/Province field now includes regions and provinces, not only regions as with v2 of your patch. Spanish regions (admin_level=4 according to [1]) are formed by one or generally more than one province (admin_level=6), so every province is part of a region. According to that, a place found in a province should also be found in the matching region, but this is not the case; some places are assigned to provinces and others are assigned to regions. 3-A new country has appeared in the Mediterranean coast of Spain, called Maresme;Barcelona;Catalunya;España (coming from node 418639079 or way 37229141). Another buggy new country is Vega de Valcarce;León;Castilla y León;España,Europe (node 474558975) 4-"España" and "Europe" are found under State/Province. 5-The problem with State/Province info not taken into account when the country field is filled in any of the search tabs remains.
[1] http://wiki.openstreetmap.org/wiki/Key:admin_level#admin_level
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't. My findings with your v3 patch: 1-Streets are now correctly assigned to cities, instead of suburbs or nearest (wrong) cities. Great!
:-) That was due to the change already committed in r1879.
2-State/Province field now includes regions and provinces, not only regions as with v2 of your patch. Spanish regions (admin_level=4 according to [1]) are formed by one or generally more than one province (admin_level=6), so every province is part of a region. According to that, a place found in a province should also be found in the matching region, but this is not the case; some places are assigned to provinces and others are assigned to regions.
Mmmh, I don't know how to fix that at the moment. I think I have only three attributes which I can set: city, region, country. The question is now, which information is assigned to which attribute. At the moment the rules are as follows: 1. Assign city with admin_level=8,9,10,6,7 (use the first available level) 2. Assign regions with admin_level=6,7,5,3,4 if city is found and admin_level=5,3,4 if the city is not assigned 3. Assign country with admin_level=2 or use the most used country in the tile I think these rule might be tuned but I fear that we will need some country specific rules in the end. This will be a mess with large config files....
3-A new country has appeared in the Mediterranean coast of Spain, called Maresme;Barcelona;Catalunya;España (coming from node 418639079 or way 37229141). Another buggy new country is Vega de Valcarce;León;Castilla y León;España,Europe (node 474558975)
The is_in tag contains "," and the Locator is not able to handle that. So this is a tagging error. But I think about changing the order of the rules how the is_in and the more specific is_in:* tags are used.
4-"España" and "Europe" are found under State/Province.
I think thats also due to Locator errors.
5-The problem with State/Province info not taken into account when the country field is filled in any of the search tabs remains.
I think that's not a problem of the autolocation patch but a problem of the MDR generation. Steve might have a look on it.
[1] http://wiki.openstreetmap.org/wiki/Key:admin_level#admin_level
Thanks a lot for your good response!! That's very helpful! I think this will be a good way to get all the location information right. WanMil
Hi Wanmil, I think you are doing a great job! I wonder if it is possible to either set/change some parameters in the style files or commandline regarding to which level streets are assigned to (because this is in each country different). For instance in the Netherlands all streets can be assigned to a place name in admin_level=10 (boundaries for settlements (woonplaatsen)). Maybe it's an idea that the user of mkgmap can set this option to a certain level, like --assign-streetname-to-adminlevel=10 or whatever (default can maybe level 8)? Cheers, Minko
El 05/03/11 18:17, WanMil escribió:
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
My findings with your v3 patch: 1-Streets are now correctly assigned to cities, instead of suburbs or nearest (wrong) cities. Great!
:-) That was due to the change already committed in r1879.
2-State/Province field now includes regions and provinces, not only regions as with v2 of your patch. Spanish regions (admin_level=4 according to [1]) are formed by one or generally more than one province (admin_level=6), so every province is part of a region. According to that, a place found in a province should also be found in the matching region, but this is not the case; some places are assigned to provinces and others are assigned to regions.
Mmmh, I don't know how to fix that at the moment. I think I have only three attributes which I can set: city, region, country. The question is now, which information is assigned to which attribute.
At the moment the rules are as follows: 1. Assign city with admin_level=8,9,10,6,7 (use the first available level) 2. Assign regions with admin_level=6,7,5,3,4 if city is found and admin_level=5,3,4 if the city is not assigned 3. Assign country with admin_level=2 or use the most used country in the tile
I think these rule might be tuned but I fear that we will need some country specific rules in the end. This will be a mess with large config files....
I apologize for talking only from my own country point of view, but this is the one I know how it works. With the above rules streets and POI's are incorrectly assigned in the case one municipality (admin_level=8) has more than one city/village/hamlet. Using information from landuse=residential polygons could help improve assignment.
3-A new country has appeared in the Mediterranean coast of Spain, called Maresme;Barcelona;Catalunya;España (coming from node 418639079 or way 37229141). Another buggy new country is Vega de Valcarce;León;Castilla y León;España,Europe (node 474558975)
The is_in tag contains "," and the Locator is not able to handle that. So this is a tagging error.
Fixed
But I think about changing the order of the rules how the is_in and the more specific is_in:* tags are used.
I don't know what current order is, but I think is_in:* tags should take precedence on is_in ones.
4-"España" and "Europe" are found under State/Province.
I think thats also due to Locator errors.
If they are gathered from is_in it's more than probable. I still have Metropolitan France and Gibraltar / United Kingdom in the list of countries. Shouldn't they be France and United Kingdom after 1881 commit?
5-The problem with State/Province info not taken into account when the country field is filled in any of the search tabs remains.
I think that's not a problem of the autolocation patch but a problem of the MDR generation. Steve might have a look on it.
[1] http://wiki.openstreetmap.org/wiki/Key:admin_level#admin_level
Thanks a lot for your good response!! That's very helpful! I think this will be a good way to get all the location information right.
It's a pleasure to collaborate developing such a great improvement.
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing. Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why. * Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?) Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
I implemented Minkos idea using the style file to set the city, region, zip and country names. How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11 If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode. The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' } mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ... mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ... mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' } These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed). This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule). Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?) The rules are not perfect up to now but please feel free to post some good rules for your country. WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country. I have changed my styles as follows,
mkgmap:region!=* & is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' } to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx, .ppt, .pptx, .mdb, mdbx Instale OpenOffice desde http://es.openoffice.org/programa/index.html OpenOffice es libre: se puede copiar, modificar y redistribuir libremente. Gratis y totalmente legal. OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas versiones.
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly. * The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files. A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" } I think your rules are ok. By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with? WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country. I have changed my styles as follows,
mkgmap:region!=* & is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
Can you provide your style-file? Thx Am 08.03.2011 um 23:07 schrieb WanMil <wmgcnfg@web.de>:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly. * The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country. I have changed my styles as follows,
mkgmap:region!=* & is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
> After the index branch has been developed so far that we can use the > results in MapSource (thanks to Steve!!) there is a big need for a > mechanism to fill missing location information. > > There has been a lot of discussion about that. It stopped because it > was > quite theoretical (from my point of view). Therefore I have > started to > implement a prototype osm hook that tries to complete the missing > location information. I think the is_in tag is quite problematic > therefore I decided to use the addr tags. > > Attached is a *very* early implementation that makes use of the > boundary > polygons. It checks all nodes and ways/polygons within all boundary > polygons and fills the missing addr tags. > > I think this is a better foundation for the discussion how we want to > fill the missing location information. > > Before flaming me about terrible code please be aware that: > * the patch is not optimized > * it's version 1, so far away from handling all situations / > countries > * it's a very early prototype > > I would be happy if you try it and make some proposals how it > could be > improved. > > Some improvements I already know and thinking about: > * The new hook should run after the multipolygon finishing and not > before > * The is_in tag should be handled > * boundary information propagation => First go through all boundaries > and add the information from higher levels to lower ones, i.e. add > addr:district to all city boundaries within that district > * Use landuse=residential information in combination with the place > nodes > * In case a boundary level is missing (which mostly will be the case > for > countries admin_level=2), select the addr information from all nodes > and > ways within a boundary by using the most used addr-tag value. (in > germany the addr:country=DE should be the most used) > > > Have fun! > WanMil >
<index_location_v6.patch> _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
It's the default style file and my changes are contained in the patch. WanMil
Can you provide your style-file?
Thx
Am 08.03.2011 um 23:07 schrieb WanMil<wmgcnfg@web.de>:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly. * The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=*& addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=*& is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=*& mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=*& is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=*& openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=*& mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=*& addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=*& mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country. I have changed my styles as follows,
mkgmap:region!=*& is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
> I have done some development and want you to share my current > findings. > > 1. The MapElement copy constructor seems to have a bug. The attributes > map which contains the city, region and country information is not > copied. From my point of view this is an important thing that > should be > fixed in the trunk also. > > 2. In the patch the Locator is disabled as much as it was possible for > me up to now. > > 3. I am using now separate tags (mkgmap:city, mkgmap:region etc.). > > I recommend to run this patch with location-autofill=-1 or 0 to see > how > the patch works and not how the old Locator works. > > WanMil > > >> After the index branch has been developed so far that we can use the >> results in MapSource (thanks to Steve!!) there is a big need for a >> mechanism to fill missing location information. >> >> There has been a lot of discussion about that. It stopped because it >> was >> quite theoretical (from my point of view). Therefore I have >> started to >> implement a prototype osm hook that tries to complete the missing >> location information. I think the is_in tag is quite problematic >> therefore I decided to use the addr tags. >> >> Attached is a *very* early implementation that makes use of the >> boundary >> polygons. It checks all nodes and ways/polygons within all boundary >> polygons and fills the missing addr tags. >> >> I think this is a better foundation for the discussion how we want to >> fill the missing location information. >> >> Before flaming me about terrible code please be aware that: >> * the patch is not optimized >> * it's version 1, so far away from handling all situations / >> countries >> * it's a very early prototype >> >> I would be happy if you try it and make some proposals how it >> could be >> improved. >> >> Some improvements I already know and thinking about: >> * The new hook should run after the multipolygon finishing and not >> before >> * The is_in tag should be handled >> * boundary information propagation => First go through all boundaries >> and add the information from higher levels to lower ones, i.e. add >> addr:district to all city boundaries within that district >> * Use landuse=residential information in combination with the place >> nodes >> * In case a boundary level is missing (which mostly will be the case >> for >> countries admin_level=2), select the addr information from all nodes >> and >> ways within a boundary by using the most used addr-tag value. (in >> germany the addr:country=DE should be the most used) >> >> >> Have fun! >> WanMil >>
<index_location_v6.patch> _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
El 08/03/11 23:07, WanMil escribió:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly. * The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with? I think that would be good, so that other people can give their opinion. I'll give a try to this new patch later today and send comments.
2011-03-09 WanMil:
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
Three letter not two? Most (all) addresses in Lithuania have addr:country=LT... And in JOSM list of values has two letter country codes: DE, US, GB, FR, IT... -- Tomas Straupis
Am 09.03.2011 12:51, schrieb Tomas Straupis:
2011-03-09 WanMil:
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
Three letter not two? Most (all) addresses in Lithuania have addr:country=LT... And in JOSM list of values has two letter country codes: DE, US, GB, FR, IT...
Yes, but the GARMIN country code is 3 letters. Don't know if we have a list somewhere. Chris
Am 09.03.2011 13:22, schrieb Chris66:
Am 09.03.2011 12:51, schrieb Tomas Straupis:
2011-03-09 WanMil:
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
Three letter not two? Most (all) addresses in Lithuania have addr:country=LT... And in JOSM list of values has two letter country codes: DE, US, GB, FR, IT...
Yes, but the GARMIN country code is 3 letters.
Don't know if we have a list somewhere.
Chris
Correct. The Locator.xml configuration uses the three letter code and I am using this configuration. If that really matters it would be possible to change that later on. WanMil
El 08/03/11 23:07, WanMil escribió:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly. In case they can help you improve the algorithm, here you have two examples of nodes wrongly assigned (according to my styles below): nodes 945168390 and 943005606, that are within boundary relation 348994 (admin_level=6) are assigned region=Extremadura (boundary relation 349050, admin_level=4). I can provide more examples if necessary. Extract of the styles: mkgmap:region!=* & is_in:province=* { set mkgmap:region='${is_in:province}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' } * The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files. With no changed on your mkgmap:country rules in styles I have (among others) Esp and España (ESP) in the list of countries. If I select Esp, a full list of places is found, but if I select España (ESP), nothing in found.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" } How can I send the echo output to a file? I tried java -jar mkgmap.jar options input_files > textfile with no success
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country. I have changed my styles as follows,
mkgmap:region!=* & is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
> After the index branch has been developed so far that we can use > the > results in MapSource (thanks to Steve!!) there is a big need for a > mechanism to fill missing location information. > > There has been a lot of discussion about that. It stopped > because it > was > quite theoretical (from my point of view). Therefore I have > started to > implement a prototype osm hook that tries to complete the missing > location information. I think the is_in tag is quite problematic > therefore I decided to use the addr tags. > > Attached is a *very* early implementation that makes use of the > boundary > polygons. It checks all nodes and ways/polygons within all boundary > polygons and fills the missing addr tags. > > I think this is a better foundation for the discussion how we > want to > fill the missing location information. > > Before flaming me about terrible code please be aware that: > * the patch is not optimized > * it's version 1, so far away from handling all situations / > countries > * it's a very early prototype > > I would be happy if you try it and make some proposals how it > could be > improved. > > Some improvements I already know and thinking about: > * The new hook should run after the multipolygon finishing and not > before > * The is_in tag should be handled > * boundary information propagation => First go through all > boundaries > and add the information from higher levels to lower ones, i.e. add > addr:district to all city boundaries within that district > * Use landuse=residential information in combination with the place > nodes > * In case a boundary level is missing (which mostly will be the > case > for > countries admin_level=2), select the addr information from all > nodes > and > ways within a boundary by using the most used addr-tag value. (in > germany the addr:country=DE should be the most used) > > > Have fun! > WanMil >
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx, .ppt, .pptx, .mdb, mdbx Instale OpenOffice desde http://es.openoffice.org/programa/index.html OpenOffice es libre: se puede copiar, modificar y redistribuir libremente. Gratis y totalmente legal. OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas versiones.
Thanks for jar file. Now to report my test results: I uploaded the map via Garmin MapInstall search for "Address" and "Intersections". I get a "No Data Available" Using: mkgmap-locator-r1892.jar Args.list: ode-page=1252 tdbfile latin1 country-abbr=PHL country-name=PHILIPPINES remove-short-arcs=5 route add-pois-to-areas family-id=639 family-name=OSM_PHIL overview-mapname=40000001 series-name=OSM_PHIL description=OSM_PHIL style-file=/home/maning/Downloads/osm/routable_garmin/git/osmphgps/styles/default generate-sea=multipolygon,extend-sea-sectors,close-gaps=1000 index make-poi-index adjust-turn-headings drive-on-right report-dead-ends ignore-maxspeeds link-pois-to-ways location-autofill=0 Some error reports: SEVERE (LocationHook): philippines.osm: Element lists created after 230 ms SEVERE (LocationHook): philippines.osm: Quadtree created after 23798 ms SEVERE (LocationHook): philippines.osm: Location admin_level=1 processed after 23815 ms SEVERE (LocationHook): philippines.osm: Location admin_level=2 processed after 23864 ms SEVERE (LocationHook): philippines.osm: Start assigning admin_level=2 after 23962 ms SEVERE (LocationHook): philippines.osm: Special admin_level=2 handling processed after 24627 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427393428 [boundary=administrative,admin_level=3,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=3 processed after 40069 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 61462633 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427387951 [boundary=administrative,admin_level=4,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 27751466 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=4 processed after 40335 ms SEVERE (LocationHook): philippines.osm: Location admin_level=5 processed after 40336 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 28045303 [mkgmap:admin_level6=Makati,boundary=administrative,admin_level=6,mkgmap:admin_level2=PHL,mkgmap:admin_level3=Metro Manila] SEVERE (LocationHook): philippines.osm: Location admin_level=6 processed after 42682 ms SEVERE (LocationHook): philippines.osm: Location admin_level=7 processed after 42683 ms SEVERE (LocationHook): philippines.osm: Location admin_level=8 processed after 42823 ms SEVERE (LocationHook): philippines.osm: Location admin_level=9 processed after 42836 ms SEVERE (LocationHook): philippines.osm: Location admin_level=10 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location admin_level=11 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location postal_code processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location hook finished in 44427 ms Next I tried using the default style and now I can search for streets via "Address". There must be something wrong with my own style Moving on, for areas with proper relation/multipolygon I get good results for streets within villages (admin_level=10 http://www.openstreetmap.org/browse/relation/371327) but I can't get streets within tows/cities (admin_level=6 http://www.openstreetmap.org/browse/relation/146949) One question, does your code split the road when it crosses another admin_level? 2011/3/10 Carlos Dávila <cdavilam@orangecorreo.es>:
El 08/03/11 23:07, WanMil escribió:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly.
In case they can help you improve the algorithm, here you have two examples of nodes wrongly assigned (according to my styles below): nodes 945168390 and 943005606, that are within boundary relation 348994 (admin_level=6) are assigned region=Extremadura (boundary relation 349050, admin_level=4). I can provide more examples if necessary. Extract of the styles: mkgmap:region!=* & is_in:province=* { set mkgmap:region='${is_in:province}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
With no changed on your mkgmap:country rules in styles I have (among others) Esp and España (ESP) in the list of countries. If I select Esp, a full list of places is found, but if I select España (ESP), nothing in found.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
How can I send the echo output to a file? I tried java -jar mkgmap.jar options input_files > textfile with no success
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=* & addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=* & is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=* & mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=* & is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=* & addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=* & mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country.
I have changed my styles as follows,
mkgmap:region!=* & is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=* & mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=* & mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=* & mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=* & mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx, .ppt, .pptx, .mdb, mdbx Instale OpenOffice desde http://es.openoffice.org/programa/index.html OpenOffice es libre: se puede copiar, modificar y redistribuir libremente. Gratis y totalmente legal. OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas versiones.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------
Am 10.03.2011 03:53, schrieb maning sambale:
Thanks for jar file. Now to report my test results: I uploaded the map via Garmin MapInstall search for "Address" and "Intersections". I get a "No Data Available"
Using: mkgmap-locator-r1892.jar
Args.list: ode-page=1252 tdbfile latin1 country-abbr=PHL country-name=PHILIPPINES remove-short-arcs=5 route add-pois-to-areas family-id=639 family-name=OSM_PHIL overview-mapname=40000001 series-name=OSM_PHIL description=OSM_PHIL style-file=/home/maning/Downloads/osm/routable_garmin/git/osmphgps/styles/default generate-sea=multipolygon,extend-sea-sectors,close-gaps=1000 index make-poi-index adjust-turn-headings drive-on-right report-dead-ends ignore-maxspeeds link-pois-to-ways location-autofill=0
Some error reports: SEVERE (LocationHook): philippines.osm: Element lists created after 230 ms SEVERE (LocationHook): philippines.osm: Quadtree created after 23798 ms SEVERE (LocationHook): philippines.osm: Location admin_level=1 processed after 23815 ms SEVERE (LocationHook): philippines.osm: Location admin_level=2 processed after 23864 ms SEVERE (LocationHook): philippines.osm: Start assigning admin_level=2 after 23962 ms SEVERE (LocationHook): philippines.osm: Special admin_level=2 handling processed after 24627 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427393428 [boundary=administrative,admin_level=3,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=3 processed after 40069 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 61462633 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427387951 [boundary=administrative,admin_level=4,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 27751466 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=4 processed after 40335 ms SEVERE (LocationHook): philippines.osm: Location admin_level=5 processed after 40336 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 28045303 [mkgmap:admin_level6=Makati,boundary=administrative,admin_level=6,mkgmap:admin_level2=PHL,mkgmap:admin_level3=Metro Manila] SEVERE (LocationHook): philippines.osm: Location admin_level=6 processed after 42682 ms SEVERE (LocationHook): philippines.osm: Location admin_level=7 processed after 42683 ms SEVERE (LocationHook): philippines.osm: Location admin_level=8 processed after 42823 ms SEVERE (LocationHook): philippines.osm: Location admin_level=9 processed after 42836 ms SEVERE (LocationHook): philippines.osm: Location admin_level=10 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location admin_level=11 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location postal_code processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location hook finished in 44427 ms
Next I tried using the default style and now I can search for streets via "Address". There must be something wrong with my own style Moving on, for areas with proper relation/multipolygon I get good results for streets within villages (admin_level=10 http://www.openstreetmap.org/browse/relation/371327) but I can't get streets within tows/cities (admin_level=6 http://www.openstreetmap.org/browse/relation/146949)
One question, does your code split the road when it crosses another admin_level?
2011/3/10 Carlos Dávila<cdavilam@orangecorreo.es>:
El 08/03/11 23:07, WanMil escribió:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly.
In case they can help you improve the algorithm, here you have two examples of nodes wrongly assigned (according to my styles below): nodes 945168390 and 943005606, that are within boundary relation 348994 (admin_level=6) are assigned region=Extremadura (boundary relation 349050, admin_level=4). I can provide more examples if necessary. Extract of the styles: mkgmap:region!=*& is_in:province=* { set mkgmap:region='${is_in:province}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
With no changed on your mkgmap:country rules in styles I have (among others) Esp and España (ESP) in the list of countries. If I select Esp, a full list of places is found, but if I select España (ESP), nothing in found.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
How can I send the echo output to a file? I tried java -jar mkgmap.jar options input_files> textfile with no success
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=*& addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=*& is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=*& mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=*& is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=*& openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=*& mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=*& addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=*& mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country.
I have changed my styles as follows,
mkgmap:region!=*& is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx, .ppt, .pptx, .mdb, mdbx Instale OpenOffice desde http://es.openoffice.org/programa/index.html OpenOffice es libre: se puede copiar, modificar y redistribuir libremente. Gratis y totalmente legal. OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas versiones.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Now *with* my answer: Your style file must contain rules which assign the country, region and city name. So something like this: mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' } These rules are contained in the default stlye of the branch. So you can copy them and modify as you like it. The warning "SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 27751466" tells you that there are boundaries without a name tag. They are useless for the index because a city without a name cannot be searched. The id is the way id, so you can fix that by modifying the way http://www.openstreetmap.org/browse/way/27751466 At the moment roads are not splitted on borders. This is on the TODO list. WanMil
Thanks for jar file. Now to report my test results: I uploaded the map via Garmin MapInstall search for "Address" and "Intersections". I get a "No Data Available"
Using: mkgmap-locator-r1892.jar
Args.list: ode-page=1252 tdbfile latin1 country-abbr=PHL country-name=PHILIPPINES remove-short-arcs=5 route add-pois-to-areas family-id=639 family-name=OSM_PHIL overview-mapname=40000001 series-name=OSM_PHIL description=OSM_PHIL style-file=/home/maning/Downloads/osm/routable_garmin/git/osmphgps/styles/default generate-sea=multipolygon,extend-sea-sectors,close-gaps=1000 index make-poi-index adjust-turn-headings drive-on-right report-dead-ends ignore-maxspeeds link-pois-to-ways location-autofill=0
Some error reports: SEVERE (LocationHook): philippines.osm: Element lists created after 230 ms SEVERE (LocationHook): philippines.osm: Quadtree created after 23798 ms SEVERE (LocationHook): philippines.osm: Location admin_level=1 processed after 23815 ms SEVERE (LocationHook): philippines.osm: Location admin_level=2 processed after 23864 ms SEVERE (LocationHook): philippines.osm: Start assigning admin_level=2 after 23962 ms SEVERE (LocationHook): philippines.osm: Special admin_level=2 handling processed after 24627 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427393428 [boundary=administrative,admin_level=3,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=3 processed after 40069 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 61462633 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 4611686018427387951 [boundary=administrative,admin_level=4,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 27751466 [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL] SEVERE (LocationHook): philippines.osm: Location admin_level=4 processed after 40335 ms SEVERE (LocationHook): philippines.osm: Location admin_level=5 processed after 40336 ms SEVERE (LocationHook): philippines.osm: Cannot process location element because it contains no name tag: 28045303 [mkgmap:admin_level6=Makati,boundary=administrative,admin_level=6,mkgmap:admin_level2=PHL,mkgmap:admin_level3=Metro Manila] SEVERE (LocationHook): philippines.osm: Location admin_level=6 processed after 42682 ms SEVERE (LocationHook): philippines.osm: Location admin_level=7 processed after 42683 ms SEVERE (LocationHook): philippines.osm: Location admin_level=8 processed after 42823 ms SEVERE (LocationHook): philippines.osm: Location admin_level=9 processed after 42836 ms SEVERE (LocationHook): philippines.osm: Location admin_level=10 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location admin_level=11 processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location postal_code processed after 44427 ms SEVERE (LocationHook): philippines.osm: Location hook finished in 44427 ms
Next I tried using the default style and now I can search for streets via "Address". There must be something wrong with my own style Moving on, for areas with proper relation/multipolygon I get good results for streets within villages (admin_level=10 http://www.openstreetmap.org/browse/relation/371327) but I can't get streets within tows/cities (admin_level=6 http://www.openstreetmap.org/browse/relation/146949)
One question, does your code split the road when it crosses another admin_level?
2011/3/10 Carlos Dávila<cdavilam@orangecorreo.es>:
El 08/03/11 23:07, WanMil escribió:
Next round of the location improvements: * The algorithm that searched which elements were contained within a boundary was (is) wrong. I updated some parameters in the Quadtree so I the probability is very much lower that an element is not assigned correctly.
In case they can help you improve the algorithm, here you have two examples of nodes wrongly assigned (according to my styles below): nodes 945168390 and 943005606, that are within boundary relation 348994 (admin_level=6) are assigned region=Extremadura (boundary relation 349050, admin_level=4). I can provide more examples if necessary. Extract of the styles: mkgmap:region!=*& is_in:province=* { set mkgmap:region='${is_in:province}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
* The mkgmap:country, addr:country and is_in:country tag is now always assigned with the three letter country code. This makes it possible to define country specific rules in the style files.
With no changed on your mkgmap:country rules in styles I have (among others) Esp and España (ESP) in the list of countries. If I select Esp, a full list of places is found, but if I select España (ESP), nothing in found.
A note ot Carlos: With this patch you can add debug rules to the style file. So if you don't understand why a specific region/country is used add the following line to your style files: mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3} 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6} is_in=${is_in:region}" }
How can I send the echo output to a file? I tried java -jar mkgmap.jar options input_files> textfile with no success
I think your rules are ok.
By the way: Should I create a branch for my changes? Maybe this makes it easier for more people to test and to play with?
WanMil
El 06/03/11 19:40, WanMil escribió:
I implemented Minkos idea using the style file to set the city, region, zip and country names.
How does it work? The new AddressHook adds special tags to each element that is inside a known boundary: mkgmap:admin_level2 mkgmap:admin_level3 mkgmap:admin_level4 .. mkgmap:admin_level11
If an admin_level is not known the relevant tag is not set. The same for the zip tag mkgmap:postalcode.
The style file contains a new rule block (in each file): mkgmap:country!=*& addr:country=* { set mkgmap:country='${addr:country}' } mkgmap:country!=*& is_in:country=* { set mkgmap:country='${is_in:country}' } mkgmap:country!=*& mkgmap:admin_level2=* { set mkgmap:country='${mkgmap:admin_level2}' }
mkgmap:region!=*& is_in:county=* { set mkgmap:region='${is_in:county}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } ...
mkgmap:city!=*& openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } mkgmap:city!=*& mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } ...
mkgmap:postal_code!=*& addr:postcode=* { set mkgmap:postal_code='${addr:postcode}' } mkgmap:postal_code!=*& mkgmap:postcode=* { set mkgmap:postal_code='${mkgmap:postalcode}' }
These rules set the mkgmap tags: mkgmap:country mkgmap:region mkgmap:city mkgmap:postal_code
Only these tags are used to set the country, the region, the city and the zip code. No more rules are applied automatically (there are still some in the Locator class but they will be removed).
This enables you to configure which admin_level information you want to use for the city name, the region names etc. Also country specific rules are possible (e.g. add a mkgmap:country=NLD rule).
Todo: The country normalization must be performed by the AddressHook so that one can be sure that the Netherlands are always tagged with mkgmap:country=NLD (or mkgmap:country=Nederland?)
The rules are not perfect up to now but please feel free to post some good rules for your country.
I have changed my styles as follows,
mkgmap:region!=*& is_in:region=* { set mkgmap:region='${is_in:region}' } mkgmap:region!=*& mkgmap:admin_level4=* { set mkgmap:region='${mkgmap:admin_level4}' } mkgmap:region!=*& mkgmap:admin_level6=* { set mkgmap:region='${mkgmap:admin_level6}' } mkgmap:region!=*& mkgmap:admin_level5=* { set mkgmap:region='${mkgmap:admin_level5}' } mkgmap:region!=*& mkgmap:admin_level3=* { set mkgmap:region='${mkgmap:admin_level3}' }
to adapt them to what I think is best for Spain, but, if I understood well, something is not working as expected: places tagged as villages that have no is_in:region information and are within admin_level=4 and admin_level=6 polygons take the region from the a_l=6 polygon, instead of a_l=4 as it should be from the style above.
WanMil
Attached is the next step for the automatic location completion: * Some speed improvements (it's still slow but not as slow as before ;-) * boundary=postal_code is now supported. * The Locator does no longer overwrite already set attributes. This seems to be a better solution. I think the Locator must be rewritten completely. Up to now I have commented some lines only without deep understanding what it is really doing.
Some things that don't work: * Lot's of POIs are assigned with the default country although the MapElements are assigned with the correct country. Example: - Spain splitted with --overlap=3000 (more overlap is better for my patch) - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is correctly assigned with country=ESP but in the POI search it is displayed with the default country. I have no idea why.
* Zips are only applied to POIs. Streets seems to be not yet supported by the MDR creation. (=> Steve?)
Things to do: * Speed improvement (At the moment the location search takes as much time as the rest of the map creation) * A better mapping of city, region to the admin-levels * Make use more OSM information (like residential areas, place=city POIs in residential areas etc.) * Clean up of the Locator class
WanMil
I have improved calculation of the country information. In case no country information is available the most used country in a tile is used for all elements without country tag.
The patch now patches the trunk and no longer the index branch.
I have observed that lots of POIs are assigned the default country although I they are assigned a different country. I have no clue where this happens. Maybe an index related problem? I was hoping that Steves commit today would fix that problem but it didn't.
WanMil
I have done some development and want you to share my current findings.
1. The MapElement copy constructor seems to have a bug. The attributes map which contains the city, region and country information is not copied. From my point of view this is an important thing that should be fixed in the trunk also.
2. In the patch the Locator is disabled as much as it was possible for me up to now.
3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
I recommend to run this patch with location-autofill=-1 or 0 to see how the patch works and not how the old Locator works.
WanMil
After the index branch has been developed so far that we can use the results in MapSource (thanks to Steve!!) there is a big need for a mechanism to fill missing location information.
There has been a lot of discussion about that. It stopped because it was quite theoretical (from my point of view). Therefore I have started to implement a prototype osm hook that tries to complete the missing location information. I think the is_in tag is quite problematic therefore I decided to use the addr tags.
Attached is a *very* early implementation that makes use of the boundary polygons. It checks all nodes and ways/polygons within all boundary polygons and fills the missing addr tags.
I think this is a better foundation for the discussion how we want to fill the missing location information.
Before flaming me about terrible code please be aware that: * the patch is not optimized * it's version 1, so far away from handling all situations / countries * it's a very early prototype
I would be happy if you try it and make some proposals how it could be improved.
Some improvements I already know and thinking about: * The new hook should run after the multipolygon finishing and not before * The is_in tag should be handled * boundary information propagation => First go through all boundaries and add the information from higher levels to lower ones, i.e. add addr:district to all city boundaries within that district * Use landuse=residential information in combination with the place nodes * In case a boundary level is missing (which mostly will be the case for countries admin_level=2), select the addr information from all nodes and ways within a boundary by using the most used addr-tag value. (in germany the addr:country=DE should be the most used)
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx, .ppt, .pptx, .mdb, mdbx Instale OpenOffice desde http://es.openoffice.org/programa/index.html OpenOffice es libre: se puede copiar, modificar y redistribuir libremente. Gratis y totalmente legal. OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas versiones.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I suggest adding the following two lines to the styles to improve assignment: mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='${openGeoDB:name}' } +mkgmap:city!=* & is_in:city=* { set mkgmap:city='${is_in:city}' } +mkgmap:city!=* & addr:city=* { set mkgmap:city='${addr:city}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' }
I've get the following error withh your two lines... Error in style: Error: (points:20): Stack size is 2 Could not open style null Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Am 11.03.2011 um 09:19 schrieb Carlos Dávila:
I suggest adding the following two lines to the styles to improve assignment: mkgmap:city!=* & openGeoDB:name=* { set mkgmap:city='$ {openGeoDB:name}' } +mkgmap:city!=* & is_in:city=* { set mkgmap:city='${is_in:city}' } +mkgmap:city!=* & addr:city=* { set mkgmap:city='${addr:city}' } mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' }
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
El 11/03/11 10:55, Martin escribió:
I've get the following error withh your two lines...
Error in style: Error: (points:20): Stack size is 2 Could not open style null Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 It works for me with no errors. Did you remove the preceding "+"?
Hej, yes, the '+' was the problem ;) Now I can search for streets within towns and cities, but sometimes, streets within the same area couldn't be found. For example: http://www.openstreetmap.org/?lat=48.014671&lon=7.818832&zoom=18&layers=M Falterweg, Am Lusbühl, Käferweg, Ameisenweg, Grillenweg, Maiackerweg, Hutweg could be found. Zikadenweg, Verlorener Weg, Am Hertweg could not be found, I see no difference between this streets. Any idea?! Greetings Martin Am 11.03.2011 um 12:19 schrieb Carlos Dávila:
El 11/03/11 10:55, Martin escribió:
I've get the following error withh your two lines...
Error in style: Error: (points:20): Stack size is 2 Could not open style null Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 Error in style: Error: (points:20): Stack size is 2 It works for me with no errors. Did you remove the preceding "+"?
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (11)
-
Carlos Dávila -
Chris Miller -
Chris66 -
Henning Scholland -
Johann Gail -
maning sambale -
Martin -
Minko -
Scott Crosby -
Tomas Straupis -
WanMil