I have created a new branch for the automatic locator changes. It can be downloaded from http://www.mkgmap.org.uk/snapshots/ WanMil
Thanks Wanmil, I tested a few tiles of the Benelux, and finally I found the street where I live in the correct city, and not in a neighbourhood village anymore. Congratulations, you have done a great job! :-) For the Netherlands, streetnames assign to admin_level=10 or else (if not specified) assign to admin_level=8 (municipality) works great: mkgmap:country=NLD & mkgmap:city!=* & mkgmap:admin_level10=* { set mkgmap:city='${mkgmap:admin_level10}' } mkgmap:country=NLD & mkgmap:city!=* & mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' } A warning that shows up a few times while compiling: "cannot process location element, because it contains no name tag" but I don't know if this is a big issue. Some test results on a Dakota: A city was assigned to the wrong Province (Amersfoort, Gelderland, NL) but the street in this city shows the correct location (Amersfoort, Utrecht, NL). Could this be caused by the fact that part of the province polygon of Utrecht was not complete in those tiles? Note that this city lies on the border of both provinces. As a result, if I look up the street and enter the city first, it finds the street within the city, but no results show up if I click on the streetname. If I skip the city name and enter the streetname directly, it shows the correct location. This is not a big issue, since I could skip the region values in the style files (and I don't care so much about provinces) but I mention it anyway. For my Benelux map I'd like to know what style definitions works best in other surrounding countries (Germany, Belgium, Luxemburg) Cheers, Minko
Hi Minko,
Thanks Wanmil,
I tested a few tiles of the Benelux, and finally I found the street where I live in the correct city, and not in a neighbourhood village anymore. Congratulations, you have done a great job! :-)
Good news!
For the Netherlands, streetnames assign to admin_level=10 or else (if not specified) assign to admin_level=8 (municipality) works great:
mkgmap:country=NLD& mkgmap:city!=*& mkgmap:admin_level10=* { set mkgmap:city='${mkgmap:admin_level10}' } mkgmap:country=NLD& mkgmap:city!=*& mkgmap:admin_level8=* { set mkgmap:city='${mkgmap:admin_level8}' }
Good news again!
A warning that shows up a few times while compiling: "cannot process location element, because it contains no name tag" but I don't know if this is a big issue.
This is a hint that there are boundaries without a name tag. I think the warning also contains the way ids, so you might fix them easily.
Some test results on a Dakota:
A city was assigned to the wrong Province (Amersfoort, Gelderland, NL) but the street in this city shows the correct location (Amersfoort, Utrecht, NL).
Could this be caused by the fact that part of the province polygon of Utrecht was not complete in those tiles? Note that this city lies on the border of both provinces.
Congratulations. You have won a virtual coffee to be the first one that reports this problem ;-) The current algorithm doesn't like elements that lie or crosses the border of a boundary. It is a bit unspecified which name is used in such a case. This has to be fixed.
As a result, if I look up the street and enter the city first, it finds the street within the city, but no results show up if I click on the streetname. If I skip the city name and enter the streetname directly, it shows the correct location.
Ok, I understand. So it is quite important that streets get the same city/region/country combination than the city itself. I will keep that in mind.
This is not a big issue, since I could skip the region values in the style files (and I don't care so much about provinces) but I mention it anyway.
For my Benelux map I'd like to know what style definitions works best in other surrounding countries (Germany, Belgium, Luxemburg)
Thanks a lot for testing! There will be some internal major changes next so it will take some time until the next commit. Up to now I put all elements into a QuadTree like structure and search this structure for each boundary. This takes a long time and so I will do it the other way round. Build up a search structure for the boundaries and do the search for each element. This should speed up the things and it will be easier to handle the border crossing problems. Have fun! WanMil
Cheers, Minko
HI, The branch seems to have issues with inner members of a relation. For example, it complains that way 34672931 does not have a name but it is the inner member of a named relation. Thanks, N.
Hi Nakor, the way http://www.openstreetmap.org/browse/way/34672931 is tagged wrong. It is an inner way of a relation which means it is *not* tagged with the relation tags. The way itself has no name tag and does not belong to any boundary relation as *outer* way. So in the end the branch complains about the way as boundary with admin_level=8 which does not have a name tag and therefore it is useless as location information. WanMil
HI,
The branch seems to have issues with inner members of a relation. For example, it complains that way 34672931 does not have a name but it is the inner member of a named relation.
Thanks,
N.
I have a big problem to continue the development on the locator branch. I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction. I have attached a picture which visualizes the problem a bit more. 1. Red and green are two relations with e.g. admin_level=4. The black rectangle shows the tile bounds. 2. The splitter does not put the complete relations into the tile. So the read boundary is contained partial only. 3. The multipolygon algorithm automatically closes the red boundary with best guess. 4. Now we have two different boundaries with admin_level=4 which cover exactly the same area. I think this will be a big source of complaints. So before continuing with the locator branch I think it will be more worthy that the splitter is able to put complete relations to a tile. Anyone here who wants to start with that? WanMil
I have created a new branch for the automatic locator changes. It can be downloaded from http://www.mkgmap.org.uk/snapshots/
WanMil
WanMil schrieb am 17.03.2011 21:12:
I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction.
Moin, just as an idea: How about collecting the is_in data of the objects and trying to use this data for recovering the missing boundary information? If you have some is_in data points inside of an area, then the is_in data should (mostly) match the boundary data. For an unclosed boundary this could be used for guessing, to which side of the boundary it is relating to. And for a boundary completely surrounding the tile, this could be used to provide the missing information. Gruss Torsten
WanMil schrieb am 17.03.2011 21:12:
I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction.
Moin,
just as an idea: How about collecting the is_in data of the objects and trying to use this data for recovering the missing boundary information?
If you have some is_in data points inside of an area, then the is_in data should (mostly) match the boundary data. For an unclosed boundary this could be used for guessing, to which side of the boundary it is relating to. And for a boundary completely surrounding the tile, this could be used to provide the missing information.
Gruss Torsten
For admin_level=2 it works. The problem starts with the other levels. The assignment between admin_level and is_in:xxx tag is somehow country specific. And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are. WanMil
And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are.
WanMil _______________________________________________
I currently really don't want to use the locator branch, due to it's speed. I need already about 12 hours to compute all maps for weekly updates and osm data alone growing is already a concern. Wouldn't it be enough to use no guessing and just input correct addresses (best with working housenumber search)? The searching for streets not addresses is anyhow kinda strange and seems much more like a workaround than a clean solution.
And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are.
WanMil _______________________________________________
I currently really don't want to use the locator branch, due to it's speed. I need already about 12 hours to compute all maps for weekly updates and osm data alone growing is already a concern. Wouldn't it be enough to use no guessing and just input correct addresses (best with working housenumber search)? The searching for streets not addresses is anyhow kinda strange and seems much more like a workaround than a clean solution.
Yes ... but how should mkgmap input the correct addresses? ;-) In an ideal world you are right. But OSM data is far (very far) away from complete address tagging. Beware that all POIs would have to contain a complete set of addr-tags (addr:country, addr:county addr:region(?), addr:city, addr:street, addr:housenumber). The same with the is_in tags for all streets that do not have any POIs attached until now. I havn't tested it but it should not be too complicated to make a statistic with osmosis and a dump file. Maybe the completeness of tagging will change within the next 2 years but we are implementing for now. WanMil
Stupid question... Why don't extract the boundaries with osmosis to a single file. mkgmap could use this file to complete the boundaries while rendering the maps... Martin Am 17.03.2011 um 22:57 schrieb WanMil:
And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are.
WanMil _______________________________________________
I currently really don't want to use the locator branch, due to it's speed. I need already about 12 hours to compute all maps for weekly updates and osm data alone growing is already a concern. Wouldn't it be enough to use no guessing and just input correct addresses (best with working housenumber search)? The searching for streets not addresses is anyhow kinda strange and seems much more like a workaround than a clean solution.
Yes ... but how should mkgmap input the correct addresses? ;-) In an ideal world you are right. But OSM data is far (very far) away from complete address tagging.
Beware that all POIs would have to contain a complete set of addr-tags (addr:country, addr:county addr:region(?), addr:city, addr:street, addr:housenumber). The same with the is_in tags for all streets that do not have any POIs attached until now. I havn't tested it but it should not be too complicated to make a statistic with osmosis and a dump file.
Maybe the completeness of tagging will change within the next 2 years but we are implementing for now.
WanMil _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Yes, we provide the same option for coastlines. So it's possible although there is the size restriction. Boundaries take around 3% of the complete OSM data (3% as osm.gz compared to osm.pbf). So think about creating a europe map from the 4.5 GB dump. The boundary data file would be around 3%*4.5GB(osm.pbf) = 135MB(osm.gz). So in the end your computer additionally needs as much main memory as you need to compile a 135MB big tile. The calculation is quite rough but the result is clear: the memory requirements grow quite much compared to the overall size of your map. WanMil
Stupid question... Why don't extract the boundaries with osmosis to a single file. mkgmap could use this file to complete the boundaries while rendering the maps...
Martin
Am 17.03.2011 um 22:57 schrieb WanMil:
And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are.
WanMil _______________________________________________
I currently really don't want to use the locator branch, due to it's speed. I need already about 12 hours to compute all maps for weekly updates and osm data alone growing is already a concern. Wouldn't it be enough to use no guessing and just input correct addresses (best with working housenumber search)? The searching for streets not addresses is anyhow kinda strange and seems much more like a workaround than a clean solution.
Yes ... but how should mkgmap input the correct addresses? ;-) In an ideal world you are right. But OSM data is far (very far) away from complete address tagging.
Beware that all POIs would have to contain a complete set of addr-tags (addr:country, addr:county addr:region(?), addr:city, addr:street, addr:housenumber). The same with the is_in tags for all streets that do not have any POIs attached until now. I havn't tested it but it should not be too complicated to make a statistic with osmosis and a dump file.
Maybe the completeness of tagging will change within the next 2 years but we are implementing for now.
WanMil
Am 18.03.2011 14:45, schrieb WanMil:
Yes, we provide the same option for coastlines. So it's possible although there is the size restriction. Boundaries take around 3% of the complete OSM data (3% as osm.gz compared to osm.pbf). So think about creating a europe map from the 4.5 GB dump. The boundary data file would be around 3%*4.5GB(osm.pbf) = 135MB(osm.gz). So in the end your computer additionally needs as much main memory as you need to compile a 135MB big tile.
The calculation is quite rough but the result is clear: the memory requirements grow quite much compared to the overall size of your map
Maybe it would be a good solution to change splitter. If one member of a relation with information about boundaries or other polygons is inside a tile, the whole relation should be in that tile. This will also fix problems with huge multipolygons. Henning
On 17.03.2011 22:57, WanMil wrote:
And another requirement is also hard to achieve: performance. The branch is not optimized very well yet. But I am sure that such searches for big areas from all items of a tile are quite expensive no matter how optimized they are.
WanMil _______________________________________________
I currently really don't want to use the locator branch, due to it's speed. I need already about 12 hours to compute all maps for weekly updates and osm data alone growing is already a concern. Wouldn't it be enough to use no guessing and just input correct addresses (best with working housenumber search)? The searching for streets not addresses is anyhow kinda strange and seems much more like a workaround than a clean solution.
Yes ... but how should mkgmap input the correct addresses? ;-) In an ideal world you are right. But OSM data is far (very far) away from complete address tagging.
Beware that all POIs would have to contain a complete set of addr-tags (addr:country, addr:county addr:region(?), addr:city, addr:street, addr:housenumber). The same with the is_in tags for all streets that do not have any POIs attached until now. I havn't tested it but it should not be too complicated to make a statistic with osmosis and a dump file.
Maybe the completeness of tagging will change within the next 2 years but we are implementing for now.
WanMil
Well if mkgmap supported it, that would be the best headstart! As already said, In Vienna I would say we have already 1/3 of all addresses with complete and correct information - on local gatherings we face problems on how to decide what to do with houses that have two streets/addresses on corners and their like. In Germany there were talks on the forum of people asking if we already have a major town address complete - result was for a few not far off. Also the consensus is clear that noone wants ise information on streets, cause streets often do not belong to one disctrict or town (sometimes either side of a street is a different district, and so on). So mkgmap should not prosper incorrect tagging, but the goal should be to support addresses correctly. A good example are non connected streets. When mkgmap got routing loads of streets were not connected, especially streets and hiking/cycling/walking ways. Now only few beginners do mistakes here and all editors got checks for this. By that time one could have argued (and I think a few people did) that mkgmap should try to connect streets intelligently - good we did not do this, as data got good enough fairly soon! I only would expect searching for streets where there are no addresses (e.g. hiking networks on tracks/paths cause usually there are no addresses). However one might argue that it would maybe better to include signposts that mark significant places of a route into the address search (and hence give them sufficient ise info). That behaviour would even be more of what one expects when searching for a route (start, finish, and major points inbetween - maybe guideposts).
Hi, I did a test of the whole Benelux area plus border regions, splitted from the europe extract, and found a lot of cities in the border areas that were assigned to the wrong country. So I don't agree that admin_level=2 is working fine. I also observed a lot of elements that were not processed because they did not contain a name tag, for example http://www.openstreetmap.org/browse/way/93135578. Of course it has no name, because it belongs to two place name relations: Ter Aar and Nieuwveen, http://www.openstreetmap.org/browse/relation/1354658 and http://www.openstreetmap.org/browse/relation/1354654 The fact that the complete relations are not completely within the tile can be a reason for this warning? Despite the warning, most of the streets in those areas seemed assigned to the correct place names btw. So it seems functioning and I'm happy with it, apart from the wrong country names. The mkgmap compiling time for the whole Benelux area grew from 1,5u to roughly 2 hours. WanMil wrote: For admin_level=2 it works. The problem starts with the other levels. The assignment between admin_level and is_in:xxx tag is somehow country specific.
Hi,
I did a test of the whole Benelux area plus border regions, splitted from the europe extract, and found a lot of cities in the border areas that were assigned to the wrong country. So I don't agree that admin_level=2 is working fine.
Ok, it would work in case the algorithm is slightly improved. But I am sure that this will work very well. The changes are not too complicated.
I also observed a lot of elements that were not processed because they did not contain a name tag, for example http://www.openstreetmap.org/browse/way/93135578. Of course it has no name, because it belongs to two place name relations: Ter Aar and Nieuwveen, http://www.openstreetmap.org/browse/relation/1354658 and http://www.openstreetmap.org/browse/relation/1354654 The fact that the complete relations are not completely within the tile can be a reason for this warning? Despite the warning, most of the streets in those areas seemed assigned to the correct place names btw. So it seems functioning and I'm happy with it, apart from the wrong country names. The mkgmap compiling time for the whole Benelux area grew from 1,5u to roughly 2 hours.
At a first reaction I would say that way 93135578 is tagged wrong. The boundary tags should be added to the relation only and not to the way itself. But now I am quite unsure if that doesn't uncover a problem in the multipolygon processing of mkgmap. Usually tags in the outer ways that are equal to relation tags are removed after multipolygon processing. So the way 93135578 should have no tags when the location handling is started. Maybe the cause is that the relations do not tag the ways with "outer". I have to check that. Can you send me your splitter areas? Which dump do you use? This will help me to analyze the problem. Thanks WanMil
WanMil wrote:
For admin_level=2 it works. The problem starts with the other levels. The assignment between admin_level and is_in:xxx tag is somehow country specific.
I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction. I think this is a fundamental issue with splitting and areas. Martin's suggestion about extracting boundaries and using them on the side makes sense, but I would say that the issue is about all large areas. I wonder if it's possible to evaluate areas (and things like boundaries which are semantically areas even if they are technically closed linear features) for overlaping with tiles, and to then replace the area by multiple areas projected onto tile boundaries.
Hi, stupid question: If the polygon points are sorted (i.e that area is always to the left side of the border; anti clockwise orientation ), then the multi polygon algorithm could know that solution 4 is incorrect. Stefan 2011/3/17 WanMil <wmgcnfg@web.de>
I have a big problem to continue the development on the locator branch.
I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction.
I have attached a picture which visualizes the problem a bit more. 1. Red and green are two relations with e.g. admin_level=4. The black rectangle shows the tile bounds. 2. The splitter does not put the complete relations into the tile. So the read boundary is contained partial only. 3. The multipolygon algorithm automatically closes the red boundary with best guess. 4. Now we have two different boundaries with admin_level=4 which cover exactly the same area.
I think this will be a big source of complaints. So before continuing with the locator branch I think it will be more worthy that the splitter is able to put complete relations to a tile. Anyone here who wants to start with that?
WanMil
I have created a new branch for the automatic locator changes.
It can be downloaded from http://www.mkgmap.org.uk/snapshots/
WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
in general they are not sorted. only coastline ways are directional. I don't think there is any algorithmic approach to solve this. the only solution is to keep the whole relation in splitter or close the gaps in splitter before throwing away the rest. On 18 Mar 2011, at 7:43 , Stefan Schunck wrote:
Hi,
stupid question: If the polygon points are sorted (i.e that area is always to the left side of the border; anti clockwise orientation ), then the multi polygon algorithm could know that solution 4 is incorrect.
Stefan
2011/3/17 WanMil <wmgcnfg@web.de> I have a big problem to continue the development on the locator branch.
I observed that a lot of admin_level boundaries (2,3,4,5 and sometimes 6,7,8,9,10,11) are not useable because they are not completely contained in the tile data. The polygons are closed automatically but this is only a good guess in which direction they have to be closed. The probability is quite high that they are closed in the wrong direction.
I have attached a picture which visualizes the problem a bit more. 1. Red and green are two relations with e.g. admin_level=4. The black rectangle shows the tile bounds. 2. The splitter does not put the complete relations into the tile. So the read boundary is contained partial only. 3. The multipolygon algorithm automatically closes the red boundary with best guess. 4. Now we have two different boundaries with admin_level=4 which cover exactly the same area.
I think this will be a big source of complaints. So before continuing with the locator branch I think it will be more worthy that the splitter is able to put complete relations to a tile. Anyone here who wants to start with that?
WanMil
I have created a new branch for the automatic locator changes. It can be downloaded from http://www.mkgmap.org.uk/snapshots/
WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
A solution to the polygons in general could be to have the splitter close the polygons properly at the boundary tiles instead of having mkgmap try to guess where the polygon should be closed. A step further would be to have splitter draw a polygon on the entire tile border for multipolygon completly surrounding the tile. This would for sure introduce "false" boundaries so those lines probably would need to be marked in a special way so that mkgmap knows to use them for determining which part of the tile is in which polygon but not drawing them. My 2c, N.
On Mon, Mar 21, 2011 at 10:16:43AM -0400, Nakor wrote:
A solution to the polygons in general could be to have the splitter close the polygons properly at the boundary tiles instead of having mkgmap try to guess where the polygon should be closed.
Right, but have the splitter close only those polygons which were closed in the first place.
A step further would be to have splitter draw a polygon on the entire tile border for multipolygon completly surrounding the tile.
Right. Possibly, have Osmosis close the polygons as well, so that the map extracts would be OK.
This would for sure introduce "false" boundaries so those lines probably would need to be marked in a special way so that mkgmap knows to use them for determining which part of the tile is in which polygon but not drawing them.
Right, but how would you identify only the false sections of the boundaries? Perhaps, by tagging the false nodes only? Currently, mkgmap does not join connected ways that are tagged with polygon tags. Marko
participants (11)
-
Apollinaris Schoell -
Felix Hartmann -
Greg Troxel -
Henning Scholland -
Marko Mäkelä -
Martin -
Minko -
Nakor -
Stefan Schunck -
Torsten Leistikow -
WanMil