Maps crashing Mapsource in big cities
To me it seems that either we do something wrong, or Mapsource cannot handle very detailed places. Try compiling Australia from Geofabrik (I used maxnodes=1300000 - so smaller than default 16...) with the following commandline: start /low /b /wait java -enableassertions -jar -Xmx4200M mkgmap.jar --max-jobs=2 --reduce-point-density=5.4 --reduce-point-density-polygon=4 --suppress-dead-end-nodes --index --delete-tags-file=deletetags --blacklist-tags-file=deletetags --adjust-turn-headings --ignore-maxspeeds --ignore-turn-restrictions --remove-short-arcs=4 --location-autofill=1 --description=openmtbmap_%abr%_%date% --route --country-abbr=%abr% --country-name=%country% --mapname=%mapid%0000 --family-id=%mapid% --product-id=1 --series-name=openmtbmap_%abr%_%date% --family-name=mtbmap_%abr%_%date% --tdbfile --overview-mapname=mapset --area-name=%country% d:\garmin\mkgmap_680\maps\%mapid%00%tile%.osm.gz At the latest on resolution=24 it crashes. If I use --generate-sea=no-mp;extend.... then it already crashes on resolution 22 (zoomlevel 700m). Simply center onto the city POI of Melbourne and zoom in. Same problem exists for Greece and Athens. (it's not related to the TYPfile - Mapsource also crashes without registered typfile). Reports I got indicate that the maps not only crash Mapsource but also BSOD Garmin GPS (though no damage done, except restart needed).
Hello Felix,
To me it seems that either we do something wrong, or Mapsource cannot handle very detailed places. Try compiling Australia from Geofabrik (I used maxnodes=1300000 - so smaller than default 16...) with the following commandline: start /low /b /wait java -enableassertions -jar -Xmx4200M mkgmap.jar --max-jobs=2 --reduce-point-density=5.4 --reduce-point-density-polygon=4 --suppress-dead-end-nodes --index --delete-tags-file=deletetags --blacklist-tags-file=deletetags --adjust-turn-headings --ignore-maxspeeds --ignore-turn-restrictions --remove-short-arcs=4 --location-autofill=1 --description=openmtbmap_%abr%_%date% --route --country-abbr=%abr% --country-name=%country% --mapname=%mapid%0000 --family-id=%mapid% --product-id=1 --series-name=openmtbmap_%abr%_%date% --family-name=mtbmap_%abr%_%date% --tdbfile --overview-mapname=mapset --area-name=%country% d:\garmin\mkgmap_680\maps\%mapid%00%tile%.osm.gz
At the latest on resolution=24 it crashes. If I use --generate-sea=no-mp;extend.... then it already crashes on resolution 22 (zoomlevel 700m). Simply center onto the city POI of Melbourne and zoom in. Same problem exists for Greece and Athens. (it's not related to the TYPfile - Mapsource also crashes without registered typfile). Reports I got indicate that the maps not only crash Mapsource but also BSOD Garmin GPS (though no damage done, except restart needed).
Could it be simply that the map has too much stuff in it and some limit in mapsource/gps is being exceeded. What happens if you use smaller tiles? Do the same problems occur? I don't understand why people make such big tiles, what's the benefit? Cheers, Mark
On 12.02.2010 12:50, Mark Burton wrote:
Hello Felix,
To me it seems that either we do something wrong, or Mapsource cannot handle very detailed places. Try compiling Australia from Geofabrik (I used maxnodes=1300000 - so smaller than default 16...) with the following commandline: start /low /b /wait java -enableassertions -jar -Xmx4200M mkgmap.jar --max-jobs=2 --reduce-point-density=5.4 --reduce-point-density-polygon=4 --suppress-dead-end-nodes --index --delete-tags-file=deletetags --blacklist-tags-file=deletetags --adjust-turn-headings --ignore-maxspeeds --ignore-turn-restrictions --remove-short-arcs=4 --location-autofill=1 --description=openmtbmap_%abr%_%date% --route --country-abbr=%abr% --country-name=%country% --mapname=%mapid%0000 --family-id=%mapid% --product-id=1 --series-name=openmtbmap_%abr%_%date% --family-name=mtbmap_%abr%_%date% --tdbfile --overview-mapname=mapset --area-name=%country% d:\garmin\mkgmap_680\maps\%mapid%00%tile%.osm.gz
At the latest on resolution=24 it crashes. If I use --generate-sea=no-mp;extend.... then it already crashes on resolution 22 (zoomlevel 700m). Simply center onto the city POI of Melbourne and zoom in. Same problem exists for Greece and Athens. (it's not related to the TYPfile - Mapsource also crashes without registered typfile). Reports I got indicate that the maps not only crash Mapsource but also BSOD Garmin GPS (though no damage done, except restart needed).
Could it be simply that the map has too much stuff in it and some limit in mapsource/gps is being exceeded.
What happens if you use smaller tiles? Do the same problems occur?
I don't understand why people make such big tiles, what's the benefit?
You're right. Decreasing Splitter maxnodes to 800.000 worked fine. 1.000.000 still crashed Mapsource. Maybe there are some limits that mkgmap is assuming higher than they should be. For the sake of safety I would decrease --maxnodes default to 400.000 - as only then Denmark/Netherlands are more or less safe without errors.
Cheers,
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Carlos Dávila wrote:
Mark Burton escribió:
I don't understand why people make such big tiles, what's the benefit?
I use as big tiles as I can to avoid inter tile routing problems. Is it justified? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
The same problem about crashing MapSource and RoadTrip has also been reported for my maps (e.g. eastern Australia, north and south of Brisbane). The mapbuilding script tries to make those maps as large as possible. Perhaps related is that I upped the max-nodes from 1.400.000 to 1.600.000 for the current map version. I'll reduce the max-nodes for the next update but, ideally, Mkgmap should complain and quit if there is too much information in the source data (This is certainly no complaint, ofcourse I know this all depends on the understanding of the Garmin format).
Hello Lambertus,
The same problem about crashing MapSource and RoadTrip has also been reported for my maps (e.g. eastern Australia, north and south of Brisbane). The mapbuilding script tries to make those maps as large as possible. Perhaps related is that I upped the max-nodes from 1.400.000 to 1.600.000 for the current map version.
I'll reduce the max-nodes for the next update but, ideally, Mkgmap should complain and quit if there is too much information in the source data (This is certainly no complaint, ofcourse I know this all depends on the understanding of the Garmin format).
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit. Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly. Cheers, Mark
On 12.02.2010 15:15, Mark Burton wrote:
Hello Lambertus,
The same problem about crashing MapSource and RoadTrip has also been reported for my maps (e.g. eastern Australia, north and south of Brisbane). The mapbuilding script tries to make those maps as large as possible. Perhaps related is that I upped the max-nodes from 1.400.000 to 1.600.000 for the current map version.
I'll reduce the max-nodes for the next update but, ideally, Mkgmap should complain and quit if there is too much information in the source data (This is certainly no complaint, ofcourse I know this all depends on the understanding of the Garmin format).
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit.
Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly.
The problem seems to me, that if a tile has more max-nodes, then a very dense city (like Athens, Melbourne, ...) in an otherwise rather empty area (huge tile) will crash. For Athens I had to go as low as 600.000, for Melbourne 800.000 was fine. 1.600.000 is definitely not only crashing a few tiles, but in my eyes around 10% of European tiles. A solution to use bigger max-nodes would be if resolution=23 were usable. Currently the rounding goes havoc if you use 23 as lowest resolution (and drop 24). In my eyes 23 would be easily enough for OSM data (which is largely less accurate anyhow).
FH> The problem seems to me, that if a tile has more max-nodes, then a FH> very FH> dense city (like Athens, Melbourne, ...) in an otherwise rather FH> empty FH> area (huge tile) will crash. For Athens I had to go as low as FH> 600.000, FH> for Melbourne 800.000 was fine. FH> 1.600.000 is definitely not only crashing a few tiles, but in my FH> eyes FH> around 10% of European tiles. One thing that is clear is that the limiting factor isn't really the number of nodes in a tile; more likely it's limited by the number (and type?) of ways or relations, or some combination thereof. This is a very difficult (expensive) calculation to perform however when trying to determine where the tile boundaries should fall, so the number of nodes is used as a fairly rough approximation instead. Of course even if there was an efficient way to subdivide based on way/relation counts, we still don't currently know what the rules should be that determine the limits for each tile. Chris
On 12 Feb 2010, at 7:17 , Chris Miller wrote:
FH> The problem seems to me, that if a tile has more max-nodes, then a FH> very FH> dense city (like Athens, Melbourne, ...) in an otherwise rather FH> empty FH> area (huge tile) will crash. For Athens I had to go as low as FH> 600.000, FH> for Melbourne 800.000 was fine. FH> 1.600.000 is definitely not only crashing a few tiles, but in my FH> eyes FH> around 10% of European tiles.
One thing that is clear is that the limiting factor isn't really the number of nodes in a tile; more likely it's limited by the number (and type?) of ways or relations, or some combination thereof. This is a very difficult (expensive) calculation to perform however when trying to determine where the tile boundaries should fall, so the number of nodes is used as a fairly rough approximation instead. Of course even if there was an efficient way to subdivide based on way/relation counts, we still don't currently know what the rules should be that determine the limits for each tile.
to make it even more complicated mkgmap can optimize ways with too many nodes. some imports like Massgis didn't reduce point density. node count in this places will be huge for no benefit in accuracy. optimizing ways can help a lot but how could splitter consider that mkgmap can throw away >50% of all nodes easily another problem in osm data is the number of unused nodes, mainly caused by server/network problems during imports.
Chris
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Mark Burton escribió:
Hello Lambertus,
The same problem about crashing MapSource and RoadTrip has also been reported for my maps (e.g. eastern Australia, north and south of Brisbane). The mapbuilding script tries to make those maps as large as possible. Perhaps related is that I upped the max-nodes from 1.400.000 to 1.600.000 for the current map version.
I'll reduce the max-nodes for the next update but, ideally, Mkgmap should complain and quit if there is too much information in the source data (This is certainly no complaint, ofcourse I know this all depends on the understanding of the Garmin format).
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit.
Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly. I split Spain (from Geofabrik) into 5 tiles using max-nodes=1800000. I didn't get any crash on MapSource neither any report of crashes from others using my maps. From splitter output: A total of 6.809.701 nodes, 480.959 ways and 10.262 relations were processed in 1 file Output osm.gz files grows up to 27.3 MB and resulting img files up to 22.8 MB Madrid, which is densely mapped (don't know in relation to Athens or Melbourne), is included in one tile of 26.7 MB (osm.gz) and 22.8 MB (img)
Mark Burton wrote:
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit.
Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly.
I don't know exactly if you were kinda mad at me writing this or if this is a language barrier thing, but please know that I have high respect your and others work on Mkgmap and Splitter. My writing certainly wasn't meant as critique. I'm truly sorry if it was felt that way. That said, is there a tool that gives the statistics that you're looking for from an OSM extract? I would certainly try to provide more useful feedback if possible.
Hi Lambertus,
Mark Burton wrote:
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit.
Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly.
I don't know exactly if you were kinda mad at me writing this or if this is a language barrier thing, but please know that I have high respect your and others work on Mkgmap and Splitter. My writing certainly wasn't meant as critique. I'm truly sorry if it was felt that way.
Don't panic, I'm not mad at all - sorry if that's what you thought. The point I was trying to make was simply that we can't add warnings about size limits being busted until we know what the limits are and that info really needs to come from the people who are trying to make big/complex maps.
That said, is there a tool that gives the statistics that you're looking for from an OSM extract? I would certainly try to provide more useful feedback if possible.
I do not know of such a tool but what we could do is get mkgmap to report a bunch of statistics (number of nodes/ways/relations, subdivisions, routing table sizes, etc.) and that could be correlated against good and bad maps. Cheers, Mark
Mark Burton (markb@ordern.com) wrote:
I do not know of such a tool but what we could do is get mkgmap to report a bunch of statistics (number of nodes/ways/relations, subdivisions, routing table sizes, etc.) and that could be correlated against good and bad maps.
If you could do this, then it would be relatively simple for us to stress test mkgmap against different tiles and so determine acceptable limits for these things relatively quickly. Maybe a --gripe switch? ;) -- Charlie
Mark Burton wrote:
Hi Lambertus,
Mark Burton wrote:
OK - you tell us what the limit is and we'll make sure that mkgmap gripes if you bust the limit.
Seriously, if you can provide any info as what works and what doesn't in terms of maps sizes, numbers of lines, polys, nodes, etc. that would be useful and we can code accordingly.
I don't know exactly if you were kinda mad at me writing this or if this is a language barrier thing, but please know that I have high respect your and others work on Mkgmap and Splitter. My writing certainly wasn't meant as critique. I'm truly sorry if it was felt that way.
Don't panic, I'm not mad at all - sorry if that's what you thought. The point I was trying to make was simply that we can't add warnings about size limits being busted until we know what the limits are and that info really needs to come from the people who are trying to make big/complex maps.
Great, I hoped so :-)
That said, is there a tool that gives the statistics that you're looking for from an OSM extract? I would certainly try to provide more useful feedback if possible.
I do not know of such a tool but what we could do is get mkgmap to report a bunch of statistics (number of nodes/ways/relations, subdivisions, routing table sizes, etc.) and that could be correlated against good and bad maps.
If that is thought to be useful then I can upload the statistics to the website for all tiles on each weekly update and do some extensive testing when problems are reported. For that I'd apreciate an option to save the statistics to a file named something like {tilenumber}.stat or whatever.
CD> I use as big tiles as I can to avoid inter tile routing problems. Is CD> it justified? Currently yes, there is some justification in doing that. The splitter doesn't currently handle all cases at the edge of tiles correctly, so in that sense it's a case of the fewer tile edges the better so the impact of this is minimised. Chris
CD> I use as big tiles as I can to avoid inter tile routing problems. Is CD> it justified?
Currently yes, there is some justification in doing that. The splitter doesn't currently handle all cases at the edge of tiles correctly, so in that sense it's a case of the fewer tile edges the better so the impact of this is minimised.
Not sure what Chris is referring to there. I am aware of 2 issues related to inter-tile routing: 1 - the inability of mapsource to route across more than 1 tile boundary. 2 - the problem where a route needs to go from one tile to another and then back to the original tile (this could be related to problem 1). Mark
MB> Not sure what Chris is referring to there. A couple of things: 1) The current approach where the splitter holds on to any nodes within a certain overlap outside each tile is slightly flawed. For example if a way crosses a tile edge, then if the nearest node in that way which falls outside the tile also falls outside the overlap area, the line segment crossing the tile edge will be lost and hence mkgmap will not be able to join the tiles seamlessly. However I'm yet to see much evidence that this is a problem in practice. 2) Relations that cross tile boundaries aren't preserved in their entirety. This has been discussed on the list recently, and is known to cause problems. Another limitation of the splitter is that it doesn't yet handle relations containing other relations. I'm not sure how much this matters as far as mkgmap is concerned.
participants (7)
-
Apollinaris Schoell -
Carlos Dávila -
charlie＠cferrero.net -
Chris Miller -
Felix Hartmann -
Lambertus -
Mark Burton