Steve, can you do me a favour? I have uploaded both versions to http://files.mkgmap.org.uk/detail/46 (only files that differ and not the gmapsupp.img). Can you run the display tool and check if you can quickly find the difference? I would have to completely install the display tool and have to get warm with it... I think you are much quicker? ;-) Thanks! WanMil
Hello WanMil,
I can reproduce the problem with the attached files like this: 1) download saarland.osm.pbf from geofabrik created 04-Jan-2012 04:36 2) Use splitter r198 with --max-nodes=200000 3) compile r2160 with my identical_output.patch 4) execute mkgmap with the parms in test.bat 5) copy the output files to somewhwere else 6) execute mkgmap again with the same commands, verify that the new output files are identical to the copy 7) compile r2160 with my identical_output.patch + c:\TEMP\remove_unused_elements_v2.patch 8) execute mkgmap again with the same commands. I would again expect identical output, but I see this C:\temp\prove_patch>diff -qb ..\prove . Files ..\prove\63240003.img and .\63240003.img differ Files ..\prove\gmapsupp.img and .\gmapsupp.img differ Files ..\prove\mkgmap.jar and .\mkgmap.jar differ Files ..\prove\osmmap.tdb and .\osmmap.tdb differ Files ..\prove\osmmap_mdr.img and .\osmmap_mdr.img differ
Ciao, Gerd
Date: Wed, 4 Jan 2012 19:10:28 +0100 From: wmgcnfg@web.de To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] [PATCH v2] Performance improvement by removing unused elements before the style processing
Hi Gerd,
yes of course I am interested, Send me all information you have.
Thanks WanMil
Hi WanMil,
I tested the patch with my tiles for Saarland. I can confirm a reduction of processing time ~ 6% compared to r2160, and also a reduction for the peek value of heap memory usage (238M -> 210M), so that's both good! BUT I also see a difference in one of the seven output img files (plus the resulting gmapsupp.img), and I think this is not intended. Both files have the same size, but are different in many bytes. Let me know when I should send details, maybe the new result is better than the old ;-)
Ciao, Gerd
WanMil wrote
The 2nd patch fixes the remaining problems: 1. Ways without tags but referenced by relations are no longer removed (they might be tagged by the relation style file and could therefore appear in the map) 2. Intersection of ways with the tile bounding box is now checked instead of only checking that at least one point is contained in the bounding box. Only very few ways are affected by this but otherwise routing problems are possible.
The performance improvement seems to be good (measured with my
test map):
r2159: ~250s patched: ~225s
I also expect (although I haven't tested) that the max memory requirement of mkgmap is decreased. I think max mem is used when the style file is just processed. At this stage all raw OSM elements and all style file processed elements are kept in memory. With the patch the number of OSM elements is noticeably reduced.
WanMil
Hi,
this is another performance improvement:
Usually the mkgmap input tiles are larger than the processed bounding box (splitter parameter overlap). So there are much many elements which are processed but thrown away at a late step in mkgmap.
The patch tries to remove them much earlier before the style files are processed and before the LocationHook starts (which ignores them but that must also be calculated).
The patch contains one drawback: Ways which have all its points outside the bounding box of the tile but which cross the tile are also removed. If that's a point the patch must be improved.
Have fun! WanMil
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.638310.n2.nabble.com/PATCH-v1-Performance-improvement-by-removing... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev