April 2017 - mkgmap-dev - The mkgmap lists

Re: [mkgmap-dev] Routing problem after mkgmap r3757
by Alexandre Loss 04 Apr '17

04 Apr '17

Hi Gerd, Any news about this? Thaks. Alexandre 2017-03-25 8:14 GMT-03:00 Alexandre Loss <alexandre.loss(a)gmail.com>: > Hi Gerd, > > Some users of the maps that I compile daily in Brazil have started to > complain about routing problems on some routes. So we started investigating > and found the problem. > We did a regression test on each of the previous versions of mkgmap until > we figured out which version the problem started to occur from. > And we have accurately identified that the problem began in version 3757. > I mean, any img generated with version 3756 or lower routes normally; And > every img generated with 3757 or higher causes the routing problem. > > Another thing we noticed was that we could not simulate the problem in > MapSource or Basecamp. Only when installed on a GPS does the problem occur. > See pictures below: > > *mkgmap r3757: FAILS* > > [image: Imagem inline 1] > [image: Imagem inline 2] > > > *mkgmap r3756: OK* > > [image: Imagem inline 3] > [image: Imagem inline 4] > > > > So, knowing that the problem started after version 3757, I checked in SVN > what was the change from version 3756 to 3757 that probably caused the > problem. And everything indicates that the problem is in the* > LineSplitterFilter.doFilter* method that you changed in version 3757. > > I believe that if you return with the original method the problem will be > solved. However, if you changed it is because you had some reason. So that > is the analysis you need to do. > > If you need data for testing, I can give you a piece of the map, all the > parameters I use in splitter and mkgmap, and our custom style. > > I hope I have been clear enough and helped to solve this problem. I tried > to be as precise as possible and if you need more information, please, let > me know. > > Thanks. > > Alexandre Loss > > > > >

3 4

Commit r3878: one more small improvement to decease peak memory in index creation
by svn commit 03 Apr '17

03 Apr '17

Version mkgmap-r3878 was committed by gerd on Mon, 03 Apr 2017 one more small improvement to decease peak memory in index creation http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3878

1 0

Commit r3877: set flag SegmentsFollowing in the copy, not in the original way.
by svn commit 03 Apr '17

03 Apr '17

Version mkgmap-r3877 was committed by gerd on Mon, 03 Apr 2017 set flag SegmentsFollowing in the copy, not in the original way. http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3877

1 0

Commit r3876: further decrease peak memory and improve speed for global index creation
by svn commit 02 Apr '17

02 Apr '17

Version mkgmap-r3876 was committed by gerd on Sun, 02 Apr 2017 further decrease peak memory and improve speed for global index creation Correct the calculation of needed bytes for sort keys so that padding bytes are considered. Now copies are only created when the help to reduce memory. http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3876

3 2

max-nodes in splitter
by Jakob Mühldorfer 01 Apr '17

01 Apr '17

Hi, I tried to read all the docs and forums about max-nodes, but I still have questions left: * Does the splitter print an error when max-nodes is too large, or is it unpredictable and will it only fail on the GPS device * Do tiles with more nodes mean higher or lower draw performance on devices, or not matter at all Thank you for your help Jakob

5 19

Commit r3875: decrease peak memory and improve speed for global index creation
by svn commit 01 Apr '17

01 Apr '17

Version mkgmap-r3875 was committed by gerd on Sat, 01 Apr 2017 decrease peak memory and improve speed for global index creation - use memory cache were it helps and free memory for caches after SortKey are calculated - always free memory for mdr19 data - implement (undocumented) --x-mdr7-excl filter for street name index. Not very useful until user knows special names which should not appear in the street search. - Use new DoubleSortKey instead of MultiSortKey in Mdr7. - Don't compare sort keys byte by byte if both are the same, which is quite likely with the caches http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3875

1 0

AssertionError with custom style and --housenumbers
by Carlos Dávila 01 Apr '17

01 Apr '17

I'm getting the error below compiling a single tile with the following command: java -Xmx2500m -ea -jar mkgmap.jar --output-dir=./tmp --bounds=bounds.zip --precomp-sea=sea.zip --max-jobs --route --latin1 --code-page=1252 --index --housenumbers --style=truck 55139114.o5m java.lang.AssertionError at uk.me.parabola.imgfmt.app.net.RoadIndex.write(RoadIndex.java:45) at uk.me.parabola.imgfmt.app.net.RoadDef.writeLevelDivs(RoadDef.java:361) at uk.me.parabola.imgfmt.app.net.RoadDef.writeNet1(RoadDef.java:230) at uk.me.parabola.imgfmt.app.net.NETFile.write(NETFile.java:66) at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:263) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:107) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:69) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:263) at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:259) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Removing --housenumbers tile compiles fine. What is causing the error is the absence of the following rule in lines file: # Mop up any unrecognised highway types highway=* & area!=yes [0x07 road_class=0 road_speed=0 resolution 23] Input data: http://alternativaslibres.org/files/tmp/55139114.o5m

3 6

[Patch v3] reduce peak memory usage for global index
by Gerd Petermann 01 Apr '17

01 Apr '17

Hi all, the attached patch is a further optimization. Peak memory for --x-split-name-index is now ~ 57% of r3873, without it ~47 % compared to r3873. The run time is a bit better and the unit tests are passed again. I've removed the debug reports for now, so that this patch is ready for commit. A binary is here: http://files.mkgmap.org.uk/download/343/mkgmap.jar Worth to mention: The --x-split-name-index option requires a lot more memory in my test sample with only 51 tiles I see 250 MB instead of 101MB (437 instead of 214 with r3873), so the relative increase for the option is even higher with the patch. A lot of the memory is required to store so called sort keys. Those are created from the street name(s). A possible alternative would be to calculate the keys only only when required. This would increase run time for sorting but should keep memory much lower. I'll experiment with this later. Gerd

1 1

Commit r3874: correct PredictFilterPoints: make sure that preserved nodes are counted
by svn commit 01 Apr '17

01 Apr '17

Version mkgmap-r3874 was committed by gerd on Sat, 01 Apr 2017 correct PredictFilterPoints: make sure that preserved nodes are counted http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3874

1 0

[Patch v1] reduce peak memory usage for global index
by Gerd Petermann 01 Apr '17

01 Apr '17

Hi all, attached is a patch which seems to help without changing the output of mkgmap. Compared to r3873 I see ~ 33% less memory usage when used with --x-split-name-index and ~50% less without it. Runtimes were similar. Actual numbers depend on the style and the used options. A binary based on 3873 is here: http://files.mkgmap.org.uk/download/341/mkgmap.jar It consists of different changes, each of them maybe used independently: 1) always release memory for mdr19, not only if it was filled for the device. 2) check if sort key has zero length, if yes, don't allocate new buffer 3) don't use MultiSortKey for mdr7 if --x-split-name-index is not used 4) count occurences of the generated key strings, cache sort keys for those keys which occur more than 3 times. 5) create smaller copy of byte array if number of allocated bytes was too high @Steve: Please review, I think 1-4 are okay, 5) may cause problems if my understanding of Sort.fillkey() is wrong. I was also surprised to see that we require such long sort keys, even with --latin1. I see many 0 bytes in the created keys, maybe that can be optimized further? Gerd

2 2