April 2011 - mkgmap-dev - The mkgmap lists

Overview map
by Steve Ratcliffe 19 Jun '12

19 Jun '12

Hi I have started to implement adding features into the overview map. You can find ready built jar files on the download page and in the 'overview' branch in subversion. It creates the overview map from the .img (as before) and so it is very quick to try out even on a set of tiles covering a whole country. There is no configuration yet, it just takes everything (lines and polygons) that is in the least detailed layer of the detailed maps and adds it into the overview. You will immediately notice a big problem in that the overview map shows up even when the detailed maps are being displayed, so you get ugly jagged lines across the top of the map. I remember this happening the last time we has an overview map and I wonder if anyone can remember if there was a solution then? I'm going to start by trying the (so-called) POI flags, as one of those has been noted to be set differently on detail and base maps before. ..Steve

5 15

License text
by Ralf Kleineisel 01 Jan '12

01 Jan '12

Is it possible to change the license info which is embedded in the IMG header? Now it reads "OSM Street map http://www.openstreetmap.org/ Map data licenced under Creative Commons Attribution ShareAlike 2.0 http://creativecommons.org/licenses/by-sa/2.0/", but this is not correct when making maps from other data sources. For example contour line layers made of SRTM data do not contain any OSM data.

6 14

Problems with latest splitter versions
by Michael Prinzing 12 Nov '11

12 Nov '11

Hi, I am getting the splitter from svn, compiling and running it unter Windows XP SP3 using JDK 1.6.0_25. There are 2 problems if I am using a splitter version newer than r161 (tryed until r167 so far). 1.) germany.osm.pbf from geofabrik cannot be split -------------------------------------------------- Invoking the splitter with java.exe -Xms256m -Xmx1536m -server -jar splitter.jar --resolution=14 --overlap=4000 --max-nodes=1200000 --max-areas=50 --geonames-file=cities1000.zip germany.osm.pbf The data is split into 76 areas, so there are 2 passes to go. First, everything looks fine, but then, at the and of the first pass, the splitter "falls asleep", the CPU load for the corresponding JVM drops to 0% and there ist no further progress (except the memory stats being printed periodically). The output looks like this (cut a lot of lines): cache= description= geonames-file=cities1000.zip legacy-mode=false mapid=63240001 max-areas=50 max-nodes=1200000 max-threads=1 (auto) mixed=false no-trim=false output-dir= overlap=4000 resolution=14 split-file= status-freq=120 write-kml= Elapsed time: 0s Memory: Current 247MB (1MB used, 246MB free) Max 1484MB Time started: Wed Mar 02 17:11:11 CET 2011 Map is being split for resolution 14: - area boundaries are aligned to 0x400 map units - areas are multiples of 0x800 map units wide and high Processing germany.osm.pbf Bounding box 5.864417 47.26543 15.050779999000001 55.14777 10.000.000 nodes processed... id=283576079 20.000.000 nodes processed... id=355168786 30.000.000 nodes processed... id=495365986 40.000.000 nodes processed... id=699656373 50.000.000 nodes processed... id=998291790 60.000.000 nodes processed... id=1127935431 1.000.000 ways processed... id=25049151 2.000.000 ways processed... id=28822644 Elapsed time: 1m 59s Memory: Current 247MB (20MB used, 227MB free) Max 1484MB 3.000.000 ways processed... id=34073254 4.000.000 ways processed... id=40593158 5.000.000 ways processed... id=49931588 6.000.000 ways processed... id=61188933 7.000.000 ways processed... id=83211109 8.000.000 ways processed... id=93106763 9.000.000 ways processed... id=101081600 100.000 ways processed... id=1124462 in 1 file Time: Wed Mar 02 17:14:00 CET 2011 Exact map coverage is (47.26541519165039,5.864403247833252) to (55.147762298583984,15.05077600479126) Trimmed and rounded map coverage is (47.28515625,5.8447265625) to (55.1513671875,15.029296875) Splitting nodes into areas containing a maximum of 1.200.000 nodes each... Area (47.5048828125,7.470703125) to (47.9443359375,8.7451171875) contains 663.137 nodes. DONE! [...] Area (53.8330078125,11.77734375) to (54.931640625,14.2822265625) contains 996.051 nodes. DONE! 76 areas: Area 63240001 covers (0x246800,0x4b000) to (0x24d800,0x4f800) DE-Dusseldorf [...] Area 63240076 covers (0x223000,0x63800) to (0x228800,0x70000) DE-Reutlingen Writing out split osm files Wed Mar 02 17:14:03 CET 2011 Processing 76 areas in 2 passes, 38 areas at a time Starting pass 1 of 2, processing 38 areas (63240001 to 63240038) Making SparseMultiMap Making SparseMultiMap Processing germany.osm.pbf Bounding box 5.864417 47.26543 15.050779999000001 55.14777 Making SparseMultiMap Making SparseMultiMap Making SparseMultiMap MAP occupancy: 832878 MAP occupancy: 158237 MAP occupancy: 8339 MAP occupancy: 546 MAP occupancy: 1652236 MAP occupancy: 326019 MAP occupancy: 19669 MAP occupancy: 2076 MAP occupancy: 2474023 MAP occupancy: 491531 MAP occupancy: 31585 MAP occupancy: 2861 MAP occupancy: 3272326 MAP occupancy: 668457 MAP occupancy: 53049 MAP occupancy: 6168 Elapsed time: 3m 59s Memory: Current 267MB (121MB used, 146MB free) Max 1484MB MAP occupancy: 4072409 MAP occupancy: 847909 MAP occupancy: 73212 MAP occupancy: 6470 MAP occupancy: 4870695 MAP occupancy: 1026785 MAP occupancy: 94770 MAP occupancy: 7750 MAP occupancy: 5673776 MAP occupancy: 1203597 MAP occupancy: 111531 MAP occupancy: 11096 10.000.000 nodes processed... id=283576079 [...] 60.000.000 nodes processed... id=1127935431 MAP occupancy: 37091591 MAP occupancy: 8145968 MAP occupancy: 704292 MAP occupancy: 58149 MAP occupancy: 37918126 MAP occupancy: 8301600 MAP occupancy: 722028 MAP occupancy: 58246 MAP occupancy: 38711472 MAP occupancy: 8485566 MAP occupancy: 744601 MAP occupancy: 58361 Writing ways Wed Mar 02 17:26:02 CET 2011 Making SparseMultiMap Making SparseMultiMap Making SparseMultiMap 1.000.000 ways processed... id=25049151 MAP occupancy: 798645 MAP occupancy: 184591 MAP occupancy: 15271 MAP occupancy: 1493 Elapsed time: 15m 59s Memory: Current 1484MB (705MB used, 779MB free) Max 1484MB 2.000.000 ways processed... id=28822644 [...] 7.000.000 ways processed... id=83211109 Elapsed time: 20m 2s Memory: Current 1484MB (928MB used, 556MB free) Max 1484MB MAP occupancy: 4795349 MAP occupancy: 1103602 MAP occupancy: 94312 MAP occupancy: 6736 MAP occupancy: 1 8.000.000 ways processed... id=93106763 9.000.000 ways processed... id=101081600 MAP occupancy: 5597978 MAP occupancy: 1283033 MAP occupancy: 110757 MAP occupancy: 8231 MAP occupancy: 1 Writing relations Wed Mar 02 17:32:01 CET 2011 100.000 ways processed... id=1124462 coords occupancy MAP occupancy: 39240504 MAP occupancy: 8605635 MAP occupancy: 752197 MAP occupancy: 58361 ways occupancy MAP occupancy: 5634593 MAP occupancy: 1292952 MAP occupancy: 111321 MAP occupancy: 8235 MAP occupancy: 1 Elapsed time: 22m 2s Memory: Current 1484MB (1144MB used, 340MB free) Max 1484MB Elapsed time: 24m 2s Memory: Current 1484MB (1144MB used, 340MB free) Max 1484MB Elapsed time: 26m 2s Memory: Current 1484MB (1144MB used, 340MB free) Max 1484MB Elapsed time: 28m 2s Memory: Current 1484MB (1144MB used, 340MB free) Max 1484MB Elapsed time: 30m 2s Memory: Current 1484MB (1144MB used, 340MB free) Max 1484MB At this point, nothing more happens, no second pass is done. 2.) File containing contour data cannot be processed ---------------------------------------------------- When generating my maps, I usually add data to draw contour lines. The file containing that data is available from http://osm.arndnet.de/contourdata.osm.gz (not my site!). It is in OSM format 0.5. When buildig a map, I am combining this data with the OSM data (please note that the contour data file contains node IDs starting at 2^31, so they collide with the IDs from "real" OSM data if used unchanged). But even when I am trying to split this file without additional OSM data, the same as above happens and the splitter stops at the end of the first and only pass: java.exe -Xms256m -Xmx1536m -server -jar splitter.jar --resolution=14 --overlap=4000 --max-nodes=1200000 --max-areas=50 --mixed --geonames-file=cities1000.zip contourdata.osm.gz cache= description= geonames-file=cities1000.zip legacy-mode=false mapid=63240001 max-areas=50 max-nodes=1200000 max-threads=1 (auto) mixed=true no-trim=false output-dir= overlap=4000 resolution=14 split-file= status-freq=120 write-kml= Elapsed time: 0s Memory: Current 247MB (1MB used, 246MB free) Max 1484MB Time started: Wed Mar 02 17:43:31 CET 2011 Map is being split for resolution 14: - area boundaries are aligned to 0x400 map units - areas are multiples of 0x800 map units wide and high Processing contourdata.osm.gz 2.500.000 nodes processed... 5.000.000 nodes processed... 7.500.000 nodes processed... 10.000.000 nodes processed... 12.500.000 nodes processed... 15.000.000 nodes processed... 17.500.000 nodes processed... in 1 file Time: Wed Mar 02 17:45:08 CET 2011 Exact map coverage is (47.2760796546936,5.877492427825928) to (55.02375841140747,14.999899864196777) Trimmed and rounded map coverage is (47.28515625,5.86669921875) to (55.0634765625,15.00732421875) Splitting nodes into areas containing a maximum of 1.200.000 nodes each... Area (47.5048828125,6.7236328125) to (49.1748046875,8.525390625) contains 958.841 nodes. DONE! [...] Area (52.822265625,10.17333984375) to (54.7119140625,14.47998046875) contains 885.962 nodes. DONE! 22 areas: Area 63240001 covers (0x23c000,0x54c00) to (0x247800,0x5ec00) DE-Koblenz [...] Area 63240022 covers (0x21c800,0x61000) to (0x22f800,0x6e800) DE-Stuttgart Writing out split osm files Wed Mar 02 17:45:10 CET 2011 Processing 22 areas in a single pass Starting pass 1 of 1, processing 22 areas (63240001 to 63240022) Making SparseMultiMap Making SparseMultiMap Processing contourdata.osm.gz Making SparseMultiMap Writing ways Wed Mar 02 17:45:14 CET 2011 Making SparseMultiMap Making SparseMultiMap Making SparseMultiMap Making SparseMultiMap Making SparseMultiMap Elapsed time: 1m 59s Memory: Current 1484MB (1090MB used, 394MB free) Max 1484MB MAP occupancy: 783578 MAP occupancy: 190519 MAP occupancy: 25903 MAP occupancy: 1545344 MAP occupancy: 394559 MAP occupancy: 60097 MAP occupancy: 2398068 MAP occupancy: 533001 MAP occupancy: 68931 2.500.000 nodes processed... MAP occupancy: 3315227 MAP occupancy: 615842 MAP occupancy: 68931 MAP occupancy: 4195362 MAP occupancy: 728752 MAP occupancy: 75886 5.000.000 nodes processed... [...] [...] Elapsed time: 7m 59s Memory: Current 1484MB (1197MB used, 287MB free) Max 1484MB MAP occupancy: 16438198 MAP occupancy: 3986436 MAP occupancy: 575366 MAP occupancy: 17300217 MAP occupancy: 4121087 MAP occupancy: 578696 17.500.000 nodes processed... coords occupancy MAP occupancy: 17896316 MAP occupancy: 4149555 MAP occupancy: 578696 ways occupancy MAP occupancy: 180407 MAP occupancy: 43178 MAP occupancy: 6700 MAP occupancy: 21 MAP occupancy: 1 Elapsed time: 9m 59s Memory: Current 1484MB (1013MB used, 471MB free) Max 1484MB Elapsed time: 11m 59s Memory: Current 1484MB (1014MB used, 470MB free) Max 1484MB Elapsed time: 13m 59s Memory: Current 1484MB (1014MB used, 470MB free) Max 1484MB Elapsed time: 15m 59s Memory: Current 1484MB (1014MB used, 470MB free) Max 1484MB ***** Full GC ***** Elapsed time: 18m 1s Memory: Current 1484MB (923MB used, 561MB free) Max 1484MB Elapsed time: 20m 1s Memory: Current 1484MB (924MB used, 560MB free) Max 1484MB Elapsed time: 22m 1s Memory: Current 1484MB (924MB used, 560MB free) Max 1484MB and so on. I've then converted the data to OSM format 0.6 and sorted it (using --migrate and --sort-0.6 for osmosis) and stored it as .osm.bz2 and .osm.pbf. When I am trying to split these files, I am getting an exception: java.exe -Xms256m -Xmx1536m -server -jar splitter.jar --resolution=14 --overlap=4000 --max-nodes=1200000 --max-areas=50 --geonames-file=cities1000.zip contourdata06.osm.bz2 cache= description= geonames-file=cities1000.zip legacy-mode=false mapid=63240001 max-areas=50 max-nodes=1200000 max-threads=1 (auto) mixed=false no-trim=false output-dir= overlap=4000 resolution=14 split-file= status-freq=120 write-kml= Elapsed time: 0s Memory: Current 247MB (1MB used, 246MB free) Max 1484MB Time started: Wed Mar 02 19:28:48 CET 2011 Map is being split for resolution 14: - area boundaries are aligned to 0x400 map units - areas are multiples of 0x800 map units wide and high Processing contourdata06.osm.bz2 2.500.000 nodes processed... 5.000.000 nodes processed... 7.500.000 nodes processed... Elapsed time: 2m 0s Memory: Current 247MB (81MB used, 166MB free) Max 1484MB 10.000.000 nodes processed... 12.500.000 nodes processed... 15.000.000 nodes processed... Elapsed time: 4m 0s Memory: Current 247MB (61MB used, 186MB free) Max 1484MB 17.500.000 nodes processed... in 1 file Time: Wed Mar 02 19:34:21 CET 2011 Exact map coverage is (47.2760796546936,5.877492427825928) to (55.02375841140747,14.999899864196777) Trimmed and rounded map coverage is (47.28515625,5.86669921875) to (55.0634765625,15.00732421875) Splitting nodes into areas containing a maximum of 1.200.000 nodes each... Area (47.5048828125,6.7236328125) to (49.1748046875,8.525390625) contains 958.841 nodes. DONE! [...] Area (52.822265625,10.17333984375) to (54.7119140625,14.47998046875) contains 885.962 nodes. DONE! 22 areas: Area 63240001 covers (0x23c000,0x54c00) to (0x247800,0x5ec00) DE-Koblenz [...] Area 63240022 covers (0x21c800,0x61000) to (0x22f800,0x6e800) DE-Stuttgart Writing out split osm files Wed Mar 02 19:34:24 CET 2011 Processing 22 areas in a single pass Starting pass 1 of 1, processing 22 areas (63240001 to 63240022) Making SparseMultiMap Making SparseMultiMap Processing contourdata06.osm.bz2 Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: Start index (-30655339) is negative at it.unimi.dsi.fastutil.Arrays.ensureFromTo(Arrays.java:45) at it.unimi.dsi.fastutil.objects.ObjectArrays.ensureFromTo(ObjectArrays.java:351) at it.unimi.dsi.fastutil.objects.ObjectArrays.fill(ObjectArrays.java:313) at it.unimi.dsi.fastutil.objects.ObjectArrayList.size(ObjectArrayList.java:308) at uk.me.parabola.splitter.SparseInt2ShortMapInline.resizeTo(SparseInt2ShortMapInline.java:96) at uk.me.parabola.splitter.SparseInt2ShortMapInline.put(SparseInt2ShortMapInline.java:123) at uk.me.parabola.splitter.SparseInt2ShortMultiMap$Inner.put(SparseInt2ShortMultiMap.java:81) at uk.me.parabola.splitter.SparseInt2ShortMultiMap.put(SparseInt2ShortMultiMap.java:31) at uk.me.parabola.splitter.SplitProcessor.writeNode(SplitProcessor.java:208) at uk.me.parabola.splitter.SplitProcessor.processNode(SplitProcessor.java:118) at uk.me.parabola.splitter.OSMParser.endElement(OSMParser.java:243) at uk.me.parabola.splitter.AbstractXppParser.parse(AbstractXppParser.java:57) at uk.me.parabola.splitter.Main.processMap(Main.java:399) at uk.me.parabola.splitter.Main.writeAreas(Main.java:355) at uk.me.parabola.splitter.Main.split(Main.java:188) at uk.me.parabola.splitter.Main.start(Main.java:116) at uk.me.parabola.splitter.Main.main(Main.java:105) Both files (the original file in OSM format 0.5 and the converted .osm.bz2 in OSM format 0.6) could be split without a problem using splitter r161. Any ideas what is going wrong there? Anything I could try? Michael

4 3

[PATCH] --ignore-builtin-relations
by Marko Mäkelä 29 Aug '11

29 Aug '11

There is an option to disable the processing of turn restriction relations (--ignore-turn-restrictions), but no option to disable the processing of multipolygon relations. The attached patch introduces --ignore-builtin-relations, which disables the built-in processing of all relations (currently, type={boundary,multipolygon,restiction}). This patch reduces the processing time of my simple map layers from 2.5 minutes to 2 minutes. OK to commit this? Marko

5 22

Amenity=shelter
by Peter Hendricks 02 Jul '11

02 Jul '11

Hi, This becomes Campground/RVPark with the default style. Surely, this is not intended? Cheers, Peter.

3 15

UK Contours
by Stuart Poulton 02 Jun '11

02 Jun '11

Hi, Has anyone got a quick recipe for STRM contours to garmin for the UK, preferably on Linux ? Cheers Stuart

5 6

Error on MapSource with r1919
by Carlos Dávila 30 May '11

30 May '11

MapSource crashes if I try to activate Spain map generated with r1919 throwing a message mentioning DSK_FIRSTIMAGESECTOR.CPP. The same input files compiled with r1916 work fine. Other maps like Morocco, Portugal, Canary Islands, all of them quite smaller, work fine with r1919.

5 36

Idea: map layers (multiple output tiles per input tile)
by Marko Mäkelä 25 May '11

25 May '11

I got an idea today: Make the style language support layers, and allow the user to specify which layers to generate and in which output files. This would have the benefit that you could generate all output layers from a map in a single run, reducing the parsing and multipolygon processing overhead, and possibly allowing more parallel processing. The default layer (say, layer 0) would always be generated, to keep compatibility with the current behaviour. If we extended the default style so that it would generate overlay lines for bridges and tunnels on a non-default layer, these lines would be omitted by default. Someone with a fancy TYP file could invoke mkgmap with special options to generate these overlay lines, either in the same output tile, or in a separate output tile, so that the map user can enable or disable bridges and tunnels by selecting map sets on the device. Another application would be generating one map family per route relation type, so that you could choose which routes you want to see on the device: ski, bicycle, bus, road network, etc. My current --style=routes lumps all types of route relations together, not very useful. Let me illustrate this rough idea with an example. The syntax is not settled yet; I would appreciate some feedback on this. I think that I should be able to implement this, if there is no serious opposition to this idea. Consider the default style with the following addition to the beginning of the lines file: waterway=* [layer 1] boundary=* [layer 2] By default, all waterways and boundaries would be omitted from the map, because they are not in the default layer. Here is an example of such a default invocation, in the mkgmap.args format: family-id: 42 mapname: 63240001 description: City input-file: 63240001.osm.gz mapname: 63240002 description: Suburb input-file: 63240002.osm.gz Now, let us generate separate map sets for layer 1 and 2: family-id: 42 mapname: 63240001 description: City layer1-family-id: 1 layer1-mapname: 73240001 layer1-description: City Waters layer2-family-id: 2 layer2-mapname: 83240001 layer2-description: City Boundaries input-file: 63240001.osm.gz mapname: 63240002 description: Suburb layer1-family-id: 1 layer1-mapname: 73240002 layer1-description: Suburb Waters layer2-family-id: 2 layer2-mapname: 83240002 layer2-description: Suburb Boundaries input-file: 63240002.osm.gz We could also merge multiple layers to a single output tile by specifying layer3-merge: 2 or even ask mkgmap to merge everything to the default output tile: layer1-merge: default (or 0) layer2-merge: default What do you think? This scheme is only useable when it is acceptable to use the same map tiles for all map sets. If you need smaller or larger tiles for certain map layers, you would need to invoke mkgmap once for every set of input tiles. Best regards, Marko

8 20

Re: [mkgmap-dev] Bad routing error here in Rome...
by Marco Certelli 21 May '11

21 May '11

Hello. I'll re-test the situation every week. Compiled italy.osm.bz2 from download.geofabrik.de 1) latest mkgmap + latest splitter: Routing Error in Mapsource and Nuvi 255 2) mkgmap 1188 + oldest splitter (51kB): No Error in Mapsource and No error in Nuvi 255 Ciao, Marco. --- Dom 31/1/10, Marco Certelli <marco_certelli(a)yahoo.it> ha scritto: > Da: Marco Certelli <marco_certelli(a)yahoo.it> > Oggetto: Re: [mkgmap-dev] Bad routing error here in Rome... > A: "Development list for mkgmap" <mkgmap-dev(a)lists.mkgmap.org.uk> > Data: Domenica 31 gennaio 2010, 16:43 > Hi Mark. > > No way. I reduced maxnode to 500000 (Italy is splitted in > 42 tiles) > > I tested latest splitter and latest mkgmap: exactly same > error as before in mapsource routing. > > Ciao, Marco. > > > --- Sab 30/1/10, Mark Burton <markb(a)ordern.com> > ha scritto: > > > Da: Mark Burton <markb(a)ordern.com> > > Oggetto: Re: [mkgmap-dev] Bad routing error here in > Rome... > > A: mkgmap-dev(a)lists.mkgmap.org.uk > > Data: Sabato 30 gennaio 2010, 17:00 > > > > Hi Marco, > > > > > Here is the part of the cmd script doing the > actual > > job: > > > > > > java -enableassertions -Xmx1000m -jar > > ..\bin\splitter.jar --mapid=66%FID%001 > --max-nodes=1000000 > > ..\OSM-Data\%osmfile% > > > > > > java -enableassertions -Xmx1000m -jar > > ..\bin\mkgmap.jar --code-page=1252 > > --country-name="%country%" --latin1 --family-id=%FID% > > --mapname=66%FID%001 --overview-mapname=66%FID%000 > --tdbfile > > --series-name="OSM-%country%" > --family-name="OpenStreetMap: > > %country%" --road-name-pois --add-pois-to-areas > > --no-poi-address --ignore-maxspeeds > --remove-short-arcs > > --preserve-element-order > > --style-file=..\bin\resources\styles\ --style=%style% > > --description="%country%" --route --net --gmapsupp -c > > template.args > > > > OK - thanks. > > > > Have you tried using smaller tile sizes? i.e. does the > same > > problem > > exist if you use, say, 500000 for max nodes? > > > > Also, does the map show any other problems in that > area > > apart from > > routing not working? > > > > Mark > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev(a)lists.mkgmap.org.uk > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > > > >

6 15

Non way element X in multipolygon Y
by Carlos Dávila 03 May '11

03 May '11

I get a lot of warnings of the form "Non way element X in multipolygon Y" (e.g. Non way element 247735163 in multipolygon 339577) in which X is a correct "admin_centre" node of a boundary multipolygon relation. Would it be possible to detect "admin_centre" tag when the MP is processed to avoid this warnings in the log?

5 5