- mkgmap-dev - The mkgmap lists

question about output-dir and indexing
by 7700 06 Oct '20

06 Oct '20

Hi. I am very new to making maps so therefor in case i am asking something obvious it is because i don't know better yet. I am trying to generate a map for a country (trying sweden) and observe a strange thing with --output-dir for mkgmap (r4585). Files are written to other directories than to the --output-dir Starting at a base directory i have created this structure: ├── gmapsupp └── se ├── splitted └── tiles The idea is that for sweden i will store splitted pbf files in a separate directory and tiles in a separate. at first i download data to the ./se/ directory: wget https://download.geofabrik.de/europe/sweden-latest.osm.pbf then split the data, adding the --output-dir=se/splitted/ this works fine. now i want to generate the *.img tiles from the splitted files. Doing this from the base directory. java -jar ../mkgmap/mkgmap-r4585/mkgmap.jar --read-config=se/splitted/template.args --remove-short-arcs --family-name="NORDIC" --country-name="Sweden" --country-abbr="SWE" --description="Sweden routable." --mapname=77700001 --family-id=770 --latin1 --net --route --index --split-name-index --output-dir=se/tiles/ se/splitted/*.osm.pbf When i run this from the base directory numbered *.img files and ovm_*.img files are first written to the base directory and then last to the target directory. After the program finishes there are files both in base directory from where i executed the program, as well as the directory given in output-dir. The files in the output-dir directory are larger than the ones in the base directory. Is this an error or should i take care of both sets of files? Trying to use --index to create a searchable index or street names, for this, do i need to add a separate file/style to make it work? if yes, are there examples which you can point to? Regards Karl

3 4

Fwd: Re: "Small" SRTM1 files and DEM maps
by Carlos Dávila 05 Oct '20

05 Oct '20

Hi Gerd I replied to your patch a couple of months ago, but I have now realized it didn't arrive to the list (it seems I used and old address). I was surprised you didn't say anything, but didn't want to bother... I used your binary for my test. Today I have tried to apply your patch, but it seems to be generated against a different version of files in trunk. -------- Mensaje reenviado -------- Asunto: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps Fecha: Tue, 21 Apr 2020 17:38:19 +0200 De: Carlos Dávila <cdavilam(a)orangecorreo.es> Para: mkgmap-dev(a)lists.mkgmap.org.uk Hi Gerd I have tested it in an area of 4x10 degrees, in the North of China, and it worked fine. El 21/4/20 a las 14:40, Gerd Petermann escribió: > Hi Carlos, > > please try the attached patch or this binary based on r4483: > http://files.mkgmap.org.uk/download/468/mkgmap.jar > I hope I found all the places that need to be changed. I've only > tested with a map around the point N50 E118. > > Gerd > > ________________________________________ > Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag > von Carlos Dávila <carlos(a)alternativaslibres.org> > Gesendet: Dienstag, 21. April 2020 11:27 > An: mkgmap-dev(a)lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps > > Great new!! Thanks for your support > > El 21/4/20 a las 11:19, Gerd Petermann escribió: >> Hi Carlos, >> >> yes, I see that the source code handles this format. I finally found >> https://dds.cr.usgs.gov/srtm/version2_1/Documentation/MIL-PDF-89020B.pdf >> I think "TABLE I. Matrix intervals for DTED Level 0." >> shows how to interpret the data. It seems there are a few more >> variants... >> For now I'll try to support also the 1801 * 3601 * 2 format >> >> Gerd >> >> ________________________________________ >> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >> von Carlos Dávila <carlos(a)alternativaslibres.org> >> Gesendet: Donnerstag, 16. April 2020 17:41 >> An: mkgmap-dev(a)lists.mkgmap.org.uk >> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >> >> HI Gerd >> I don't think conversion is wrong. If input geotiff file is 24MB, output >> hgt file is 24 MB, if input tiff is 12MB, output hgt is 12MB >> I use this command on Linux: >> for i in *.tif ; do gdal_translate -strict -q -eco -of SRTMHGT $i >> `basename $i tif`hgt ; done ; rm -f *.hgt.aux.xml >> BTW, gdal is also able to deal with small SRTM files >> >> El 16/4/20 a las 11:31, Gerd Petermann escribió: >>> Hi Carlos, >>> >>> so your conversion routine is wrong. The file size is 12.970.802 >>> bytes. There is no integer n for which 2 * n^2 results in this value. >>> >>> Gerd >>> >>> ________________________________________ >>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >>> von Carlos Dávila <cdavilam(a)orangecorreo.es> >>> Gesendet: Donnerstag, 16. April 2020 11:19 >>> An: mkgmap-dev(a)lists.mkgmap.org.uk >>> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>> >>> Hi Gerd >>> Forgot to mention files are downloaded as geotiff, but I then convert >>> them to hgt, sorry. >>> You can download a sample one from >>> https://alternativaslibres.org/tmp/N50E118.hgt >>> >>> El 16/4/20 a las 9:18, Gerd Petermann escribió: >>>> Hi Carlos, >>>> >>>> are you talking about geotiff format? I think someone documented >>>> how that format can be converted to *.hgt. >>>> Can you post a link to such a file? >>>> >>>> Gerd >>>> >>>> >>>> >>>> ________________________________________ >>>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >>>> von Carlos Dávila <cdavilam(a)orangecorreo.es> >>>> Gesendet: Montag, 13. April 2020 14:08 >>>> An: mkgmap-dev(a)lists.mkgmap.org.uk >>>> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>>> >>>> Hi Gerd >>>> They are downloaded by phyghtmap. This is what I found digging into >>>> phyghtmap files: >>>> |def getSRTMFileServer(self, resolution, srtmVersion):|| >>>> || if srtmVersion == 2.1:|| >>>> || return >>>> "https://dds.cr.usgs.gov/srtm/version2_1/SRTM{0:d}".format(resolution)|| >>>> || elif srtmVersion == 3.0:|| >>>> || if resolution == 1:|| >>>> || urlRe = >>>> "https://earthexplorer.usgs.gov/download/8360/SRTM1{:s}V3/GEOTIFF/EE"|| >>>> || elif resolution == 3:|| >>>> || urlRe = >>>> "https://earthexplorer.usgs.gov/download/4960/SRTM3{:s}V2/GEOTIFF3/EE"|| >>>> || return urlRe| >>>> They should come from the second URL above. >>>> >>>> El 13/4/20 a las 8:24, Gerd Petermann escribió: >>>>> Hi Carlos, >>>>> >>>>> sorry for the late response. I found no hint on the phyghtmap page >>>>> http://katze.tfiu.de/projects/phyghtmap/ >>>>> regarding a new format for srtm data. The hgt format is not >>>>> compressed. Do you know from where these 12.4 MB files came from? >>>>> >>>>> Gerd >>>>> >>>>> ________________________________________ >>>>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im >>>>> Auftrag von Carlos Dávila <cdavilam(a)orangecorreo.es> >>>>> Gesendet: Samstag, 11. April 2020 17:09 >>>>> An: Development list for mkgmap >>>>> Betreff: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>>>> >>>>> I use phyghtmap to build contour lines and also to download hgt data >>>>> that I then use to build DEM maps with mkgmap. For a long time, SRTM1 >>>>> files (and also VFP1) have all been ~24.7 MB in size, but lately (I >>>>> don't remember since when) SRTM1 are also served as 12.4 MB files. >>>>> phyghtmap is able to generate contour lines from those small files, as >>>>> they seem to contain the same information but with a different >>>>> compression ratio, but mkgmap fails to read them, and report them as >>>>> missing. Would it be possible to get support for such files in mkgmap? >>>>> _______________________________________________ >>>>> mkgmap-dev mailing list >>>>> mkgmap-dev(a)lists.mkgmap.org.uk >>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >>>>> _______________________________________________ >

2 2

Commit r4586: support HGT files with 2 * 1801 * 3601 = 12.970.802 bytes
by svn commit 05 Oct '20

05 Oct '20

Version mkgmap-r4586 was committed by gerd on Mon, 05 Oct 2020 support HGT files with 2 * 1801 * 3601 = 12.970.802 bytes 1801x3601-hgt.patch published in April 2020, got lost somehow http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4586

1 0

DEM file Number too big for binary encoding with 14 bits
by Joao Almeida 24 Sep '20

24 Sep '20

Hi, Any idea how I can diagnose this issue? SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301057.o5m: Attention: Tile contains both drive-on-left (81) and drive-on-right roads (1031) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301054.o5m: Attention: Tile contains both drive-on-left (4817) and drive-on-right roads (4982) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301053.o5m: Attention: Tile contains both drive-on-left (3133) and drive-on-right roads (1255) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301052.o5m: Attention: Tile contains both drive-on-left (5668) and drive-on-right roads (603) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301061.o5m: Attention: Tile contains both drive-on-left (2381) and drive-on-right roads (81) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301065.o5m: Attention: Tile contains both drive-on-left (745) and drive-on-right roads (18) SEVERE (MapFailedException): C:\Garmin\SPLITT~4\67301053.o5m: (thrown in DEMTile.writeValAsBin()) Number too big for binary encoding with 14 bits:32559 SEVERE (MapBuilder): C:\Garmin\SPLITT~4\67301053.o5m: exception while creating DEM file Number too big for binary encoding with 14 bits:32559 SEVERE (MapFailedException): C:\Garmin\SPLITT~4\67301053.o5m: (thrown in MapBuilder.buildDem()) DEM SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301069.o5m: Attention: Tile contains both drive-on-left (4648) and drive-on-right roads (227) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301070.o5m: Attention: Tile contains both drive-on-left (6416) and drive-on-right roads (2) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301097.o5m: Attention: Tile contains both drive-on-left (19) and drive-on-right roads (2) SEVERE (StyledConverter): C:\Garmin\SPLITT~4\67301099.o5m: Attention: Tile contains both drive-on-left (2) and drive-on-right roads (2) Number of MapFailedExceptions: 1 Number of ExitExceptions: 0 Getting this while trying to build the map with DEM The result is attached. Thanks a lot Joao

2 15

Another AssertionError
by Carlos Dávila 23 Sep '20

23 Sep '20

I get the error reproduced below when I combine osm and contour lines precompiled tiles for the map of all Asia. Command line used is: java -Xmx27G -ea -jar mkgmap_trunk.jar --gmapi --max-jobs --index --latin1 --code-page=1252 --description="OSM-$MAPA-Topo" --family-name="OSM $MAPA Topo" --family-id=3$FID --product-id=1 --product-version=$VERSION --series-name="OSM-$MAPA-Topo" --area-name="$MAPA" --overview-mapname=$ABR-3$FID --overview-mapnumber=653${FID}000 --road-name-config=$CONFIG --x-mdr7-del=GRADE0,GRADE1,GRADE2,GRADE3,GRADE4,GRADE5,GRADE6,GRADE7,UNNAMED --show-profiles=1 tmp/551${FID}*.img curvas/602${FID}*.img typ/$ABR-3${FID}.TYP Exception in thread "main" java.lang.AssertionError at uk.me.parabola.imgfmt.app.trergn.TREHeader.readFileHeader(TREHeader.java:107) at uk.me.parabola.imgfmt.app.CommonHeader.readHeader(CommonHeader.java:95) at uk.me.parabola.imgfmt.app.trergn.TREFileReader.<init>(TREFileReader.java:56) at uk.me.parabola.mkgmap.combiners.FileInfo.treInfo(FileInfo.java:300) at uk.me.parabola.mkgmap.combiners.FileInfo.imgInfo(FileInfo.java:283) at uk.me.parabola.mkgmap.combiners.FileInfo.getFileInfo(FileInfo.java:156) at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:612) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:125) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:144) at uk.me.parabola.mkgmap.main.Main.main(Main.java:115) I use the same command for many other maps, including big ones like Europe or North America without problems, only Asia fails. I can provide all input files if necessary, but they sum several GB.

2 2

Commit r4585: try to improve error handling when reading input files in one of the combiners
by svn commit 23 Sep '20

23 Sep '20

Version mkgmap-r4585 was committed by gerd on Wed, 23 Sep 2020 try to improve error handling when reading input files in one of the combiners http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4585

1 0

Consolidation of mkgmap changes
by dave pitney 22 Sep '20

22 Sep '20

Hello Back in December I suggested creating a periodic listing of the major changes to mkgmap so that users can more easily understand how the code base is changing and new features. Now that I have access to a computer I have created a list based on commits from April to the present, as attached. I have compressed the commits from 100 to 7 entries. I believe a good place to post this is in the Latest News section of mkgmap.org.uk. Please review and provide me feedback, including the Format and data of my entries Also if my list is too long or short If the community finds this useful, I will need a contact to review the lists and posting privileges on mkgmap.org. I can commit to producing this twice a year. Thanks, -- Dave Pitney mappitney(a)gmail.com ----------------- end of message ------------------------

1 0

Unable to build unicode map with index
by Carlos Dávila 22 Sep '20

22 Sep '20

Hello all I am trying to build an unicode map of Israel or Greece but they always fail with java.lang.AssertionError as soon as I add --index to command line. I have tried both my own style and default one. Only if I use an small osm file as input it succeeds. The simplest command line that throws the error is: java -Xmx3G -ea -jar mkgmap_trunk.jar --bounds=bounds.zip --route --unicode --family-name="Test" --family-id=190 --product-id=1 --description="Test" --series-name="Test" --overview-mapname=55190000 --mapname=55190000 --index --style=default --remove-ovm-work-files=true 51178007.o5m Time started: Tue Sep 22 10:37:47 CEST 2020 Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.AssertionError: 914 at uk.me.parabola.imgfmt.app.FileBackedImgFileWriter.put1u(FileBackedImgFileWriter.java:166) at uk.me.parabola.imgfmt.app.mdr.Mdr8.writeSectData(Mdr8.java:45) at uk.me.parabola.imgfmt.app.mdr.MDRFile.writeSection(MDRFile.java:421) at uk.me.parabola.imgfmt.app.mdr.MDRFile.writeSections(MDRFile.java:373) at uk.me.parabola.imgfmt.app.mdr.MDRFile.write(MDRFile.java:269) at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334) at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:675) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:125) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:144) at uk.me.parabola.mkgmap.main.Main.main(Main.java:115) Test files can be found are 51178007.o5m <https://alternativaslibres.org/tmp/51178007.o5m> and jerusalem.o5m <https://alternativaslibres.org/tmp/jerusalem.o5m>

2 2

Commit r4584: don't write MDR12 section with unicode codepage
by svn commit 22 Sep '20

22 Sep '20

Version mkgmap-r4584 was committed by gerd on Tue, 22 Sep 2020 don't write MDR12 section with unicode codepage http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4584

1 0

Another Assertion error
by Joao Almeida 22 Sep '20

22 Sep '20

Hi I'm trying to compile a map of Iberia with DEM and Contours but getting the following assertionError can someone give e a hint of what might be wrong? Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.AssertionError: 226391 at uk.me.parabola.imgfmt.app.BufferedImgFileWriter.put2u(BufferedImgFileWriter.java:164) at uk.me.parabola.imgfmt.app.trergn.Zoom.write(Zoom.java:81) at uk.me.parabola.imgfmt.app.trergn.TREFile.writeMapLevels(TREFile.java:214) at uk.me.parabola.imgfmt.app.trergn.TREFile.writeBody(TREFile.java:122) at uk.me.parabola.imgfmt.app.trergn.TREFile.write(TREFile.java:341) at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:301) at uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:192) at uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:104) at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:675) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:125) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:144) at uk.me.parabola.mkgmap.main.Main.main(Main.java:115) Thanks a lot Joao

2 2