July 2020 - mkgmap-dev - The mkgmap lists

Fwd: Re: "Small" SRTM1 files and DEM maps
by Carlos Dávila 05 Oct '20

05 Oct '20

Hi Gerd I replied to your patch a couple of months ago, but I have now realized it didn't arrive to the list (it seems I used and old address). I was surprised you didn't say anything, but didn't want to bother... I used your binary for my test. Today I have tried to apply your patch, but it seems to be generated against a different version of files in trunk. -------- Mensaje reenviado -------- Asunto: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps Fecha: Tue, 21 Apr 2020 17:38:19 +0200 De: Carlos Dávila <cdavilam(a)orangecorreo.es> Para: mkgmap-dev(a)lists.mkgmap.org.uk Hi Gerd I have tested it in an area of 4x10 degrees, in the North of China, and it worked fine. El 21/4/20 a las 14:40, Gerd Petermann escribió: > Hi Carlos, > > please try the attached patch or this binary based on r4483: > http://files.mkgmap.org.uk/download/468/mkgmap.jar > I hope I found all the places that need to be changed. I've only > tested with a map around the point N50 E118. > > Gerd > > ________________________________________ > Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag > von Carlos Dávila <carlos(a)alternativaslibres.org> > Gesendet: Dienstag, 21. April 2020 11:27 > An: mkgmap-dev(a)lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps > > Great new!! Thanks for your support > > El 21/4/20 a las 11:19, Gerd Petermann escribió: >> Hi Carlos, >> >> yes, I see that the source code handles this format. I finally found >> https://dds.cr.usgs.gov/srtm/version2_1/Documentation/MIL-PDF-89020B.pdf >> I think "TABLE I. Matrix intervals for DTED Level 0." >> shows how to interpret the data. It seems there are a few more >> variants... >> For now I'll try to support also the 1801 * 3601 * 2 format >> >> Gerd >> >> ________________________________________ >> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >> von Carlos Dávila <carlos(a)alternativaslibres.org> >> Gesendet: Donnerstag, 16. April 2020 17:41 >> An: mkgmap-dev(a)lists.mkgmap.org.uk >> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >> >> HI Gerd >> I don't think conversion is wrong. If input geotiff file is 24MB, output >> hgt file is 24 MB, if input tiff is 12MB, output hgt is 12MB >> I use this command on Linux: >> for i in *.tif ; do gdal_translate -strict -q -eco -of SRTMHGT $i >> `basename $i tif`hgt ; done ; rm -f *.hgt.aux.xml >> BTW, gdal is also able to deal with small SRTM files >> >> El 16/4/20 a las 11:31, Gerd Petermann escribió: >>> Hi Carlos, >>> >>> so your conversion routine is wrong. The file size is 12.970.802 >>> bytes. There is no integer n for which 2 * n^2 results in this value. >>> >>> Gerd >>> >>> ________________________________________ >>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >>> von Carlos Dávila <cdavilam(a)orangecorreo.es> >>> Gesendet: Donnerstag, 16. April 2020 11:19 >>> An: mkgmap-dev(a)lists.mkgmap.org.uk >>> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>> >>> Hi Gerd >>> Forgot to mention files are downloaded as geotiff, but I then convert >>> them to hgt, sorry. >>> You can download a sample one from >>> https://alternativaslibres.org/tmp/N50E118.hgt >>> >>> El 16/4/20 a las 9:18, Gerd Petermann escribió: >>>> Hi Carlos, >>>> >>>> are you talking about geotiff format? I think someone documented >>>> how that format can be converted to *.hgt. >>>> Can you post a link to such a file? >>>> >>>> Gerd >>>> >>>> >>>> >>>> ________________________________________ >>>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im Auftrag >>>> von Carlos Dávila <cdavilam(a)orangecorreo.es> >>>> Gesendet: Montag, 13. April 2020 14:08 >>>> An: mkgmap-dev(a)lists.mkgmap.org.uk >>>> Betreff: Re: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>>> >>>> Hi Gerd >>>> They are downloaded by phyghtmap. This is what I found digging into >>>> phyghtmap files: >>>> |def getSRTMFileServer(self, resolution, srtmVersion):|| >>>> || if srtmVersion == 2.1:|| >>>> || return >>>> "https://dds.cr.usgs.gov/srtm/version2_1/SRTM{0:d}".format(resolution)|| >>>> || elif srtmVersion == 3.0:|| >>>> || if resolution == 1:|| >>>> || urlRe = >>>> "https://earthexplorer.usgs.gov/download/8360/SRTM1{:s}V3/GEOTIFF/EE"|| >>>> || elif resolution == 3:|| >>>> || urlRe = >>>> "https://earthexplorer.usgs.gov/download/4960/SRTM3{:s}V2/GEOTIFF3/EE"|| >>>> || return urlRe| >>>> They should come from the second URL above. >>>> >>>> El 13/4/20 a las 8:24, Gerd Petermann escribió: >>>>> Hi Carlos, >>>>> >>>>> sorry for the late response. I found no hint on the phyghtmap page >>>>> http://katze.tfiu.de/projects/phyghtmap/ >>>>> regarding a new format for srtm data. The hgt format is not >>>>> compressed. Do you know from where these 12.4 MB files came from? >>>>> >>>>> Gerd >>>>> >>>>> ________________________________________ >>>>> Von: mkgmap-dev <mkgmap-dev-bounces(a)lists.mkgmap.org.uk> im >>>>> Auftrag von Carlos Dávila <cdavilam(a)orangecorreo.es> >>>>> Gesendet: Samstag, 11. April 2020 17:09 >>>>> An: Development list for mkgmap >>>>> Betreff: [mkgmap-dev] "Small" SRTM1 files and DEM maps >>>>> >>>>> I use phyghtmap to build contour lines and also to download hgt data >>>>> that I then use to build DEM maps with mkgmap. For a long time, SRTM1 >>>>> files (and also VFP1) have all been ~24.7 MB in size, but lately (I >>>>> don't remember since when) SRTM1 are also served as 12.4 MB files. >>>>> phyghtmap is able to generate contour lines from those small files, as >>>>> they seem to contain the same information but with a different >>>>> compression ratio, but mkgmap fails to read them, and report them as >>>>> missing. Would it be possible to get support for such files in mkgmap? >>>>> _______________________________________________ >>>>> mkgmap-dev mailing list >>>>> mkgmap-dev(a)lists.mkgmap.org.uk >>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >>>>> _______________________________________________ >

2 2

Error in Africa?
by valentin3151＠gmail.com 14 Sep '20

14 Sep '20

Hi all, My daily compilation of Africa was stopped because error is appear: SEVERE (MapSplitter): 63240201.osm.pbf: Single item predicted to exceed subdivision http://www.openstreetmap.org/?mlat=10.412207&mlon=-13.010709&zoom=17 Number of MapFailedExceptions: 0 Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 113, Size: 3 at java.util.ArrayList.rangeCheck(Unknown Source) at java.util.ArrayList.get(Unknown Source) at uk.me.parabola.imgfmt.app.net.NETFileReader.getRoads(NETFileReader.java:116) at uk.me.parabola.imgfmt.app.map.MapReader.getRoads(MapReader.java:205) at uk.me.parabola.mkgmap.combiners.MdrBuilder.addStreets(MdrBuilder.java:298) at uk.me.parabola.mkgmap.combiners.MdrBuilder.onMapEnd(MdrBuilder.java:171) at uk.me.parabola.mkgmap.combiners.GmapsuppBuilder.onMapEnd(GmapsuppBuilder.java:164) at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:665) at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:125) at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:144) at uk.me.parabola.mkgmap.main.Main.main(Main.java:115) Link to my tile where the error is: https://maptourist.org/tmp/63240201.osm.pbf I tried to find the cause of this error in OSM but could not. The link from above error log points to the location with completely no data somewhere in the middle of Guinea. As an experiment, I loaded the problem tile into JOSM and fixed all automatically found errors. Then I tried to compile the corrected file. No positive effect. The error is still there. I use mkgmap-r4564, but I found there is no dependency with other versions. How do I find what exactly is causing this error in the OSM data? Previously, this map was compiled many times over a long time without such problem. -- Best regards, Valentin mailto:valentin3151@gmail.com

4 17

garmin zumo XT
by brad 09 Aug '20

09 Aug '20

I was successfully building maps for a Montana with my own style. Since the Montana died, I replaced it with a Zumo XT. It isn't working so well now. I can use my map on qmapshack and it looks as expected, but on the Zumo it appears that Garmin does not recognize some (or all) my custom type codes. Any ideas what the limitations are, or how to work with this? They've added something called 'Map Theme', which is a small xml file. I don't see a way to load my typ file, but I didn't need to do that before since I included that with the mkgmap build. Brad

7 14

default style lines enhancements
by Ticker Berkin 31 Jul '20

31 Jul '20

Hi all style gurus Patch attached of enhancements to default style "lines" which I've been accumulating for a while. Also includes the parallel changes needed to mapnik.txt I'm happy to explain any of these change in more detail and, if anyone considers a particular change incorrect, provide a revised patch. Summary of changes, roughly in order of diffs in the patch. 1/ Sometimes charges for using a pedestrian highway are expressed as a fee rather than a toll, so pick this up in mkgmap:toll. 2/ Show bridges using type 0x10107. With the mapnik addition, these look good for narrow highways, but might not show for wide representations like motorways. 3/ Where it is likely that bits of highway might not be connected to the road/path system, use mkgmap:set_unconnected_type and mkgmap:set_semi_connected_type to stop the NET/NOD rather than relying on NETnoNOD (now disabled) and --check-routing-island-len=+ve, which is being suspected of causing problems on some devices and BaseCamp. 4/ Quote some filter subst strings that contain spaces - no actual effect but clearer and safer. 5/ Have line for leisure=track even if area. 6/ Change the type for tracks/raceways etc from 0x30, which doesn't show on BaseCamp or MapSource, to 0x2a. 7/ For piers, if 'unconnecting', use marine type 0x1040c (Obstruction: Pier or Jetty) instead of footway. 8/ Change type for the various barriers from 0x17, which doesn't show on BaseCamp to 0x2b, see: http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q1/030348.html 9/ Consider natural=cliff a barrier. 10/ Add motorway[_link] roundabouts (yes, some do exist). 11/ Unquote some numbers - no actual effect. 12/ Tweak some road speeds 13/ Use 0x09 (high-speed ramp) for road class 4 links 14/ Add footway around car parks if 'connecting'. 15/ Disable coastline. For a long time the tag was suppressed by the Sea processing and it adds little to the map. 16/ Improve railway platform names and suppress footway if not 'connecting'. 17/ Show disused:railway in the same way as railway=disused. 18/ Have cable_car, gondola, funicular as routable, by default with a toll and pedestrian only. 19/ Show "Course of old Railway", unless a highway has taken over the way (for you Eric, but I like it as well). 20/ Speed up car ferries. 21/ A few other layout/space fixes. Ticker

6 31

Re: [mkgmap-dev] Sorry, Google is being restrictive. Try 2 with the files.
by Ticker Berkin 31 Jul '20

31 Jul '20

Hi Randolph One other thing - The resolution that Garmin devices show icons can depend on the device and type/subtype values. For some you need to zoom in quite a lot before they appear. Ticker

1 0

Re: [mkgmap-dev] Sorry, Google is being restrictive. Try 2 with the files.
by Ticker Berkin 31 Jul '20

31 Jul '20

Hi Randolph I think your test.txt TYP file might be exceeding various device limitations and have a few other problems. I've just converted it with mkgmap and then using TYPViewer to examine the resultant .typ file, firstly it takes a couple of mins to open the file, then it seems to be missing some entries. I'd recommend: Don't have [end][_point] on same line Stick to Types in range 0x01 .. 0x7f. Be consistent with Type= and Subtype=; I think you have entries that resolve to the same thing. If Type >= 0x100, the bottom 8 bits are the sub-type and the rest is the type. Don't have sub-types for cities (ie 0x01 .. 0x0d). Anyway, test-map doesn't generate them which is why you see the single line of icons to the left of the big block. Have sub-types (up to 0x1f) for the ranges where they have definite significance, ie: 0x2a..0x30 and 0x64..0x66 There are a few other places where subtype also have meaning but this is manifest in the default icon. Ticker

1 0

Re: [mkgmap-dev] [Fwd: Re: Changes that I made in a recent set of default style files and the reason I have for making those changes.]
by Ticker Berkin 30 Jul '20

30 Jul '20

Hi Randolph It is best to do city... / population test in the way it is done in the default style. If, after the "place & population > xxx" rules you try place=city & population!=* [...] it will fail if, for instance, the tag was [population=unknown] The current default style doesn't conflate pizza with italian; each has and uses the obvious Garmin type. Some conflations are there because there is a limited and strange set of Garmin cuisine types, including fast_food. There is no perfect (or even good) solution. If you search for "American" cuisine, what would you expect to find? Should burgers be here or under "Barbeque" or "Fast Food"? BaseCamp and my 2 eTrexes don't find 0x2c1e or 0x2c1f when searching for "Places of Worship". They also don't appear in you list of TYP codes. My older eTrex doesn't find 0x2c0d .. 0x2c10 either. My preference for handling this situation is to add rules for types only supported by newer devices commented out, before the generic one. Then, if someone has a device that supports them they can un-comment. It is also good documentation to show which types to avoid for other purposes. Older devices tended to find types off the end of known values with, say, Find > Attractions > Other, and this is why there are problems like the one you noticed where 0x2c0d was used for an unspecified tourism=*. So, it is a bad idea to try and invent meanings for any POI in the searchable ranges, rather use the closest meaning subtype or subtype 0. There was a thread a short while ago about type codes and their meanings. Here are links to 2 parts of the thread where lists are attached, but it is worth checking the whole thread. http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q2/030945.html http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q2/030969.html You can generate a test map with something like: $ java -jar mkgmap-rVVVV/mkgmap.jar \ --verbose \ --order-by-decreasing-area \ test-map:all-elements \ mkgmap-rVVVV/resources/typ-files/sameOrder.txt + --gmapsupp or --gmapi or whatever Source is: {svn repos}/src/uk/me/parabola/mkgmap/reader/test/AllElements.java Ticker

2 1

Changes that I made in a recent set of default style files and the reason I have for making those changes.
by Randolph J. Herber 29 Jul '20

29 Jul '20

In case that I grabbed points and polygons files that I had already modified, I included both before and after versions as well as a patch file for each. I never really liked the cities, towns, villages, suburbs, and hamlets handling. Now, they are by size only. In the USA, the difference has more to do with legal privileges than anything else. There are such entities with populations over 20 million: Shanghai, Beijing and Lagos and there are extant entities with names and no recorded population. They are often called ghost towns. In some states, they even have legal protection. I wanted to break out places of worship by religion for at least some of the major religions. I can supply icons for them. Making icons is easy for me. amenity=place_of_worship & religion=christian [0x2c0e resolution 24] amenity=place_of_worship & religion=muslim [0x2c0d resolution 24] amenity=place_of_worship & religion=jewish [0x2c10 resolution 24] amenity=place_of_worship & religion=buddhist [0x2c0f resolution 24] amenity=place_of_worship & religion=hindu [0x2c1f resolution 24] amenity=place_of_worship & religion=bahai [0x2c1e resolution 24] amenity=place_of_worship [0x2c0b resolution 24] Therefore, I changed the code in this line to move that item out of the block of codes that I am using. tourism=* & tourism!=no & tmp:stopMopUp!=yes [0x2f00 resolution 24] I used Wolfram Mathematica (mostly, just because it was convenient) to process the style files and some lists from the Maps Tool Kit to see what codes are available and which mkgmap style files were using. I attached that file as well in case it may be of use to you. I separated prisons and military areas in both points and polygons.

2 4

check-roundabout-flares
by Mike Baggaley 28 Jul '20

28 Jul '20

Dear all, I have been working on improvements to the --check-roundabout-flares option to provide better decisions as to what should be considered a flare and have managed to greatly reduce the number of messages that relate to roads that are not flares. Of course it is not possible to discard all unwanted messages without discarding the odd message this is relevant, hence the rules need to be configurable to suit the circumstances in the area being mapped. I propose to deprecate the --check-roundabout-flares and --max-flare-length-ratio options, with the former moving into the --check-roundabouts option and the latter into a new set of rules. The --check-roundabouts option would be extended to accept the following list of values [all|loop|direction[:fix]|junctions|flares,...] where: * all applies all the checks and fixes. * loop checks that roundabouts are formed from a single loop with no forks or gaps. * direction checks that the one way direction is in the expected direction for the side of the road on which traffic drives. If the fix suffix is supplied then roundabouts failing this check have their direction reversed. * junctions checks that no more than one connecting highway joins at each node. * flares checks that roundabout flare roads are one-way, point in the correct direction, and don't extend beyond the flare. This gives more flexibility over which checks are applied whilst reducing the number of command line options. For backwards compatibility, specifying the option with no value would be the same as --check-roundabouts=loop,direction:fix,junctions, and the default would be to not check anything. Specifying --check-roundabout-flares would set --check-roundabouts=flares and display an error message saying the option had been deprecated. For the flare processing, I have used a set of rules and am considering the best way to allow users to specify the rules that are to be applied in the decision making process. I don't want to add them as individual options as this complicates the interface too much. The choice therefore seems to be between a separate rules file and a command line option with a list of rule names and values. The rules I have are as follows: max-flare-length-multiple=NUM - Discard candidates where the length of one of the possible flares is greater than NUM times the length of the other. Default is 3. Set to 0 to disable. max-flare-to-separation-multiple=NUM - Discard candidates where the length of both of the possible flares are greater than NUM times the distance between the nodes on the roundabout that the flare roads connect to. Default is 7. Set to 0 to disable. (This is equivalent to the existing --max-flare-length-ratio option) max-flare-angle=NUM - Discard candidates where the angle subtended between the two possible flares is greater than NUM. Default is 90. Set to 0 to disable. max-flare-bearing=NUM - Discard candidates where the angle between the initial bearing of a possible flare road and the centre of the roundabout is greater than NUM. Default is 90. Set to 0 to disable. max-entry-angle=NUM - Discard candidates where the angle subtended from the centre of the roundabout to the two roundabout junction nodes is greater then NUM. Default is 145. Set to 0 to disable. both-flares-extend=[=yes|no] - Discard candidates where both roads extend beyond the point where they meet. more-roads-meet-roundabout=[=yes|no] - Discard candidates where the point where the roads meet contains more roads directly connecting to the roundabout. Default yes. multiple-flares=[=yes|no] - Discard candidates where the next node of a continuation of one of the roads has a road leading directly to the roundabout, forming a bigger flare. Commonly encountered where adjacent flares cross. Default yes. different-names=[=yes|no] - Discard candidates where the two roads have different names and neither name is a substring of the other. Default yes. Specifying -- max-flare-length-ratio would set the max-flare-to-separation-multiple value and display an error message saying the option had been deprecated. At the moment I have all the rules hard coded to the default values mentioned above. I am asking for feedback on whether the rules would be better placed in a rules file, or in a command line option, i.e. either --roundabout-flare-rules=LIST-OF-RULES or --roundabout-flare-rules-file=FILENAME. Also, if you have any ideas for more rules that could be used to remove unwanted flare candidates then I could look at including them. Ideally, I would like to compare the Garmin type code of the two roads, but I can't see how to get to it from a RoadDef. Perhaps you can suggest how this can be achieved? When a candidate is discarded, a message is logged, so it is also possible to use those messages to identify errors in the OSM data (I found and corrected a few name typos by looking at the roads discarded because of having different names). If you think any of the keywords could be improved, please let me know. I realise that some of the rules really need pictures to describe how they operate! Regards, Mike

2 2

192GB recommended
by Andrzej Popowski 28 Jul '20

28 Jul '20

Hi, I recently have noticed the following suggestion in mkgmap output: "To reduce the run time, consider increasing the amnount of memory available for use by mkgmap by using the Java -Xmx flag to set the memory to more than 196700 MB, providing this is less than the amount of physical memory installed." I run mkgmap to create files for PC installation - overview map, index, tdb. There is no actual compilation of any map. I guess, that condition for triggering the message is wrong in my case and maybe the calculations too. I mean this part of code in Main.java: if ((taskCount > threadCount + 1) && (maxJobs == 0) && (threadCount < runtime.availableProcessors())) { System.out.println("To reduce the run time, consider increasing the amnount of memory available for use by mkgmap by using the Java -Xmx flag to set the memory to more than " + 100* (1 + ((runtime.maxMemory() * runtime.availableProcessors()) / (threadCount * 1024 * 1024 * 100))) + " MB, providing this is less than the amount of physical memory installed."); } Is "taskCount" correct, when input files are only *.img? "threadCount" is probably forced to 1, because I see message: "Setting max-jobs to 1". Then probably there is no need to multiply memory for all processors. I'm not sure about "runtime.maxMemory()". It looks like 24GB which I have allocated with option -Xmx24G. Shouldn't it be a real memory used? Well, all is not a problem really. I just found the message funny and investigated a bit. Btw, there is a typo: "amnount". -- Best regards, Andrzej

2 2