Hi Nick, thanks for reporting. Yes, I saw a similar issue with a horizontal line at 47° / 48° today and I think I fixed it. Seems that I have to do something different for vertical boundaries. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von osm@pinns <osm@pinns.co.uk> Gesendet: Dienstag, 19. Dezember 2017 19:12:57 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Congratulations Gerd! Brilliant. I know its just the beginning. Have just done Luxembourg - very fast - however there is an obvious dip perhaps where 2 hgt files join? http://files.mkgmap.org.uk/download/375/luxstripe.jpg r Nick On 19/12/2017 17:41, Gerd Petermann wrote: Hi all, FYI: r4006 in the dem-tdb branch implements a new option --x-dem=path_to_dir_with_hgt_files It will crash if any hgt file is missing, probably there are also other stupid problems, e.g. with negative lat/lon values. It cannot (yet) read zip files, so the directory must contain unzipped *.hgt files. The hgt files can be in 3'' or 1'' resolution, but for now DEM data is always stored with 3'' res. I plan to change that soon, so that 1'' is used if at least a part of the tile is covered by 1'' SRTM data. It doesn't yet create DEM for the overview map and it only creates DEM for level 0. So far I've only implemented a very basic hgt reader which doesn't do any interpolation, this might change soon. I did not check if the flags in the tdb file are set correctly now, I've only created some small maps for parts of the alps so far and they looked good in Basecamp /Mapsource. If you want to try this version, see http://www.mkgmap.org.uk/download/mkgmap-dem-tdb-r4006.zip @Frank: My currennt understanding is that mkgmap should do the calculation of the DEM resolution based on the levels used to create the rest of the map. I have to find out how to calculate the values for lower resolutions. I assume it has to be the average of the values, so if at res 24 we have 4 points with 123, 124,128, and 122 the value for res 23 would be Math.round((123+124+128+122)/4) = 124. For res 22 it would be the average of 4*4=16 values, and so on. Does that make sense? ciao, Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Nick, sorry, I don't understand the problem. How do you add osm/.mp files to your main pfb file ? I assume you mean you call mkgmap with multiple sets of input files which overlap ? In that case maybe you can use the dem option only for one subset ? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von osm@pinns <osm@pinns.co.uk> Gesendet: Dienstag, 19. Dezember 2017 20:06:59 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Hi Gerd There is another minor issue which is unrelated to the problems you have identified, but one I have noticed with Franks program as well: The problem with a blanket --dem option is that , if you add say contour osm/.mp files to your main pfb file, it also calculates dem subfiles for those extra files. There is no easy solution and interestingly Garmin does not object to having duplicate dem imgs , ie more than one img with the same dem subfile. There might me a way of flagging duplicate zones? Nick On 19/12/2017 18:16, Gerd Petermann wrote:

Hi Nick,

thanks for reporting. Yes, I saw a similar issue with a horizontal line at 47° / 48° today and I think I fixed it. Seems that I have to do something different for vertical boundaries.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von osm@pinns <osm@pinns.co.uk> Gesendet: Dienstag, 19. Dezember 2017 19:12:57 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Congratulations Gerd! Brilliant.

I know its just the beginning.

Have just done Luxembourg - very fast - however there is an obvious dip perhaps where 2 hgt files join?

http://files.mkgmap.org.uk/download/375/luxstripe.jpg

r

Nick

On 19/12/2017 17:41, Gerd Petermann wrote:

Hi all,

FYI: r4006 in the dem-tdb branch implements a new option --x-dem=path_to_dir_with_hgt_files

It will crash if any hgt file is missing, probably there are also other stupid problems, e.g. with negative lat/lon values.

It cannot (yet) read zip files, so the directory must contain unzipped *.hgt files. The hgt files can be in 3'' or 1'' resolution, but for now DEM data is always stored with 3'' res. I plan to change that soon, so that 1'' is used if at least a part of the tile is covered by 1'' SRTM data.

It doesn't yet create DEM for the overview map and it only creates DEM for level 0. So far I've only implemented a very basic hgt reader which doesn't do any interpolation, this might change soon. I did not check if the flags in the tdb file are set correctly now, I've only created some small maps for parts of the alps so far and they looked good in Basecamp /Mapsource.

If you want to try this version, see http://www.mkgmap.org.uk/download/mkgmap-dem-tdb-r4006.zip

@Frank: My currennt understanding is that mkgmap should do the calculation of the DEM resolution based on the levels used to create the rest of the map. I have to find out how to calculate the values for lower resolutions. I assume it has to be the average of the values, so if at res 24 we have 4 points with 123, 124,128, and 122 the value for res 23 would be Math.round((123+124+128+122)/4) = 124. For res 22 it would be the average of 4*4=16 values, and so on. Does that make sense?

ciao, Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Thanks Gerd On 19/12/2017 20:53, Gerd Petermann wrote:

Hi Nick,

I've fixed this problem with r4007.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von osm@pinns <osm@pinns.co.uk> Gesendet: Dienstag, 19. Dezember 2017 19:12:57 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Congratulations Gerd! Brilliant.

I know its just the beginning.

Have just done Luxembourg - very fast - however there is an obvious dip perhaps where 2 hgt files join?

http://files.mkgmap.org.uk/download/375/luxstripe.jpg

r

Nick

On 19/12/2017 17:41, Gerd Petermann wrote:

Hi all,

FYI: r4006 in the dem-tdb branch implements a new option --x-dem=path_to_dir_with_hgt_files

It will crash if any hgt file is missing, probably there are also other stupid problems, e.g. with negative lat/lon values.

It cannot (yet) read zip files, so the directory must contain unzipped *.hgt files. The hgt files can be in 3'' or 1'' resolution, but for now DEM data is always stored with 3'' res. I plan to change that soon, so that 1'' is used if at least a part of the tile is covered by 1'' SRTM data.

It doesn't yet create DEM for the overview map and it only creates DEM for level 0. So far I've only implemented a very basic hgt reader which doesn't do any interpolation, this might change soon. I did not check if the flags in the tdb file are set correctly now, I've only created some small maps for parts of the alps so far and they looked good in Basecamp /Mapsource.

If you want to try this version, see http://www.mkgmap.org.uk/download/mkgmap-dem-tdb-r4006.zip

@Frank: My currennt understanding is that mkgmap should do the calculation of the DEM resolution based on the levels used to create the rest of the map. I have to find out how to calculate the values for lower resolutions. I assume it has to be the average of the values, so if at res 24 we have 4 points with 123, 124,128, and 122 the value for res 23 would be Math.round((123+124+128+122)/4) = 124. For res 22 it would be the average of 4*4=16 values, and so on. Does that make sense?

ciao, Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Frank, okay, I'll have a closer look again at the demo maps. In the moment my code only works with one resolution and I did not yet find out why :-( When I use a different res MapSource shows only parts of the map while DemDisplay still says that all is good. Hope I find out more tomorrow. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Frank Stinner <Frank.Stinner@kabelmail.de> Gesendet: Dienstag, 19. Dezember 2017 21:53:06 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Hi Gerd, my first idea was, 1 bit less for resolution means doubling the distance. Or in other words, then we use simply only every second point. But for me it don't looks like good. I use "levels=0:24, 1:23, 2:22, 3:21, 4:20, 5:19" and for --dlon 0.00027761, 0.00049, 0.00075, 0.00106, 0.0017 and 0.0025. The relations are rounded 1.8, 1.5, 1.4, 1.6, 1.5. With such relations we don't have an easy way to calculate one DEM level from another. I think, it's better to use ever the original data for calculating and interpolating a level. Frank --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Andrzej, I looked at the three demo topo maps that I have which have more that one zoom level and I found this: All have the same levels 24,22,20,18. topo2 uses DEM point distances 1648, 6624, 26512, 106048 and was created 17 Dez 2007 Transalpin uses DEM point distances 3312, 13248, 26512, 53024 and was created 2 Sep 2009 topo-2010 uses DEM point distances 5520, 16560, 44178, 88368 and was created 28 Apr 2010 The distance between two DEM points is 360° / 2^32 or ~40000km / 2^32 ~ 0,00931 m (lat), so 3312 *0,00931 m is ~ 30 m. So, I see no obvious rule for the ratios, esp. there are not always exact multiples, so we probably need an option for this. I see no need to store the ratio in an extra field in the header. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Andrzej Popowski <popej@poczta.onet.pl> Gesendet: Mittwoch, 20. Dezember 2017 00:28:59 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Hi Gerd, great news!

It will crash if any hgt file is missing I have them all, but there is no HGT for sea area, missing files is a standard case.

I did not check if the flags in the tdb file are set correctly now I didn't check either, but tdb created by mkgmap v3999 works correctly for maps with DEM.

My currennt understanding is that mkgmap should do the calculation of the DEM resolution based on the levels used to create the rest of the map. I think it should be like that and some Garmin maps follow this rule. I have tried to create DEM with pixel size ratio like 1:4:16:64 for levels 24,22,20,18. It didn't work correctly in Mapsource. What works for me is ratio 1:2:4:8. I think some of unknown fields in DEM header could contain this ratio. It is something to investigate further.

-- Best regards, Andrzej _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Thanks Gerd for this. This might be "sometimes mapsource displays parts of the map and relief looks wrong. Unless i'm wrong. Tried to send it to GPSMAP 62s, seems aok except on mapsource/basecamp https://www.dropbox.com/s/y2vu5i9eahdo3df/dem-4008.jpg?dl=0 Ervin M. *Schadow1 Expeditions* - A Filipino must not be a stranger to his own motherland. http://www.s1expeditions.com On Thu, Dec 21, 2017 at 12:36 AM, Gerd Petermann < gpetermann_muenchen@hotmail.com> wrote:

Hi Henning,

please try with r4008 which implements that and also interpolation of hgt values. There seems to be a major bug somewhere in the encoder, sometimes MapSource displays only parts of the map and relief looks wrong. I did not yet find out where exactly the problem is. I tried to compare the results from mkgmp with those from BuildDEMFile. It seems that mkgmap sometimes calculates different positions because of rounding differences. I tried to do most calculations with integers while BuildDEMFile uses doubles. Will try again tomorrow.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Mittwoch, 20. Dezember 2017 17:06:25 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Hi Gerd, it's a great news. I wanted to try it on my map of China. But unfortunately failed because of missing files in the sea area between Korea and Japan. Maybe it's a good 1st approach to interpret a missing file as an area with 0 elevation.

Anyway, thanks for all the research work on DEM!!!

Bye, Henning _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Henning, yes, that matches my findings. So far I found a small difference in the encoding of plateau lenghts but even when I changed my code to match that of BuildDEMFile I did not see an improvement. I am now trying to find out why BuildDEMFile sometimes interpolates a different height value. The values differ by a few cm and thus sometimes mkgmap rounds down while BuildDEMFile rounds up (or vice versa). I am pretty sure that this is not a problem but it makes it nearly impossible to compare the resulting bit streams. Just a matter of time ... Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Donnerstag, 21. Dezember 2017 00:59:38 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Hi Gerd, just a brief feedback: Overall it seems to be fine. In MapSource and Basecamp most of the DEM-Data is displayed. But there a many squares completely empty (I guess not read at all by Garmin). Of these tiles a lot are along the coast and nearby lake-area. Maybe this is related to large areas with no changing elevation. But also there are some coast-tiles working well. If you need additional information, please let me know. Bye Henning On 21.12.2017 00:36, Gerd Petermann wrote:

Hi Henning,

please try with r4008 which implements that and also interpolation of hgt values. There seems to be a major bug somewhere in the encoder, sometimes MapSource displays only parts of the map and relief looks wrong. I did not yet find out where exactly the problem is. I tried to compare the results from mkgmp with those from BuildDEMFile. It seems that mkgmap sometimes calculates different positions because of rounding differences. I tried to do most calculations with integers while BuildDEMFile uses doubles. Will try again tomorrow.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Mittwoch, 20. Dezember 2017 17:06:25 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Hi Gerd, it's a great news. I wanted to try it on my map of China. But unfortunately failed because of missing files in the sea area between Korea and Japan. Maybe it's a good 1st approach to interpret a missing file as an area with 0 elevation.

Anyway, thanks for all the research work on DEM!!!

Bye, Henning _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Henning, of course the new DEM data requires additional space in the image, so unless we tell splitter to take that into account you have to use a smaller maxnodes value. Please wait until mkgmap is finally able to write multiple levels before you actually try to find out a new maxnodes value for splitter. Maybe I will change splitter also to take DEM into account somehow. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Donnerstag, 21. Dezember 2017 14:56:33 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data Of course you definitely should take your time ;) As we were waiting for ages for DEM, it doesn't matter :D It seems that 4008 compared to 3997 has the issue that several tiles are not compiled and the not enough room exception is written in the logs. I remember you mentioned it before, but couldn't find it. Im usually using 1,200,000 for splitter, reduced now to 1,000,000 but still have problems. Please find the tiles as osm.pbf file. The areas tagged as water are not compiled with 4008, the meadow-tile is fine. It seems the limit is not depending on real area size, as the measdow tile is much larger as several of the water-tiles. But there are not so many hills in, compared to the smaller ones. As said before, 3997 is all fine. The additional time looks pretty good already. I attached you also my mkgmap logs. If I should log it more detailed or send you some more data, please let me know. Henning On 21.12.2017 16:51, Gerd Petermann wrote:

Hi Henning,

yes, that matches my findings. So far I found a small difference in the encoding of plateau lenghts but even when I changed my code to match that of BuildDEMFile I did not see an improvement. I am now trying to find out why BuildDEMFile sometimes interpolates a different height value. The values differ by a few cm and thus sometimes mkgmap rounds down while BuildDEMFile rounds up (or vice versa). I am pretty sure that this is not a problem but it makes it nearly impossible to compare the resulting bit streams. Just a matter of time ...

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Donnerstag, 21. Dezember 2017 00:59:38 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Hi Gerd, just a brief feedback: Overall it seems to be fine. In MapSource and Basecamp most of the DEM-Data is displayed. But there a many squares completely empty (I guess not read at all by Garmin). Of these tiles a lot are along the coast and nearby lake-area. Maybe this is related to large areas with no changing elevation. But also there are some coast-tiles working well. If you need additional information, please let me know.

Bye Henning

On 21.12.2017 00:36, Gerd Petermann wrote:

...

Hi Henning,

please try with r4008 which implements that and also interpolation of hgt values. There seems to be a major bug somewhere in the encoder, sometimes MapSource displays only parts of the map and relief looks wrong. I did not yet find out where exactly the problem is. I tried to compare the results from mkgmp with those from BuildDEMFile. It seems that mkgmap sometimes calculates different positions because of rounding differences. I tried to do most calculations with integers while BuildDEMFile uses doubles. Will try again tomorrow.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Henning Scholland <osm@hscholland.de> Gesendet: Mittwoch, 20. Dezember 2017 17:06:25 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] r4006: 1st alpha version to write DEM data

Hi Gerd, it's a great news. I wanted to try it on my map of China. But unfortunately failed because of missing files in the sea area between Korea and Japan. Maybe it's a good 1st approach to interpret a missing file as an area with 0 elevation.

Anyway, thanks for all the research work on DEM!!!

Bye, Henning _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd and Henning, the count of nodes haven't influence to creation of dem, only the size (and the surface) of the area. I haven't see a problem with the size of dem file, but perhaps it is a good idea if we had a additional restriction for splitter. Not only maxnode, but a "maxwidth" and/or "maxheight". Then we can limit the size of areas like deserts. On my "explorations" i have see, that a defective 64x64 tile most troubling the whole maptile. It is not easy to find the cause of that. Perhaps the decoder in mapsource don't stop automaticly at the end of the data range for a 64x64 tile. By the way, if you save the input int-data for your encoder in a textfile, you can "feed" builddemfile with this int-data (option --data). Or you can test this data with the prog input2. Frank --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus