Hi all, today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents. Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help. Gerd
Hi Gerd I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great. Kind Regards Joris -----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment Hi all, today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents. Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help. Gerd
I also face wrong sea from time to time. I agree a pre-check would be much useful. Also your sea-check style+empty tiles, thanks Gerd. El 10/9/20 a las 11:27, Joris Bo escribió:
Hi Gerd
I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great.
Kind Regards Joris
-----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment
Hi all,
today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents.
Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi all, OK, I'll try to find a good solution. For hgt files we have the file resources/known-hgt.bin which tells mkgmap which *.hgt files it should be able to find. I think the same can be done for those rectangles which will always be "land" in the index.txt file. I just have to filter those which are close to the coast so that they might change. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Donnerstag, 10. September 2020 18:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] precomp sea test environment I also face wrong sea from time to time. I agree a pre-check would be much useful. Also your sea-check style+empty tiles, thanks Gerd. El 10/9/20 a las 11:27, Joris Bo escribió:
Hi Gerd
I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great.
Kind Regards Joris
-----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment
Hi all,
today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents.
Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi all, it seems the correctness of the input files provided at [1] is not as good as Wanmil and I thought. Wrong data like a single closed way tagged natural=coastline near the center of Africa was not removed. I've corrected it a few minutes ago, see version 1 of https://www.openstreetmap.org/way/845273583. Still working on a simple check ... ciao, Gerd [1] https://osmdata.openstreetmap.de/data/land-polygons.html ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 10. September 2020 20:20 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] precomp sea test environment Hi all, OK, I'll try to find a good solution. For hgt files we have the file resources/known-hgt.bin which tells mkgmap which *.hgt files it should be able to find. I think the same can be done for those rectangles which will always be "land" in the index.txt file. I just have to filter those which are close to the coast so that they might change. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Donnerstag, 10. September 2020 18:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] precomp sea test environment I also face wrong sea from time to time. I agree a pre-check would be much useful. Also your sea-check style+empty tiles, thanks Gerd. El 10/9/20 a las 11:27, Joris Bo escribió:
Hi Gerd
I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great.
Kind Regards Joris
-----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment
Hi all,
today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents.
Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi all, my current approach for a verification routine is this: The precomp-sea data contains tiles 512*256 tiles, each covering an area of 50.000x50.000 Garmin units. There is an index that tells mkgmap if a tile is completely land or sea or if it contains a mix of both. Only the mixed tiles are stored in pbf format. Obviously the content of the mixed tiles are likely to change whenever someone changes the natural=coastline data. Also tiles with sea-only may change to mixed when a small island is mapped for the first time, so the check should only look at land-only tiles in the index. Next thought: If an edge of such a land-only tile is very close to the coast it might become a (correct) mixed tile soon, but it should never be sea-only for a long time even with lots of global warming. So, I have compiled a text file with 256 lines, each containing 512 characters (either x or blank) to indicate the land-only areas for a sea.zip created with land-polygons-split-4326 data from 2020-09-13. (I might have used only one bit for each tile but it is much easier to maintain the text file and it is zipped to less than 4kb). If you open the file with a text editor and chose a small font you'll see the familiar shapes of the continents and large islands. My plan is to add this file as a reference to ressources and load it when the precomp-sea index is loaded (this happens only once when mkgmap is started). If the reference file contains an 'x' and the corresponding entry in the index says "sea-only" something is probably wrong. I see two use cases: 1) PrecompSeaGenerator should warn when it computes a new set of precomp-sea data 2) mkgmap should warn when such a bad precomp-sea file is used. Open question: Should it always warn or only when the generated sea in any map tiles are affected? Another possible approach would be to store also the mixed and sea-only status in the reference (see sea-check-lms.zip). This would allow to do more plausibility checks, e.g. if a previously sea-only tile became land-only. Not sure if this ever happened in the past. What do you think? Did I miss something? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Samstag, 12. September 2020 12:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] precomp sea test environment Hi all, it seems the correctness of the input files provided at [1] is not as good as Wanmil and I thought. Wrong data like a single closed way tagged natural=coastline near the center of Africa was not removed. I've corrected it a few minutes ago, see version 1 of https://www.openstreetmap.org/way/845273583. Still working on a simple check ... ciao, Gerd [1] https://osmdata.openstreetmap.de/data/land-polygons.html ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 10. September 2020 20:20 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] precomp sea test environment Hi all, OK, I'll try to find a good solution. For hgt files we have the file resources/known-hgt.bin which tells mkgmap which *.hgt files it should be able to find. I think the same can be done for those rectangles which will always be "land" in the index.txt file. I just have to filter those which are close to the coast so that they might change. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Donnerstag, 10. September 2020 18:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] precomp sea test environment I also face wrong sea from time to time. I agree a pre-check would be much useful. Also your sea-check style+empty tiles, thanks Gerd. El 10/9/20 a las 11:27, Joris Bo escribió:
Hi Gerd
I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great.
Kind Regards Joris
-----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment
Hi all,
today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents.
Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hello Gerd Sounds good. I think both use cases are necessary. When compiling a new sea.zip it should complain avoiding it to be published or used. When using a zip (with lots of thankyou to http://osm.thkukuk.de !! ) it should detect an accidentally bad zip and avoid starting the build process. The chances of current land suddenly turning into sea is far less then broken coastlines, so I guess we don't need to worry about that. I assume the reference file can be 'fixed' as well within the mkgmap commits. Maybe the check should only be done if a specific option is given on the commandline. Then you can always continue in case of a bad reference not yet fixed. Kind regards, Joris -----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: maandag 14 september 2020 11:33 Aan: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Onderwerp: Re: [mkgmap-dev] precomp sea test environment Hi all, my current approach for a verification routine is this: The precomp-sea data contains tiles 512*256 tiles, each covering an area of 50.000x50.000 Garmin units. There is an index that tells mkgmap if a tile is completely land or sea or if it contains a mix of both. Only the mixed tiles are stored in pbf format. Obviously the content of the mixed tiles are likely to change whenever someone changes the natural=coastline data. Also tiles with sea-only may change to mixed when a small island is mapped for the first time, so the check should only look at land-only tiles in the index. Next thought: If an edge of such a land-only tile is very close to the coast it might become a (correct) mixed tile soon, but it should never be sea-only for a long time even with lots of global warming. So, I have compiled a text file with 256 lines, each containing 512 characters (either x or blank) to indicate the land-only areas for a sea.zip created with land-polygons-split-4326 data from 2020-09-13. (I might have used only one bit for each tile but it is much easier to maintain the text file and it is zipped to less than 4kb). If you open the file with a text editor and chose a small font you'll see the familiar shapes of the continents and large islands. My plan is to add this file as a reference to ressources and load it when the precomp-sea index is loaded (this happens only once when mkgmap is started). If the reference file contains an 'x' and the corresponding entry in the index says "sea-only" something is probably wrong. I see two use cases: 1) PrecompSeaGenerator should warn when it computes a new set of precomp-sea data 2) mkgmap should warn when such a bad precomp-sea file is used. Open question: Should it always warn or only when the generated sea in any map tiles are affected? Another possible approach would be to store also the mixed and sea-only status in the reference (see sea-check-lms.zip). This would allow to do more plausibility checks, e.g. if a previously sea-only tile became land-only. Not sure if this ever happened in the past. What do you think? Did I miss something? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Samstag, 12. September 2020 12:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] precomp sea test environment Hi all, it seems the correctness of the input files provided at [1] is not as good as Wanmil and I thought. Wrong data like a single closed way tagged natural=coastline near the center of Africa was not removed. I've corrected it a few minutes ago, see version 1 of https://www.openstreetmap.org/way/845273583. Still working on a simple check ... ciao, Gerd [1] https://osmdata.openstreetmap.de/data/land-polygons.html ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 10. September 2020 20:20 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] precomp sea test environment Hi all, OK, I'll try to find a good solution. For hgt files we have the file resources/known-hgt.bin which tells mkgmap which *.hgt files it should be able to find. I think the same can be done for those rectangles which will always be "land" in the index.txt file. I just have to filter those which are close to the coast so that they might change. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos@alternativaslibres.org> Gesendet: Donnerstag, 10. September 2020 18:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] precomp sea test environment I also face wrong sea from time to time. I agree a pre-check would be much useful. Also your sea-check style+empty tiles, thanks Gerd. El 10/9/20 a las 11:27, Joris Bo escribió:
Hi Gerd
I recently (24-8-2020) had the same problem. The bad sea.zip was about 14Mb smaller then usual. A pre-check and warning would be great.
Kind Regards Joris
-----Oorspronkelijk bericht----- Van: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> Namens Gerd Petermann Verzonden: donderdag 10 september 2020 11:16 Aan: mkgmap-dev@lists.mkgmap.org.uk Onderwerp: [mkgmap-dev] precomp sea test environment
Hi all,
today I stumbled over a bad sea.zip file that was created in 2019-11-08. Most of the two Americas was flooded. I wondered if I should add some code to mkgmap to detect this case? Eg. mkgmap could check some points which are clearly inside continents.
Besides that it is rather simple to produce a test style for the sea which just creates an overview map. I've created some empty tiles which cover full planet and used them to create an overview map containing only sea polygons. This compilation takes ~1 minute. A quick glance at osmmap.img should show the familiar shapes of the continents, if not something is wrong and that probably means that the precomp-sea data is bad. I've attached this, maybe it is of some help.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (4)
-
Carlos Dávila -
Gerd Petermann -
Gerd Petermann
-
Joris Bo