Norway not buildable with the no-trim option
I'm not able to build the map for Norway with the "no-trim" option. - Source:norway extract from geofabrik from 22.12.2012 - Bounds from 18.11.2012, Sea Tiles from 12.12.2012 - mkgmap 2412 / splitter 263 Splitter options: Splitter version 263 compiled 2012-12-17T19:04:48+0000 cache= description= geonames-file=/home/kto/Freizeitkarte-Entwicklung-1212/cities/cities15000.zip keep-complete=true mapid=65780001 max-areas=512 max-nodes=800000 max-threads=3 mixed=false no-trim=true output=xml output-dir=/home/kto/Freizeitkarte-Entwicklung-1212/work/Freizeitkarte_Norwegen overlap=-1 polygon-file= problem-file= problem-report= resolution=13 split-file= status-freq=120 stop-after=dist write-kml= This seems to be the problem area (which spans over a large area): Area 65780001 covers (57.568359375,11.689453125) to (61.5234375,31.552734375) and contains 674846 nodes (84 %) Area 65780001 covers (0x28f000,0x85000) to (0x2bc000,0x167000) RU-Saint Petersburg I let splitter calculate the tiles: # List of areas # Generated Sat Dec 22 09:39:42 UTC 2012 # 65780001: 2682880,544768 to 2867200,1470464 # : 57.568359,11.689453 to 61.523438,31.552734 65780002: 3205120,544768 to 3340288,765952 # : 68.774414,11.689453 to 71.674805,16.435547 65780003: 3205120,765952 to 3223552,864256 # : 68.774414,16.435547 to 69.169922,18.544922 mkgmap refuses to build the tile 65780001: java -Xmx6000M -jar /home/kto/Freizeitkarte-Entwicklung-1212/tools/mkgmap/mkgmap.jar --max-jobs=3 -c Freizeitkarte_Norwegen.cfg Time started: Sat Dec 22 09:48:08 UTC 2012 There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first. Perhaps a problem within the map data ... how to investigate it ? Screenshots with the trim option: <http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-trim.png> <http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-trim.png> Screenshot with the no-trim option: <http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-notrim.png> <http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-notrim.png> Regards Klaus PS: With the trim option the map is successfully buildable. -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Klaus, I'll have a look at it, but probably not during the next days ;-) Gerd toc-rox wrote
I'm not able to build the map for Norway with the "no-trim" option.
- Source:norway extract from geofabrik from 22.12.2012 - Bounds from 18.11.2012, Sea Tiles from 12.12.2012 - mkgmap 2412 / splitter 263
Splitter options:
Splitter version 263 compiled 2012-12-17T19:04:48+0000 cache= description= geonames-file=/home/kto/Freizeitkarte-Entwicklung-1212/cities/cities15000.zip keep-complete=true mapid=65780001 max-areas=512 max-nodes=800000 max-threads=3 mixed=false no-trim=true output=xml output-dir=/home/kto/Freizeitkarte-Entwicklung-1212/work/Freizeitkarte_Norwegen overlap=-1 polygon-file= problem-file= problem-report= resolution=13 split-file= status-freq=120 stop-after=dist write-kml=
This seems to be the problem area (which spans over a large area): Area 65780001 covers (57.568359375,11.689453125) to (61.5234375,31.552734375) and contains 674846 nodes (84 %) Area 65780001 covers (0x28f000,0x85000) to (0x2bc000,0x167000) RU-Saint Petersburg
I let splitter calculate the tiles:
# List of areas # Generated Sat Dec 22 09:39:42 UTC 2012 # 65780001: 2682880,544768 to 2867200,1470464 # : 57.568359,11.689453 to 61.523438,31.552734
65780002: 3205120,544768 to 3340288,765952 # : 68.774414,11.689453 to 71.674805,16.435547
65780003: 3205120,765952 to 3223552,864256 # : 68.774414,16.435547 to 69.169922,18.544922
mkgmap refuses to build the tile 65780001:
java -Xmx6000M -jar /home/kto/Freizeitkarte-Entwicklung-1212/tools/mkgmap/mkgmap.jar --max-jobs=3 -c Freizeitkarte_Norwegen.cfg Time started: Sat Dec 22 09:48:08 UTC 2012 There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first.
Perhaps a problem within the map data ... how to investigate it ?
Screenshots with the trim option: <http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-trim.png>
<http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-trim.png>
Screenshot with the no-trim option:
<http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-notrim.png>
<http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-notrim.png>
Regards Klaus
PS: With the trim option the map is successfully buildable.
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi Klaus, first of all I think Norway is a typical case for NOT using no-trim. I was not able to reproduce the problem, I already get different results from splitter: Area 65780001 covers (57.568359375,13.0078125) to (68.7744140625,31.552734375) and contains 650273 nodes (81 %) I assume that you merge srtm data into your file? Please let me know how you produce the input for splitter and the content of your Freizeitkarte_Norwegen.cfg Gerd
Date: Sat, 22 Dec 2012 02:11:02 -0800 From: easyclasspage@googlemail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: [mkgmap-dev] Norway not buildable with the no-trim option
I'm not able to build the map for Norway with the "no-trim" option.
- Source:norway extract from geofabrik from 22.12.2012 - Bounds from 18.11.2012, Sea Tiles from 12.12.2012 - mkgmap 2412 / splitter 263
Splitter options:
Splitter version 263 compiled 2012-12-17T19:04:48+0000 cache= description= geonames-file=/home/kto/Freizeitkarte-Entwicklung-1212/cities/cities15000.zip keep-complete=true mapid=65780001 max-areas=512 max-nodes=800000 max-threads=3 mixed=false no-trim=true output=xml output-dir=/home/kto/Freizeitkarte-Entwicklung-1212/work/Freizeitkarte_Norwegen overlap=-1 polygon-file= problem-file= problem-report= resolution=13 split-file= status-freq=120 stop-after=dist write-kml=
This seems to be the problem area (which spans over a large area): Area 65780001 covers (57.568359375,11.689453125) to (61.5234375,31.552734375) and contains 674846 nodes (84 %) Area 65780001 covers (0x28f000,0x85000) to (0x2bc000,0x167000) RU-Saint Petersburg
I let splitter calculate the tiles:
# List of areas # Generated Sat Dec 22 09:39:42 UTC 2012 # 65780001: 2682880,544768 to 2867200,1470464 # : 57.568359,11.689453 to 61.523438,31.552734
65780002: 3205120,544768 to 3340288,765952 # : 68.774414,11.689453 to 71.674805,16.435547
65780003: 3205120,765952 to 3223552,864256 # : 68.774414,16.435547 to 69.169922,18.544922
mkgmap refuses to build the tile 65780001:
java -Xmx6000M -jar /home/kto/Freizeitkarte-Entwicklung-1212/tools/mkgmap/mkgmap.jar --max-jobs=3 -c Freizeitkarte_Norwegen.cfg Time started: Sat Dec 22 09:48:08 UTC 2012 There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first.
Perhaps a problem within the map data ... how to investigate it ?
Screenshots with the trim option:
<http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-trim.png>
<http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-trim.png>
Screenshot with the no-trim option:
<http://gis.19327.n5.nabble.com/file/n5741482/Map-Norway-notrim.png>
<http://gis.19327.n5.nabble.com/file/n5741482/Tiles-Norway-notrim.png>
Regards Klaus
PS: With the trim option the map is successfully buildable.
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, thanks for looking into it ... yes I add contours to the map data. In case of Norway the map data are 86 MB and the elevation data are 145 MB (both in pbf format). You can find the requested data here: http://www.freizeitkarte-osm.de/maps/Tmp/Norwegen/Freizeitkarte_Norwegen/ Regards Klaus PS: With "NOT using no-trim" you mean it's better to use "trim" (no-trim=true) - right ? -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Am 23.12.2012 10:38, schrieb toc-rox:
Hi Gerd,
thanks for looking into it ... yes I add contours to the map data. In case of Norway the map data are 86 MB and the elevation data are 145 MB (both in pbf format).
You can find the requested data here: http://www.freizeitkarte-osm.de/maps/Tmp/Norwegen/Freizeitkarte_Norwegen/
Regards Klaus
PS: With "NOT using no-trim" you mean it's better to use "trim" (no-trim=true) - right ? If you only use data for Norway and using no-trim=true without any bounding polygon you'll have main parts of Sweden in your map, which are quiet empty. This could be problematic. So maybe you should use a polygon that cut Sweden away from your map.
Henning
Hello Klaus, not using no-trim means: use either --no-trim=false or remove the option, because --no-trim=false is the default. I know that --no-trim was needed in the past to avoid holes, but the new split algorithm will not produce holes when trimming. I see only one reason for using --no-trim with r263: If you want to have tiles that cover the original bounding box of the input file completely. ciao, Gerd
Date: Sun, 23 Dec 2012 01:38:27 -0800 From: easyclasspage@googlemail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Norway not buildable with the no-trim option
Hi Gerd,
thanks for looking into it ... yes I add contours to the map data. In case of Norway the map data are 86 MB and the elevation data are 145 MB (both in pbf format).
You can find the requested data here: http://www.freizeitkarte-osm.de/maps/Tmp/Norwegen/Freizeitkarte_Norwegen/
Regards Klaus
PS: With "NOT using no-trim" you mean it's better to use "trim" (no-trim=true) - right ?
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I see only one reason for using --no-trim with r263: If you want to have tiles that cover the original bounding box of the input file completely.
Hi Gerd, yes, that's exactly the reason fr using the "--no-trim" option. The results is, that the map looks like a (rectangular) map ... I think Norway is a good example for this. Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
The problem is (again) caused by the --precomp-sea option which adds a lot of data to this tile. This was already reported before and I don't see a simple solution besides the trimming. Possible solution: Splitter calculates the tile sizes only based on the number of nodes in the input file. What we need is the number of nodes added by the --precomp-sea option So, either splitter could read these files and calculate the number of nodes, or the sea generator could create a file that contains the number. This would save processing time because sea tiles don't change often. Problem with this 2nd approach: The resolution of the grid that is used in splitter depends on the resolution parameter. Any other ideas? Gerd
Date: Sun, 23 Dec 2012 01:38:27 -0800 From: easyclasspage@googlemail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Norway not buildable with the no-trim option
Hi Gerd,
thanks for looking into it ... yes I add contours to the map data. In case of Norway the map data are 86 MB and the elevation data are 145 MB (both in pbf format).
You can find the requested data here: http://www.freizeitkarte-osm.de/maps/Tmp/Norwegen/Freizeitkarte_Norwegen/
Regards Klaus
PS: With "NOT using no-trim" you mean it's better to use "trim" (no-trim=true) - right ?
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Am 23.12.2012 11:40, schrieb Gerd Petermann:
The problem is (again) caused by the --precomp-sea option which adds a lot of data to this tile. This was already reported before and I don't see a simple solution besides the trimming.
Possible solution: Splitter calculates the tile sizes only based on the number of nodes in the input file. What we need is the number of nodes added by the --precomp-sea option
So, either splitter could read these files and calculate the number of nodes, or the sea generator could create a file that contains the number. This would save processing time because sea tiles don't change often. Problem with this 2nd approach: The resolution of the grid that is used in splitter depends on the resolution parameter.
Hi Gerd, this could also cause other problems because nodes are conted twice. In case of Norway precomp-sea adds nearly complete baltic sea and Norwegian coastline. In input-data there are already Norwegian coastline counted, so they are counted twice. Henning
Henning Scholland wrote
this could also cause other problems because nodes are conted twice. In case of Norway precomp-sea adds nearly complete baltic sea and Norwegian coastline. In input-data there are already Norwegian coastline counted, so they are counted twice.
Yes, sure. The algo that I have in mind would first count the nodes in the normal input file. It would then detect all grid elements with a count of 0. For these elements it would add the number of "sea nodes ". Maybe not 100% precise, but for sure thats better than nothing. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi Gerd, does it makes sense to build a norway map with the "new" splitter ? Currently I don't think so - right ? Regards Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Hi Klaus, I think I fixed the problem for your configuration, but as you already saw it revealed new issues. The current status is this: - splitter can calculate tiles based also on the number of nodes in the precomp-sea tiles. For your case, that results in a few more tiles for norway some of them are only filled with data from the generate-sea option. These tiles may cause crashes on some devices (I did never try that) - the multipolygon handling in mkgmap has two major problems: 1) it is removing a lot of the small polygons created by the generate-sea method, this leads to the white stripes that you see in low resolution 2) it is saving large polygons with far too many details for low resolution, this leads to the message "There is not enough room in a single garmin map for all the input data..." I have a work around for the 2nd issue, but the first one requires a major redesign of the program. Gerd
Date: Thu, 3 Jan 2013 07:56:53 -0800 From: easyclasspage@googlemail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Norway not buildable with the no-trim option
Hi Gerd,
does it makes sense to build a norway map with the "new" splitter ? Currently I don't think so - right ?
Regards Klaus
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, I have tried to build the norway map (unmodified data) with splitter 275 (on top of splitter 270) ... but the result was negativ: There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first. Question: Is r275 the right version ? Regards Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
toc-rox wrote
Hi Gerd,
I have tried to build the norway map (unmodified data) with splitter 275 (on top of splitter 270) ... but the result was negativ:
There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first.
Question: Is r275 the right version ?
Regards Klaus
Hmm, I did not commit the latest changes to weight the sea nodes with a factor of two. The open question is: will we see problems when a tile contains only sea and land polygons? Do you want to try that? I think the more promising approach is to use generate-sea=polygons, but I don't know what the wiki means when it says that you have sea below the land... Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
GerdP wrote
... Hmm, I did not commit the latest changes to weight the sea nodes with a factor of two. The open question is: will we see problems when a tile contains only sea and land polygons? Do you want to try that?
I think the more promising approach is to use generate-sea=polygons, but I don't know what the wiki means when it says that you have sea below the land... ...
I want to try both ... let's start with the first approach. I can test negative effects on a Dakota and a Montana. If others are interested in this issue too, I also can publish the map on the internet. Then we will see ... Regards Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Klaus, I've just committed r276 . Change: precomp-sea: Count the sea nodes with a factor of 2. Please make sure that you use parm --precomp-sea in splitter, else you will not see a difference. Gerd
Date: Thu, 3 Jan 2013 10:57:22 -0800 From: easyclasspage@googlemail.com To: mkgmap-dev@lists.mkgmap.org.uk Subject: Re: [mkgmap-dev] Norway not buildable with the no-trim option
Hi Gerd,
I have tried to build the norway map (unmodified data) with splitter 275 (on top of splitter 270) ... but the result was negativ:
There is not enough room in a single garmin map for all the input data The .osm file should be split into smaller pieces first.
Question: Is r275 the right version ?
Regards Klaus
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, here are the results with splitter r276: Splitter version unknown compiled 2013-01-03T19:08:35+0000 cache= description= geonames-file=/home/kto/Freizeitkarte-Entwicklung-1212/cities/cities15000.zip keep-complete=true mapid=65780001 max-areas=512 max-nodes=800000 max-threads=3 mixed=false no-trim=true output=xml output-dir=/home/kto/Freizeitkarte-Entwicklung-1212/work/Freizeitkarte_Norwegen overlap=auto polygon-file= precomp-sea=/home/kto/Freizeitkarte-Entwicklung-1212/sea problem-file= problem-report= resolution=13 split-file= status-freq=120 stop-after=dist write-kml= <http://gis.19327.n5.nabble.com/file/n5742612/Bildschirmfoto_2013-01-03_um_21.06.04.png> <http://gis.19327.n5.nabble.com/file/n5742612/Bildschirmfoto_2013-01-03_um_21.07.44.png> Ar first view it looks good in BaseCamp ... tomorrow I will test the image in both of my Garmins and publish the map (BaseCamp, GPSr, QLandkarte) for further investigations ... Regards Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
I have tested the norway map in both of my devices (Montana, Dakota) and couldn't find any problem. Regards Klaus -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
toc-rox wrote
I have tested the norway map in both of my devices (Montana, Dakota) and couldn't find any problem.
Regards Klaus
Thanks for testing. The problem is that - afaik - we did never exactly find out what the reason for the crash was, we just know that it is likely to occur in almost empty areas. So, the absence of a crash with your data doesn't mean the problem is solved :-( It may also be related to gps firmware. @Felix: Are you able to reproduce the problem reported in this thread? http://gis.19327.n5.nabble.com/Serious-Bug-Mkgmap-creating-map-that-puts-new... Please try also the removeShortArcs_v2.patch. This reduces the amount of data needed to store the coast line information in the *.img file by ~ 25%, so it reduces the likelyhood for the "There is not enough room in a single garmin map for all the input data" message. I hope that WanMils work on the new mp_cut branch will provide a much better solution for this. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
On 08.01.2013 10:45, GerdP wrote:
toc-rox wrote
I have tested the norway map in both of my devices (Montana, Dakota) and couldn't find any problem.
Regards Klaus Thanks for testing. The problem is that - afaik - we did never exactly find out what the reason for the crash was, we just know that it is likely to occur in almost empty areas. So, the absence of a crash with your data doesn't mean the problem is solved :-( It may also be related to gps firmware.
@Felix: Are you able to reproduce the problem reported in this thread? http://gis.19327.n5.nabble.com/Serious-Bug-Mkgmap-creating-map-that-puts-new... with new splitter, no problems with the bay of biscay anymore. But I don't really understand what that bug has to do with "not enough room in osm file" problem - which is a problem about too few interaction between mkgmap and splitter.... (integrated fallback by mkgmap calling splitter and splitting the file into two parts would be best IMHO).
Please try also the removeShortArcs_v2.patch. This reduces the amount of data needed to store the coast line information in the *.img file by ~ 25%, so it reduces the likelyhood for the "There is not enough room in a single garmin map for all the input data" message. I hope that WanMils work on the new mp_cut branch will provide a much better solution for this.
Gerd
-- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
Felix Hartmann-2 wrote
with new splitter, no problems with the bay of biscay anymore. But I don't really understand what that bug has to do with "not enough room in osm file" problem - which is a problem about too few interaction between mkgmap and splitter.... (integrated fallback by mkgmap calling splitter and splitting the file into two parts would be best IMHO).
OK, I try to explain. The new split algo tries to fill each tile with at least max-nodes/3 nodes. For a coast like that of Norway it is likely that you have a very large tile that covers the baltic sea. If mkgmap is used with --generate-sea in combination with precomp-sea, mkgmap adds a lot of coastline data to that tile. This results in the "There is not enough room ..." message. No matter how small the max-nodes value is, splitter will always generate this big tile if --no-trim is active. So, I've added the parm --precomp-sea to splitter to allow it to count the coastline nodes as well. Now, it generates smaller tiles, but some of them might contain nothing but coastline data. The question is if that will cause the crash again. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Norway-not-buildable-with-the-no-trim-option-... Sent from the Mkgmap Development mailing list archive at Nabble.com.
participants (5)
-
Felix Hartmann -
Gerd Petermann -
GerdP
-
Henning Scholland -
toc-rox