Ahh, sorry, I just noticed that the file alaska.osm.pbf is small. The problem is here is that the bounding box spans from -180 to 180, but this box is most empty. I have to run splitter now to find the details. It works without --no-trim, probably also with an appropriate polygon file. Does that help? Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 24. März 2016 21:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hi Patrik, please provide the complete log from splitter and the densities-out.txt Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 20:25 An: Development list for mkgmap Betreff: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together, I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik. The exception is: Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157) The complete command line with the splitter call is: java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf The pbf source file comes from: http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf The osmconvert statistics about that file is: PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung> Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated. Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ? Any ideas are very welcome.... Cheers Patrik

Gerd, Yes, alaska.osm.pbf is small. It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option. Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ? You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ? Thanks already now for your help. Patrik On 24.03.2016 21:14, Gerd Petermann wrote:

Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.

The problem is here is that the bounding box spans from -180 to 180,

but this box is most empty. I have to run splitter now to find the details.

It works without --no-trim, probably also with an appropriate polygon file.

Does that help?

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com> *Gesendet:* Donnerstag, 24. März 2016 21:01 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hi Patrik,

please provide the complete log from splitter and the densities-out.txt

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 20:25 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together,

I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik.

The exception is:

Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157)

The complete command line with the splitter call is:

java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf

The pbf source file comes from:

http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf

The osmconvert statistics about that file is:

PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung>

Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated.

Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ?

Any ideas are very welcome....

Cheers Patrik

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Gerd, Sorry, your explanations came in during I was writing up the test results ... Think it's all clear so far. As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems... I'll contact geofabrik with the details. Many thanks and happy Easter. Patrik On 24.03.2016 22:07, Gerd Petermann wrote:

Hi Patrik,

I don't need the files, I downloaded the alaska file and tried some variants.

See also my last post, send a few minutes ago.

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:03 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd,

Yes, alaska.osm.pbf is small.

It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option.

Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ?

You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ?

Thanks already now for your help. Patrik

On 24.03.2016 21:14, Gerd Petermann wrote:

...

Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.

The problem is here is that the bounding box spans from -180 to 180,

but this box is most empty. I have to run splitter now to find the details.

It works without --no-trim, probably also with an appropriate polygon file.

Does that help?

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com> *Gesendet:* Donnerstag, 24. März 2016 21:01 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hi Patrik,

please provide the complete log from splitter and the densities-out.txt

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 20:25 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together,

I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik.

The exception is:

Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157)

The complete command line with the splitter call is:

java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf

The pbf source file comes from:

http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf

The osmconvert statistics about that file is:

PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung>

Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated.

Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ?

Any ideas are very welcome....

Cheers Patrik

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Gerd, I'll play a bit more with this option and check what suits me best. Again thanks for the incredibly quick answer to my problem/question. Cheers Patrik On 24.03.2016 22:20, Gerd Petermann wrote:

Hi Patrik,

I think the wanted effect of the no-trim option is a rectangular map,

which some people prefer, esp. on the PC.

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:16 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd,

Sorry, your explanations came in during I was writing up the test results ...

Think it's all clear so far.

As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems...

I'll contact geofabrik with the details.

Many thanks and happy Easter. Patrik

On 24.03.2016 22:07, Gerd Petermann wrote:

...

Hi Patrik,

I don't need the files, I downloaded the alaska file and tried some variants.

See also my last post, send a few minutes ago.

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:03 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd,

Yes, alaska.osm.pbf is small.

It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option.

Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ?

You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ?

Thanks already now for your help. Patrik

On 24.03.2016 21:14, Gerd Petermann wrote:

...

Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.

The problem is here is that the bounding box spans from -180 to 180,

but this box is most empty. I have to run splitter now to find the details.

It works without --no-trim, probably also with an appropriate polygon file.

Does that help?

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com> *Gesendet:* Donnerstag, 24. März 2016 21:01 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hi Patrik,

please provide the complete log from splitter and the densities-out.txt

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 20:25 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together,

I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik.

The exception is:

Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157)

The complete command line with the splitter call is:

java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf

The pbf source file comes from:

http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf

The osmconvert statistics about that file is:

PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung>

Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated.

Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ?

Any ideas are very welcome....

Cheers Patrik

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Gerd, I couldn't go to sleep without getting to the ground of this... I performed the following tests on linux, always with my standard --no-trim present: * original osm.pbf file downloaded from geofabrik (with the problematic bounds tag in it) -> FAILURE * converted with the help of osmconvert to normal osm format -> FAILURE (which was to be expected, same file, just different format) * deleted the osm tag with a simple command like 'cat file.osm | grep -v bounds > file-nobounds.osm' and ran splitter against it, always with the same options -> SUCCESS This leads me to the question why the bounds tag is read/used if it also works without and even gives less problem ? .... sorry for bombarding you (and the list), but I think it's nevertheless and interesting question. Cheers Patrik On 24.03.2016 23:51, KeenOnKites wrote:

Gerd,

I did dig a bit deeper... also as it rang a bell: we had quite a similar problem with the wrong bounding box in Alaska already October 2014. It was an illegal value of maxlon="180.0005" causing splitter to bail out

When I convert the osm.pbf file with the help of osmconvert to osm and look at the first few lines I see a 'bounds' tag announcing the problematic bounding box (not illegal as in 2014, but still 'suboptimal'):

<?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="osmconvert 0.8.2" timestamp="2016-03-23T20:13:02Z"> <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794" maxlon="179.9999"/> <node id="27207079" lat="64.7487541" lon="-147.3242821" version="2" ...

Getting the statistics via osmconvert with --out-statistics seems to read through the file and checks the 'real' bounding box:

... lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 ...

I'm now wondering if splitter get's confused by the existing but obviously strange bounds tag.

According to the findings in 2014 splitter can handle files without the bounds tag and just gets the real boundings from all the elements in the file. I did not test to run it through splitter without the bounds tag as I'm having troubles converting the file properly on my windows... but I'll try that probably tomorrow again on linux (sort of late already today). The process would be

osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to have same source file format) .... and then run the splitter and see what happens.

But if I have a proper osm file with the 'problematic' bounds tag in it, I can also try to reproduce the problem with the osm file. If it is reproducible I just drop the tag and try it again.

BTW: I've contacted geofabrik already via email

Patrik

On 24.03.2016 22:27, KeenOnKites wrote:

...

Gerd,

I'll play a bit more with this option and check what suits me best.

Again thanks for the incredibly quick answer to my problem/question.

Cheers Patrik

On 24.03.2016 22:20, Gerd Petermann wrote:

...

Hi Patrik,

I think the wanted effect of the no-trim option is a rectangular map,

which some people prefer, esp. on the PC.

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:16 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd,

Sorry, your explanations came in during I was writing up the test results ...

Think it's all clear so far.

As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems...

I'll contact geofabrik with the details.

Many thanks and happy Easter. Patrik

On 24.03.2016 22:07, Gerd Petermann wrote:

...

Hi Patrik,

I don't need the files, I downloaded the alaska file and tried some variants.

See also my last post, send a few minutes ago.

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:03 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd,

Yes, alaska.osm.pbf is small.

It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option.

Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ?

You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ?

Thanks already now for your help. Patrik

On 24.03.2016 21:14, Gerd Petermann wrote:

...

Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.

The problem is here is that the bounding box spans from -180 to 180,

but this box is most empty. I have to run splitter now to find the details.

It works without --no-trim, probably also with an appropriate polygon file.

Does that help?

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com> *Gesendet:* Donnerstag, 24. März 2016 21:01 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hi Patrik,

please provide the complete log from splitter and the densities-out.txt

Gerd

------------------------------------------------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 20:25 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together,

I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik.

The exception is:

Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157)

The complete command line with the splitter call is:

java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf

The pbf source file comes from:

http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf

The osmconvert statistics about that file is:

PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung>

Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated.

Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ?

Any ideas are very welcome....

Cheers Patrik

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

I have the same problem , with splitter for australia , or australia-oceania even when i have search limit at 60,000 or 90,000 or in between these Figures I stil get the same response in splitter log file . Stephen On Fri, Mar 25, 2016 at 4:33 PM, Gerd Petermann < GPetermann_muenchen@hotmail.com> wrote:

Hi Patrik,

good question. I also wondered if splitter can be changed to ignore wrong bounds

tags. My thoughts:

In your case the bbox is far too large, means it claims to contain data that is not there.

The normal case is vice versa: The file contains some nodes outside of the bbox,

maybe from ferry lines or other long ways, and my understanding is that we don't

want to calculate the tiles based on those few nodes, since we might get a map with

larger empty areas at the "border". No idea if that would cause more trouble.

With typical data from geofabrik and the --no-trim option there will always be large

empty areas as most countries are not rectangular ;-)

Possible changes in the code:

1) add a --ignore-osm-bounds option and set the default to false (means one gets

the same result like now when he uses the default)

2) add a --use-osm-bounds option and set the default to false

(means one might get a different result when using the default)

3) add code to check how the collected data matches the given bounds,

use whatever is smaller. This might also be triggered by an option if needed.

I'd like to get some feedback from others first.

Gerd

------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> *Gesendet:* Freitag, 25. März 2016 00:13 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Gerd,

I couldn't go to sleep without getting to the ground of this...

I performed the following tests on linux, always with my standard --no-trim present:

- original osm.pbf file downloaded from geofabrik (with the problematic bounds tag in it) -> FAILURE - converted with the help of osmconvert to normal osm format -> FAILURE (which was to be expected, same file, just different format) - deleted the osm tag with a simple command like 'cat file.osm | grep -v bounds > file-nobounds.osm' and ran splitter against it, always with the same options -> SUCCESS

This leads me to the question why the bounds tag is read/used if it also works without and even gives less problem ?

.... sorry for bombarding you (and the list), but I think it's nevertheless and interesting question.

Cheers Patrik

On 24.03.2016 23:51, KeenOnKites wrote:

Gerd,

I did dig a bit deeper... also as it rang a bell: we had quite a similar problem with the wrong bounding box in Alaska already October 2014. It was an illegal value of maxlon="180.0005" causing splitter to bail out

When I convert the osm.pbf file with the help of osmconvert to osm and look at the first few lines I see a 'bounds' tag announcing the problematic bounding box (not illegal as in 2014, but still 'suboptimal'):

<?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="osmconvert 0.8.2" timestamp="2016-03-23T20:13:02Z"> <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794" maxlon="179.9999"/> <node id="27207079" lat="64.7487541" lon="-147.3242821" version="2" ...

Getting the statistics via osmconvert with --out-statistics seems to read through the file and checks the 'real' bounding box:

... lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 ...

I'm now wondering if splitter get's confused by the existing but obviously strange bounds tag.

According to the findings in 2014 splitter can handle files without the bounds tag and just gets the real boundings from all the elements in the file. I did not test to run it through splitter without the bounds tag as I'm having troubles converting the file properly on my windows... but I'll try that probably tomorrow again on linux (sort of late already today). The process would be

osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to have same source file format) .... and then run the splitter and see what happens.

But if I have a proper osm file with the 'problematic' bounds tag in it, I can also try to reproduce the problem with the osm file. If it is reproducible I just drop the tag and try it again.

BTW: I've contacted geofabrik already via email

Patrik

On 24.03.2016 22:27, KeenOnKites wrote:

Gerd,

I'll play a bit more with this option and check what suits me best.

Again thanks for the incredibly quick answer to my problem/question.

Cheers Patrik

On 24.03.2016 22:20, Gerd Petermann wrote:

Hi Patrik,

I think the wanted effect of the no-trim option is a rectangular map,

which some people prefer, esp. on the PC.

Gerd

------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net> <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:16 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Gerd,

Sorry, your explanations came in during I was writing up the test results ...

Think it's all clear so far.

As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems...

I'll contact geofabrik with the details.

Many thanks and happy Easter. Patrik

On 24.03.2016 22:07, Gerd Petermann wrote:

Hi Patrik,

I don't need the files, I downloaded the alaska file and tried some variants.

See also my last post, send a few minutes ago.

Gerd

------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> <patrik.brunner@gmx.net><patrik.brunner@gmx.net> <patrik.brunner@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 22:03 *An:* mkgmap-dev@lists.mkgmap.org.uk *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Gerd,

Yes, alaska.osm.pbf is small.

It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option.

Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ?

You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ?

Thanks already now for your help. Patrik

On 24.03.2016 21:14, Gerd Petermann wrote:

Ahh, sorry, I just noticed that the file alaska.osm.pbf is small.

The problem is here is that the bounding box spans from -180 to 180,

but this box is most empty. I have to run splitter now to find the details.

It works without --no-trim, probably also with an appropriate polygon file.

Does that help?

Gerd ------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen@hotmail.com><GPetermann_muenchen@hotmail.com> <GPetermann_muenchen@hotmail.com> *Gesendet:* Donnerstag, 24. März 2016 21:01 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hi Patrik,

please provide the complete log from splitter and the densities-out.txt

Gerd

------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net><keenonkites@gmx.net> <keenonkites@gmx.net> *Gesendet:* Donnerstag, 24. März 2016 20:25 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem with splitter: Failed to find a correct split

Hello together,

I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik.

The exception is:

Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157)

The complete command line with the splitter call is:

java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf

The pbf source file comes from:

<http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf> http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf

The osmconvert statistics about that file is:

PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung>

Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated.

Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ?

Any ideas are very welcome....

Cheers Patrik

_______________________________________________ mkgmap-dev mailing listmkgmap-dev@lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing listmkgmap-dev@lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing listmkgmap-dev@lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing listmkgmap-dev@lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing listmkgmap-dev@lists.mkgmap.org.ukhttp://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Patrik, reg. performance for option 3: I don't expect any extra costs for this, splitter already collects all the data and applies the bbox after that. So, it always creates a (virtual) grid for the whole planet. With normal resolutions this never was a bottle neck. reg. implementation: I don't see any problem for any of the three options, but option 3 may requrie more thoughts. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> Gesendet: Freitag, 25. März 2016 10:35 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, yes, it's a complicated world with these non-rectancular countries, can't we change that ? ;-) I did some tests yesterday playing around with --no-trim and --polygon-files with Luxembourge (small enough to do quick tests), the result is not that much different if I'm skipping --no-trim or use the polygon file, the result ist still 'quite' rectancular, just a few bits are really left aside.... but I will have to do more tests with other countries (more complicated ones and bigger ones than LUX) to see if in my case I could/should skip --no-trim for building other maps too. Regarding your implementation option: * I would rather prefer not having option 2) implemented for the same reason as you mention: it changes the default behaviour for everyone. And as we have to say that for (rough guess) 99% of the cases the actual behaviour is working that's too much. * I like option 3). But enabling that by default would possibly increase the time used to split as it first has to run through the file to see which bounds (bounds tag or calculated one) have to be used... but I'm sure you could answer that much better than me... ;-) * Option 1) is possibly the cheapest to implement ? Thanks Patrik On 25.03.2016 07:33, Gerd Petermann wrote: Hi Patrik, good question. I also wondered if splitter can be changed to ignore wrong bounds tags. My thoughts: In your case the bbox is far too large, means it claims to contain data that is not there. The normal case is vice versa: The file contains some nodes outside of the bbox, maybe from ferry lines or other long ways, and my understanding is that we don't want to calculate the tiles based on those few nodes, since we might get a map with larger empty areas at the "border". No idea if that would cause more trouble. With typical data from geofabrik and the --no-trim option there will always be large empty areas as most countries are not rectangular ;-) Possible changes in the code: 1) add a --ignore-osm-bounds option and set the default to false (means one gets the same result like now when he uses the default) 2) add a --use-osm-bounds option and set the default to false (means one might get a different result when using the default) 3) add code to check how the collected data matches the given bounds, use whatever is smaller. This might also be triggered by an option if needed. I'd like to get some feedback from others first. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Freitag, 25. März 2016 00:13 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, I couldn't go to sleep without getting to the ground of this... I performed the following tests on linux, always with my standard --no-trim present: * original osm.pbf file downloaded from geofabrik (with the problematic bounds tag in it) -> FAILURE * converted with the help of osmconvert to normal osm format -> FAILURE (which was to be expected, same file, just different format) * deleted the osm tag with a simple command like 'cat file.osm | grep -v bounds > file-nobounds.osm' and ran splitter against it, always with the same options -> SUCCESS This leads me to the question why the bounds tag is read/used if it also works without and even gives less problem ? .... sorry for bombarding you (and the list), but I think it's nevertheless and interesting question. Cheers Patrik On 24.03.2016 23:51, KeenOnKites wrote: Gerd, I did dig a bit deeper... also as it rang a bell: we had quite a similar problem with the wrong bounding box in Alaska already October 2014. It was an illegal value of maxlon="180.0005" causing splitter to bail out When I convert the osm.pbf file with the help of osmconvert to osm and look at the first few lines I see a 'bounds' tag announcing the problematic bounding box (not illegal as in 2014, but still 'suboptimal'): <?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="osmconvert 0.8.2" timestamp="2016-03-23T20:13:02Z"> <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794" maxlon="179.9999"/> <node id="27207079" lat="64.7487541" lon="-147.3242821" version="2" ... Getting the statistics via osmconvert with --out-statistics seems to read through the file and checks the 'real' bounding box: ... lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 ... I'm now wondering if splitter get's confused by the existing but obviously strange bounds tag. According to the findings in 2014 splitter can handle files without the bounds tag and just gets the real boundings from all the elements in the file. I did not test to run it through splitter without the bounds tag as I'm having troubles converting the file properly on my windows... but I'll try that probably tomorrow again on linux (sort of late already today). The process would be osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to have same source file format) .... and then run the splitter and see what happens. But if I have a proper osm file with the 'problematic' bounds tag in it, I can also try to reproduce the problem with the osm file. If it is reproducible I just drop the tag and try it again. BTW: I've contacted geofabrik already via email Patrik On 24.03.2016 22:27, KeenOnKites wrote: Gerd, I'll play a bit more with this option and check what suits me best. Again thanks for the incredibly quick answer to my problem/question. Cheers Patrik On 24.03.2016 22:20, Gerd Petermann wrote: Hi Patrik, I think the wanted effect of the no-trim option is a rectangular map, which some people prefer, esp. on the PC. Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:16 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Sorry, your explanations came in during I was writing up the test results ... Think it's all clear so far. As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems... I'll contact geofabrik with the details. Many thanks and happy Easter. Patrik On 24.03.2016 22:07, Gerd Petermann wrote: Hi Patrik, I don't need the files, I downloaded the alaska file and tried some variants. See also my last post, send a few minutes ago. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <mailto:patrik.brunner@gmx.net> <patrik.brunner@gmx.net><mailto:patrik.brunner@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:03 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Yes, alaska.osm.pbf is small. It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option. Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ? You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ? Thanks already now for your help. Patrik On 24.03.2016 21:14, Gerd Petermann wrote: Ahh, sorry, I just noticed that the file alaska.osm.pbf is small. The problem is here is that the bounding box spans from -180 to 180, but this box is most empty. I have to run splitter now to find the details. It works without --no-trim, probably also with an appropriate polygon file. Does that help? Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <mailto:GPetermann_muenchen@hotmail.com> <GPetermann_muenchen@hotmail.com><mailto:GPetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 24. März 2016 21:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hi Patrik, please provide the complete log from splitter and the densities-out.txt Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <mailto:keenonkites@gmx.net> <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 20:25 An: Development list for mkgmap Betreff: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together, I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik. The exception is: Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157) The complete command line with the splitter call is: java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf The pbf source file comes from: <http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf>http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf The osmconvert statistics about that file is: PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung> Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated. Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ? Any ideas are very welcome.... Cheers Patrik _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi all, I tried to find out why splitter uses the bounding box provided in the file. The feature was added by Chris Miller with splitter release 102 with this comment: "Added support for the <bounds/> tag in OSM files. Also added a --status-freq parameter to periodically display status information." I searched the archives to find out why this was done but found no hint, can anybody remember what problem he wanted to solve ? For now I've added option --ignore-osm-bounds to splitter r437. See http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=436 and http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=437<http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=436> (sorry for the two changes, I first thought that I cannot use --ignore-osm-bounds as option name) Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> Gesendet: Freitag, 25. März 2016 13:32 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Good news, Gerd. .. as long as it's not too much thoughts... ;-) Just to give you a heads up regarding the 'source' of the problem: according to GeoFabrik the wrong bounds tag seems to be caused by osmium-history-splitter used to split the world file. Cheers Patrik On 25.03.2016 11:00, Gerd Petermann wrote: Hi Patrik, reg. performance for option 3: I don't expect any extra costs for this, splitter already collects all the data and applies the bbox after that. So, it always creates a (virtual) grid for the whole planet. With normal resolutions this never was a bottle neck. reg. implementation: I don't see any problem for any of the three options, but option 3 may requrie more thoughts. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net><mailto:patrik.brunner@gmx.net> Gesendet: Freitag, 25. März 2016 10:35 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, yes, it's a complicated world with these non-rectancular countries, can't we change that ? ;-) I did some tests yesterday playing around with --no-trim and --polygon-files with Luxembourge (small enough to do quick tests), the result is not that much different if I'm skipping --no-trim or use the polygon file, the result ist still 'quite' rectancular, just a few bits are really left aside.... but I will have to do more tests with other countries (more complicated ones and bigger ones than LUX) to see if in my case I could/should skip --no-trim for building other maps too. Regarding your implementation option: * I would rather prefer not having option 2) implemented for the same reason as you mention: it changes the default behaviour for everyone. And as we have to say that for (rough guess) 99% of the cases the actual behaviour is working that's too much. * I like option 3). But enabling that by default would possibly increase the time used to split as it first has to run through the file to see which bounds (bounds tag or calculated one) have to be used... but I'm sure you could answer that much better than me... ;-) * Option 1) is possibly the cheapest to implement ? Thanks Patrik On 25.03.2016 07:33, Gerd Petermann wrote: Hi Patrik, good question. I also wondered if splitter can be changed to ignore wrong bounds tags. My thoughts: In your case the bbox is far too large, means it claims to contain data that is not there. The normal case is vice versa: The file contains some nodes outside of the bbox, maybe from ferry lines or other long ways, and my understanding is that we don't want to calculate the tiles based on those few nodes, since we might get a map with larger empty areas at the "border". No idea if that would cause more trouble. With typical data from geofabrik and the --no-trim option there will always be large empty areas as most countries are not rectangular ;-) Possible changes in the code: 1) add a --ignore-osm-bounds option and set the default to false (means one gets the same result like now when he uses the default) 2) add a --use-osm-bounds option and set the default to false (means one might get a different result when using the default) 3) add code to check how the collected data matches the given bounds, use whatever is smaller. This might also be triggered by an option if needed. I'd like to get some feedback from others first. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <mailto:keenonkites@gmx.net> <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Freitag, 25. März 2016 00:13 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, I couldn't go to sleep without getting to the ground of this... I performed the following tests on linux, always with my standard --no-trim present: * original osm.pbf file downloaded from geofabrik (with the problematic bounds tag in it) -> FAILURE * converted with the help of osmconvert to normal osm format -> FAILURE (which was to be expected, same file, just different format) * deleted the osm tag with a simple command like 'cat file.osm | grep -v bounds > file-nobounds.osm' and ran splitter against it, always with the same options -> SUCCESS This leads me to the question why the bounds tag is read/used if it also works without and even gives less problem ? .... sorry for bombarding you (and the list), but I think it's nevertheless and interesting question. Cheers Patrik On 24.03.2016 23:51, KeenOnKites wrote: Gerd, I did dig a bit deeper... also as it rang a bell: we had quite a similar problem with the wrong bounding box in Alaska already October 2014. It was an illegal value of maxlon="180.0005" causing splitter to bail out When I convert the osm.pbf file with the help of osmconvert to osm and look at the first few lines I see a 'bounds' tag announcing the problematic bounding box (not illegal as in 2014, but still 'suboptimal'): <?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="osmconvert 0.8.2" timestamp="2016-03-23T20:13:02Z"> <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794" maxlon="179.9999"/> <node id="27207079" lat="64.7487541" lon="-147.3242821" version="2" ... Getting the statistics via osmconvert with --out-statistics seems to read through the file and checks the 'real' bounding box: ... lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 ... I'm now wondering if splitter get's confused by the existing but obviously strange bounds tag. According to the findings in 2014 splitter can handle files without the bounds tag and just gets the real boundings from all the elements in the file. I did not test to run it through splitter without the bounds tag as I'm having troubles converting the file properly on my windows... but I'll try that probably tomorrow again on linux (sort of late already today). The process would be osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to have same source file format) .... and then run the splitter and see what happens. But if I have a proper osm file with the 'problematic' bounds tag in it, I can also try to reproduce the problem with the osm file. If it is reproducible I just drop the tag and try it again. BTW: I've contacted geofabrik already via email Patrik On 24.03.2016 22:27, KeenOnKites wrote: Gerd, I'll play a bit more with this option and check what suits me best. Again thanks for the incredibly quick answer to my problem/question. Cheers Patrik On 24.03.2016 22:20, Gerd Petermann wrote: Hi Patrik, I think the wanted effect of the no-trim option is a rectangular map, which some people prefer, esp. on the PC. Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:16 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Sorry, your explanations came in during I was writing up the test results ... Think it's all clear so far. As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems... I'll contact geofabrik with the details. Many thanks and happy Easter. Patrik On 24.03.2016 22:07, Gerd Petermann wrote: Hi Patrik, I don't need the files, I downloaded the alaska file and tried some variants. See also my last post, send a few minutes ago. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <mailto:patrik.brunner@gmx.net> <patrik.brunner@gmx.net><mailto:patrik.brunner@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:03 An: <mailto:mkgmap-dev@lists.mkgmap.org.uk> mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Yes, alaska.osm.pbf is small. It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option. Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ? You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ? Thanks already now for your help. Patrik On 24.03.2016 21:14, Gerd Petermann wrote: Ahh, sorry, I just noticed that the file alaska.osm.pbf is small. The problem is here is that the bounding box spans from -180 to 180, but this box is most empty. I have to run splitter now to find the details. It works without --no-trim, probably also with an appropriate polygon file. Does that help? Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <mailto:GPetermann_muenchen@hotmail.com> <mailto:GPetermann_muenchen@hotmail.com> <GPetermann_muenchen@hotmail.com><mailto:GPetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 24. März 2016 21:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hi Patrik, please provide the complete log from splitter and the densities-out.txt Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <mailto:keenonkites@gmx.net> <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 20:25 An: Development list for mkgmap Betreff: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together, I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik. The exception is: Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157) The complete command line with the splitter call is: java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf The pbf source file comes from: <http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf>http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf The osmconvert statistics about that file is: PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung> Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated. Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ? Any ideas are very welcome.... Cheers Patrik _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hmm, seem that the build process hangs again... Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> Gesendet: Samstag, 26. März 2016 08:43 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Many thanks, Gerd. If there is a compiled Version, i can test it this evening.... Patrik Am 26.03.2016 8:38 vorm. schrieb Gerd Petermann <GPetermann_muenchen@hotmail.com>: Hi all, I tried to find out why splitter uses the bounding box provided in the file. The feature was added by Chris Miller with splitter release 102 with this comment: "Added support for the <bounds/> tag in OSM files. Also added a --status-freq parameter to periodically display status information." I searched the archives to find out why this was done but found no hint, can anybody remember what problem he wanted to solve ? For now I've added option --ignore-osm-bounds to splitter r437. See http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=436 and http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=437<http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=436> (sorry for the two changes, I first thought that I cannot use --ignore-osm-bounds as option name) Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net> Gesendet: Freitag, 25. März 2016 13:32 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Good news, Gerd. .. as long as it's not too much thoughts... ;-) Just to give you a heads up regarding the 'source' of the problem: according to GeoFabrik the wrong bounds tag seems to be caused by osmium-history-splitter used to split the world file. Cheers Patrik On 25.03.2016 11:00, Gerd Petermann wrote: Hi Patrik, reg. performance for option 3: I don't expect any extra costs for this, splitter already collects all the data and applies the bbox after that. So, it always creates a (virtual) grid for the whole planet. With normal resolutions this never was a bottle neck. reg. implementation: I don't see any problem for any of the three options, but option 3 may requrie more thoughts. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <patrik.brunner@gmx.net><mailto:patrik.brunner@gmx.net> Gesendet: Freitag, 25. März 2016 10:35 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, yes, it's a complicated world with these non-rectancular countries, can't we change that ? ;-) I did some tests yesterday playing around with --no-trim and --polygon-files with Luxembourge (small enough to do quick tests), the result is not that much different if I'm skipping --no-trim or use the polygon file, the result ist still 'quite' rectancular, just a few bits are really left aside.... but I will have to do more tests with other countries (more complicated ones and bigger ones than LUX) to see if in my case I could/should skip --no-trim for building other maps too. Regarding your implementation option: * I would rather prefer not having option 2) implemented for the same reason as you mention: it changes the default behaviour for everyone. And as we have to say that for (rough guess) 99% of the cases the actual behaviour is working that's too much. * I like option 3). But enabling that by default would possibly increase the time used to split as it first has to run through the file to see which bounds (bounds tag or calculated one) have to be used... but I'm sure you could answer that much better than me... ;-) * Option 1) is possibly the cheapest to implement ? Thanks Patrik On 25.03.2016 07:33, Gerd Petermann wrote: Hi Patrik, good question. I also wondered if splitter can be changed to ignore wrong bounds tags. My thoughts: In your case the bbox is far too large, means it claims to contain data that is not there. The normal case is vice versa: The file contains some nodes outside of the bbox, maybe from ferry lines or other long ways, and my understanding is that we don't want to calculate the tiles based on those few nodes, since we might get a map with larger empty areas at the "border". No idea if that would cause more trouble. With typical data from geofabrik and the --no-trim option there will always be large empty areas as most countries are not rectangular ;-) Possible changes in the code: 1) add a --ignore-osm-bounds option and set the default to false (means one gets the same result like now when he uses the default) 2) add a --use-osm-bounds option and set the default to false (means one might get a different result when using the default) 3) add code to check how the collected data matches the given bounds, use whatever is smaller. This might also be triggered by an option if needed. I'd like to get some feedback from others first. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <mailto:keenonkites@gmx.net> <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Freitag, 25. März 2016 00:13 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, I couldn't go to sleep without getting to the ground of this... I performed the following tests on linux, always with my standard --no-trim present: * original osm.pbf file downloaded from geofabrik (with the problematic bounds tag in it) -> FAILURE * converted with the help of osmconvert to normal osm format -> FAILURE (which was to be expected, same file, just different format) * deleted the osm tag with a simple command like 'cat file.osm | grep -v bounds > file-nobounds.osm' and ran splitter against it, always with the same options -> SUCCESS This leads me to the question why the bounds tag is read/used if it also works without and even gives less problem ? .... sorry for bombarding you (and the list), but I think it's nevertheless and interesting question. Cheers Patrik On 24.03.2016 23:51, KeenOnKites wrote: Gerd, I did dig a bit deeper... also as it rang a bell: we had quite a similar problem with the wrong bounding box in Alaska already October 2014. It was an illegal value of maxlon="180.0005" causing splitter to bail out When I convert the osm.pbf file with the help of osmconvert to osm and look at the first few lines I see a 'bounds' tag announcing the problematic bounding box (not illegal as in 2014, but still 'suboptimal'): <?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="osmconvert 0.8.2" timestamp="2016-03-23T20:13:02Z"> <bounds minlat="49.8089" minlon="-179.9532" maxlat="73.79794" maxlon="179.9999"/> <node id="27207079" lat="64.7487541" lon="-147.3242821" version="2" ... Getting the statistics via osmconvert with --out-statistics seems to read through the file and checks the 'real' bounding box: ... lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 ... I'm now wondering if splitter get's confused by the existing but obviously strange bounds tag. According to the findings in 2014 splitter can handle files without the bounds tag and just gets the real boundings from all the elements in the file. I did not test to run it through splitter without the bounds tag as I'm having troubles converting the file properly on my windows... but I'll try that probably tomorrow again on linux (sort of late already today). The process would be osm.pbf -> osm -> get rid of the bounds tag -> back to osm.pbf again (to have same source file format) .... and then run the splitter and see what happens. But if I have a proper osm file with the 'problematic' bounds tag in it, I can also try to reproduce the problem with the osm file. If it is reproducible I just drop the tag and try it again. BTW: I've contacted geofabrik already via email Patrik On 24.03.2016 22:27, KeenOnKites wrote: Gerd, I'll play a bit more with this option and check what suits me best. Again thanks for the incredibly quick answer to my problem/question. Cheers Patrik On 24.03.2016 22:20, Gerd Petermann wrote: Hi Patrik, I think the wanted effect of the no-trim option is a rectangular map, which some people prefer, esp. on the PC. Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:16 An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Sorry, your explanations came in during I was writing up the test results ... Think it's all clear so far. As we're creating lot of different maps I'm just wondering if I can/should drop the option --no-trim for all map building or if I would suddenly run into other/new problems... I'll contact geofabrik with the details. Many thanks and happy Easter. Patrik On 24.03.2016 22:07, Gerd Petermann wrote: Hi Patrik, I don't need the files, I downloaded the alaska file and tried some variants. See also my last post, send a few minutes ago. Gerd ________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Patrik Brunner <mailto:patrik.brunner@gmx.net> <patrik.brunner@gmx.net><mailto:patrik.brunner@gmx.net> Gesendet: Donnerstag, 24. März 2016 22:03 An: <mailto:mkgmap-dev@lists.mkgmap.org.uk> mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Gerd, Yes, alaska.osm.pbf is small. It works without --no-trim. And it also works with the polygon file that belongs to alaska.osm.pbf (also downloaded from Geofabrik) which, according to documentation, anyway would disable --no-trim option. Do you still need the resulting densities-out of the failure ? It's about 700 kb... if yes, how should I provide it ? Just attach it here ? You have to excuse my question, I'm not the crack: is this now a problem of the pbf file, or the splitter ? ... or just the way I try to use it ? Thanks already now for your help. Patrik On 24.03.2016 21:14, Gerd Petermann wrote: Ahh, sorry, I just noticed that the file alaska.osm.pbf is small. The problem is here is that the bounding box spans from -180 to 180, but this box is most empty. I have to run splitter now to find the details. It works without --no-trim, probably also with an appropriate polygon file. Does that help? Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <mailto:GPetermann_muenchen@hotmail.com> <mailto:GPetermann_muenchen@hotmail.com> <GPetermann_muenchen@hotmail.com><mailto:GPetermann_muenchen@hotmail.com> Gesendet: Donnerstag, 24. März 2016 21:01 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hi Patrik, please provide the complete log from splitter and the densities-out.txt Gerd ________________________________ Von: mkgmap-dev <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> <mkgmap-dev-bounces@lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von KeenOnKites <mailto:keenonkites@gmx.net> <keenonkites@gmx.net><mailto:keenonkites@gmx.net> Gesendet: Donnerstag, 24. März 2016 20:25 An: Development list for mkgmap Betreff: [mkgmap-dev] Problem with splitter: Failed to find a correct split Hello together, I'm running into a problem with the splitter (r435 aswell as r427) when splitting the US_ALASKA file downloadable from Geofabrik. The exception is: Warning: No solution found for partition (49.7900390625,-179.9560546875) to (73.828125,180.0) with 6'702'717 nodes uk.me.parabola.splitter.SplitFailedException: Failed to find a correct split at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:152) at uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:196) at uk.me.parabola.splitter.Main.calculateAreas(Main.java:645) at uk.me.parabola.splitter.Main.split(Main.java:258) at uk.me.parabola.splitter.Main.start(Main.java:187) at uk.me.parabola.splitter.Main.main(Main.java:157) The complete command line with the splitter call is: java -Xmx1536M -jar D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/tools/splitter/splitter.jar --max-threads=2 --geonames-file=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/cities/c ities15000.zip --no-trim --precomp-sea=D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/sea --keep-complete=true --mapid=98200001 --max-nodes=800000 --output=xml --output-dir=D:/fzk/develop/fzk -mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA D:/fzk/develop/fzk-mde-garmin/Freizeitkarte-Entwicklung/work/Freizeitkarte_US_ALASKA/Freizeitkarte_US_ALASKA.osm.pbf The pbf source file comes from: <http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf>http://download.geofabrik.de/north-america/us/alaska-latest.osm.pbf The osmconvert statistics about that file is: PS Freizeitkarte-Entwicklung> .\tools\osmconvert\windows\osmconvert.exe .\work\Freizeitkarte_US_ALASKA\Freizeitkarte_US_ALASKA.osm.pbf --out-statistics timestamp min: 2007-06-05T03:23:59Z timestamp max: 2016-03-23T05:41:43Z lon min: -180.0000000 lon max: -122.5122525 lat min: 48.6234931 lat max: 71.6061501 nodes: 4360214 ways: 185550 relations: 2245 node id min: 27207079 node id max: 4072166815 way id min: 4708608 way id max: 404980503 relation id min: 13971 relation id max: 6033189 keyval pairs max: 310 keyval pairs max object: relation 60189 noderefs max: 2000 noderefs max object: way 42394334 relrefs max: 681 relrefs max object: relation 3337277 PS Freizeitkarte-Entwicklung> Interesting to mention: - splitter exception mentiones a complete different coverage area than osmconvert statistics. - the area is near -180 / +180... always complicated. Do I miss something ? All other pbf's I've tried are splitting properly without any problems. Do I need to change something in the arguments ? Or is it a simple problem of the actual pbf file ? Any ideas are very welcome.... Cheers Patrik _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Steve, could it be that the process is "unhappy" with two or more commits within a short time ? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Samstag, 26. März 2016 10:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split On 26/03/16 08:53, Gerd Petermann wrote:

Hmm, seem that the build process hangs again...

Re-started and the r437 build is now available. ..Steve _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

this was done with australia-latest unshure if it looks better or not Steve Australia This email has been sent from a virus-free computer protected by Avast. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Sat, Mar 26, 2016 at 7:21 PM, Gerd Petermann < GPetermann_muenchen@hotmail.com> wrote:

Hi Steve,

could it be that the process is "unhappy" with two or more commits within a short time ?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Samstag, 26. März 2016 10:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

On 26/03/16 08:53, Gerd Petermann wrote:

...

Hmm, seem that the build process hangs again...

Re-started and the r437 build is now available.

..Steve

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

does not look different to me , on the australia latest file Stephen australia On Sat, Mar 26, 2016 at 10:29 PM, Gerd Petermann < GPetermann_muenchen@hotmail.com> wrote:

Steve,

Why didn't you use the new option?

Gerd

------------------------------ *Von:* mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Sgalowski <steve.sgalowski@gmail.com> *Gesendet:* Samstag, 26. März 2016 11:18 *An:* Development list for mkgmap *Betreff:* Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

this was done with australia-latest unshure if it looks better or not

Steve Australia

This email has been sent from a virus-free computer protected by Avast. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#m_-2588138055278704449_DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Sat, Mar 26, 2016 at 7:21 PM, Gerd Petermann < GPetermann_muenchen@hotmail.com> wrote:

...

Hi Steve,

could it be that the process is "unhappy" with two or more commits within a short time ?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Samstag, 26. März 2016 10:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

On 26/03/16 08:53, Gerd Petermann wrote:

...

Hmm, seem that the build process hangs again...

Re-started and the r437 build is now available.

..Steve

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Gerd, I found the built version and tested it against the same testfile I had problems with a few days ago. All the problemcases I had (building Alaska without contour data, building with contour data) run through without any problems and the built map of Alaska looks so far ok. I'll do some more tests tomorrow... I'm also interested in building map for Bremen without the new option and with, to compare the result. But this has to wait till tomorrow. Thanks for the quick implementation, Gerd. Cheers Patrik On 26.03.2016 10:21, Gerd Petermann wrote:

Hi Steve,

could it be that the process is "unhappy" with two or more commits within a short time ?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Steve Ratcliffe <steve@parabola.me.uk> Gesendet: Samstag, 26. März 2016 10:17 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem with splitter: Failed to find a correct split

On 26/03/16 08:53, Gerd Petermann wrote:

...

Hmm, seem that the build process hangs again... Re-started and the r437 build is now available.

..Steve

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev