On Tue, 20 Sep 2011 22:14:36 +0200, Michael Prinzing wrote:

On Mon, 19 Sep 2011 17:53:03 +0200, Michael Prinzing wrote:

...

there seems to be a problem with the latest splitter and data in pbf format if it was created with osmosis 0.39.

After some further testing, it now looks like a problem with osmosis only. If I am converting the pbf data created with osmosis 0.39 back to xml then there also is an empty bounding box that did not exist in the original data and that does not arise if the pbf file was built with osmosis 0.38.

I will report this on the mailing list for osmosis.

Well, there really was a problem with osmosis and it is fixed in the latest osmosis-snapshot. This version now can read pbf data created with osmosis 0.39 or newer without seeing an empty bounding box where no one is present. However, the bug was in the pbf reader, not with the creation of pbf files. This means the created pbf data did not change, and therefore the problem with the splitter still persists. Igor Podolskiy who fixed the problem in osmosis also looked into the splitter code. He assumes there could be something similar wrong as it was in osmosis. Please see the thread beginning with http://lists.openstreetmap.org/pipermail/osmosis-dev/2011-September/001125.h... for details. Here again how to reproduce the problem with the splitter: I am saving the following xml data (not containing a bounding box!) to data.osm: -------------------------------------------------------------------- <?xml version='1.0' encoding='UTF-8'?> <osm version="0.6" generator="Osmosis 0.35"> <node id="0000004406" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.6325833" lon="6.2575833"/> <node id="0000004407" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.6334167" lon="6.2584167"/> <node id="0000004408" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.63425" lon="6.25925"/> <node id="0000004409" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.6334167" lon="6.2600833"/> <node id="0000004410" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.6325833" lon="6.2600833"/> <node id="0000004411" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.63175" lon="6.25925"/> <node id="0000004412" version="1" timestamp="2011-09-11T03:07:14Z" lat="51.63175" lon="6.2584167"/> <way id="1" version="1" timestamp="2011-09-11T03:07:14Z"> <nd ref="0000004406"/> <nd ref="0000004407"/> <nd ref="0000004408"/> <nd ref="0000004409"/> <nd ref="0000004410"/> <nd ref="0000004411"/> <nd ref="0000004412"/> <nd ref="0000004406"/> <tag k="ele" v="-220"/> <tag k="contour" v="elevation"/> <tag k="contour_ext" v="elevation_medium"/> </way> </osm> -------------------------------------------------------------------- Then, I am converting this xml data to pbf with: osmosis --read-xml file="data.osm" --write-pbf omitmetadata=true file="data_nnn.osm.pbf" Depending on the osmosis version I am using for this, I am getting the following: With osmosis 0.39: -------------------------------------------------------------------- 00000000: 0000 000d 0a09 4f53 4d48 6561 6465 7218 ......OSMHeader. 00000010: 2f10 231a 2b78 9c53 e2f3 2fce 0d4e ce48 /.#.+x.S../..N.H 00000020: cd4d d40d 33d0 3353 e272 49cd 2b4e f5cb .M..3.3S.rI.+N.. 00000030: 4f49 2d6e 6264 31d0 33b6 0400 bcf0 0a2d OI-nbd1.3......- 00000040: 0000 000c 0a07 4f53 4d44 6174 6118 8c01 ......OSMData... 00000050: 109f 011a 8601 789c e372 e062 e012 48cd ......x..r.b..H. 00000060: 492d 4b2c c9cc cf8b cf4d 4dc9 2ccd e562 I-K,.....MM.,..b 00000070: 068a 70b1 27e7 e795 e497 1671 7143 19f1 ..p.'......qqC.. 00000080: a915 255c 2cba 4646 065c 9c70 2d42 9642 ..%\,.FF.\.p-B.B 00000090: e65c 1c6f 5c98 c0c0 4974 52fb 9637 cc73 .\.o\...ItR..7.s 000000a0: 9a18 6735 31ce 6c62 9c0d 2619 bc44 365d ..g51.lb..&..D6] 000000b0: be66 0d11 0522 8699 6061 2149 2971 0e46 .f..."..`a!I)q.F 000000c0: 2166 2666 1629 6656 3646 274e 9831 dc1d !f&f.)fV6F'N.1.. 000000d0: 8c29 1318 5fb0 0300 fd3e 2ca3 .).._....>,. -------------------------------------------------------------------- With osmosis-snapshot: -------------------------------------------------------------------- 00000000: 0000 000d 0a09 4f53 4d48 6561 6465 7218 ......OSMHeader. 00000010: 3b10 2f1a 3778 9c53 e2f3 2fce 0d4e ce48 ;./.7x.S../..N.H 00000020: cd4d d40d 33d0 3353 e272 49cd 2b4e f5cb .M..3.3S.rI.+N.. 00000030: 4f49 2d6e 6214 08f6 730c 08f6 f00f d12d OI-nb...s......- 00000040: cdcb cecb 2fcf 0300 5893 0f1c 0000 000c ..../...X....... 00000050: 0a07 4f53 4d44 6174 6118 8c01 109f 011a ..OSMData....... 00000060: 8601 789c e372 e062 e012 48cd 492d 4b2c ..x..r.b..H.I-K, 00000070: c9cc cf8b cf4d 4dc9 2ccd e562 068a 70b1 .....MM.,..b..p. 00000080: 27e7 e795 e497 1671 7143 19f1 a915 255c '......qqC....%\ 00000090: 2cba 4646 065c 9c70 2d42 9642 e65c 1c6f ,.FF.\.p-B.B.\.o 000000a0: 5c98 c0c0 4974 52fb 9637 cc73 9a18 6735 \...ItR..7.s..g5 000000b0: 31ce 6c62 9c0d 2619 bc44 365d be66 0d11 1.lb..&..D6].f.. 000000c0: 0522 8699 6061 2149 2971 0e46 2166 2666 ."..`a!I)q.F!f&f 000000d0: 1629 6656 3646 274e 9831 dc1d 8c29 1318 .)fV6F'N.1...).. 000000e0: 5fb0 0300 fd3e 2ca3 _....>,. -------------------------------------------------------------------- Both files can be converted back to xml only with the latest osmosis from snapshot. osmosis 0.39 or older are reading an empty bounding box from the pbf data. This was the bug that was fixed in osmosis now. When I am trying to split one of these files, the splitter sees an empty bounding box and of course does not write any data to the output: -------------------------------------------------------------------- cache= description= geonames-file= legacy-mode=false mapid=63240001 max-areas=255 max-nodes=1600000 max-threads=1 (auto) mixed=false no-trim=false output=unset output-dir= overlap=2000 resolution=13 split-file= status-freq=120 write-kml= Elapsed time: 0s Memory: Current 15MB (1MB used, 14MB free) Max 247MB Time started: Fri Sep 23 23:27:55 CEST 2011 Map is being split for resolution 13: - area boundaries are aligned to 0x800 map units - areas are multiples of 0x1000 map units wide and high Processing data_039.osm.pbf Bounding box 0.0 0.0 0.0 0.0 in 1 file Time: Fri Sep 23 23:27:56 CEST 2011 Exact map coverage is (0.0,0.0) to (2.1457672119140625E-5,2.1457672119140625E-5) Trimmed and rounded map coverage is (0.0,0.0) to (2.1457672119140625E-5,2.1457672119140625E-5) Splitting nodes into areas containing a maximum of 1.600.000 nodes each... 0 areas: Writing out split osm files Fri Sep 23 23:27:56 CEST 2011 Processing 0 areas in a single pass Time finished: Fri Sep 23 23:27:56 CEST 2011 Total time taken: 0s -------------------------------------------------------------------- If however I am creating the pbf file with osmosis 0.38, then the pbf data looks like the following: -------------------------------------------------------------------- 00000000: 0000 000c 0a07 4f53 4d44 6174 6118 8c01 ......OSMData... 00000010: 109f 011a 8601 789c e372 e062 e012 48cd ......x..r.b..H. 00000020: 492d 4b2c c9cc cf8b cf4d 4dc9 2ccd e562 I-K,.....MM.,..b 00000030: 068a 70b1 27e7 e795 e497 1671 7143 19f1 ..p.'......qqC.. 00000040: a915 255c 2cba 4646 065c 9c70 2d42 9642 ..%\,.FF.\.p-B.B 00000050: e65c 1c6f 5c98 c0c0 4974 52fb 9637 cc73 .\.o\...ItR..7.s 00000060: 9a18 6735 31ce 6c62 9c0d 2619 bc44 365d ..g51.lb..&..D6] 00000070: be66 0d11 0522 8699 6061 2149 2971 0e46 .f..."..`a!I)q.F 00000080: 2166 2666 1629 6656 3646 274e 9831 dc1d !f&f.)fV6F'N.1.. 00000090: 8c29 1318 5fb0 0300 fd3e 2ca3 .).._....>,. -------------------------------------------------------------------- The obvious difference is that there is no "OSMHeader" included. This file can be processed by the splitter without an empty bounding box showing up. Because of this the problem with the splitter does not occur with pbf files created with osmosis 0.38 or older. Could please someone look into the splitter code and check if the corrections that were done to osmosis have to by applied to the splitter, too? Thank you, Michael