splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis. When I call splitter on that file, I get the error message: Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=2000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints. Kind regards, Bernhard
Hi Bernhard, splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-( Do you have an idea where this id comes from? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil@gmx.de> Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis. When I call splitter on that file, I get the error message: Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=2000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints. Kind regards, Bernhard _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, I am investigation the origin of that id. I tried to run splitter on the file generated by srtm2osm. After half an hour, I got following exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=3000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Node ids are not sorted. Use e.g. osmosis to sort the input data. This is not supported with keep-complete=true or --problem-list uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) which is OK, because srtm2osm does not write its output in some sorted way. Next, I tried to run splitter on the file produced by osmosis after sorting. Now, it did not take long to get the exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=3000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) That is, the wrong node id exists in the file of sorted elevation data. I cannot find out if osmosis or srtm2osm produced that node id. Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann:
Hi Bernhard,
splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-(
Do you have an idea where this id comes from?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil@gmx.de> Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis.
When I call splitter on that file, I get the error message:
Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=ue or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79)
The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description=ernieMap Bike Thai" --mapidG120001 --max-nodes 00000 --max-threads=4 --write-kml=slitter.kml --no-trim 1>splitter.log
Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints.
Kind regards, Bernhard _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On Mon, Oct 29, Bernhard Hiller wrote:
Hi Gerd, I am investigation the origin of that id. I tried to run splitter on the file generated by srtm2osm.
Try phyghtmap. In my experience, that works much better than srtm2osm. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
Hi Bernhard, I would not say that the error cannot be in the file produced by srtm2osm. The bad id might come after splitter complains about the wrong order. You may try osmconvert with --out-statistics to check the file. But be careful, current 64 Bit Windows version of osmconvert seems to be buggy, make sure to use the one from the english wiki page with large file support. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil@gmx.de> Gesendet: Montag, 29. Oktober 2018 19:16:31 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi Gerd, I am investigation the origin of that id. I tried to run splitter on the file generated by srtm2osm. After half an hour, I got following exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=3000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Node ids are not sorted. Use e.g. osmosis to sort the input data. This is not supported with keep-complete=true or --problem-list uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) which is OK, because srtm2osm does not write its output in some sorted way. Next, I tried to run splitter on the file produced by osmosis after sorting. Now, it did not take long to get the exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=3000000 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) That is, the wrong node id exists in the file of sorted elevation data. I cannot find out if osmosis or srtm2osm produced that node id. Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann:
Hi Bernhard,
splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-(
Do you have an idea where this id comes from?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil@gmx.de> Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis.
When I call splitter on that file, I get the error message:
Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=ue or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79)
The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description=ernieMap Bike Thai" --mapidG120001 --max-nodes 00000 --max-threads=4 --write-kml=slitter.kml --no-trim 1>splitter.log
Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints.
Kind regards, Bernhard _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, Please find enclosed file "splitter.log" which shows node ids in that region: 10.000.000 nodes parsed... id=9223372036671913720 splitter crashes after the entry 190.000.000 nodes parsed... id=9223372036851913720 The source code of srtm2osm at https://svn.openstreetmap.org/applications/utils/Srtm2Osm/trunk/Srtm2Osm/Srt... shows that private long firstNodeId = long.MaxValue; is the default value, the source of the IdCounter shows that it does decrement for the further values by default. => The value comes from the generation of contour lines via srtm2osm. By the way, also firstWayId is initialized to long.MaxValue. Would that also cause problems with splitter? Strange that this fact did not cause problems in February, when I created a map of Taiwan with DEM and elevation contour lines. Perhaps a bounding box in an osmosis command happened to remove this node id by chance. So, I guess, we have a solution. The bad thing is: I'll have to re-run the generation of the contourlines file with srtm2osm (with additional parameters for first node and way ids), which takes several hours, plus 1 hour of sorting, .... Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann:
Hi Bernhard,
splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-(
Do you have an idea where this id comes from?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil@gmx.de> Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis.
When I call splitter on that file, I get the error message:
Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=ue or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79)
The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description=ernieMap Bike Thai" --mapidG120001 --max-nodes 00000 --max-threads=4 --write-kml=slitter.kml --no-trim 1>splitter.log
Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints.
Kind regards, Bernhard _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (3)
-
Bernhard Hiller -
Gerd Petermann -
Thorsten Kukuk