some questions about splitter and europe extract
Hi, In order to process my routable cyclemap of the Benelux countries [1], I have to split the osm europe extract from geofabrik into 45 tiles [2]. I noticed last month that the time needed to split the europe file considerably slowed down from 2hours processing time beginning september to 7 hours (!) from yesterdays extract. Last week it took 5 hours. I also noticed that geofabrik is recently producing the extracts in another format (protobuf binary format), the osm.bz2 format comes available almost 12 hours later. The arguments I use to split the europe file are: java -Xmx1024m -jar splitter.jar --split-file=areas_europe.list --overlap=3000 europe.osm.bz2 Im using splitter v110 on a windows laptop (4mb internal memory) Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter? Cheers, Minko [1] http://sites.google.com/site/openfietsmap [2] http://mijndev.openstreetmap.nl/~ligfietser/openfietsmap/Scripts/areas_europ...
Hello, I think, it is better, you cut the europe-extract first with osmosis and gave only this file to the splitter. osmosis can also read and write pbf. You'll need v0.37 and the optioen --read-pbf and --write-pbf instead of --read-xml and --write-xml. Till now, splitter doesn't read the new format, so you have to input *.osm data. aighes -- View this message in context: http://gis.638310.n2.nabble.com/some-questions-about-splitter-and-europe-ext... Sent from the Mkgmap Development mailing list archive at Nabble.com.
El 09/10/10 11:54, aighes escribió:
Hello, I think, it is better, you cut the europe-extract first with osmosis and gave only this file to the splitter.
I have not used this method recently, but in the past I found cutting Europe extract with a polygon using osmosis needed much longer than cutting it with splitter, using a predefined areas.list. Could you explain how do you use osmosis to cut the Europe extract?
osmosis can also read and write pbf. You'll need v0.37 and the optioen --read-pbf and --write-pbf instead of --read-xml and --write-xml.
Till now, splitter doesn't read the new format, so you have to input *.osm data.
aighes
Hi I use the following command with osmosis v0.37. This Version can read the new binary format. set java=C:\Programme\Java\jre6\bin\java.exe -Xmx7000M set osmosis_file=D:\OpenStreetMap\mkgmap\bin\osmosis %java% -cp "%osmosis_file%\lib\default\plexus-classworlds-2.2.2.jar" -Dapp.home="%osmosis_file%" -Dclassworlds.conf="%osmosis_file%\config\plexus.conf" org.codehaus.classworlds.Launcher --read-pbf data\europe.osm.pbf --bounding-box bottom=49.4 left=2.4 top=54.9 right=7.4 clipIncompleteEntities=yes idTrackerType=BitSet --write-xml file=data\BeNeLux.osm %* Instead of --bounding-box bottom=49.4 left=2.4 top=54.9 right=7.4 clipIncompleteEntities=yes idTrackerType=BitSet you can alos use --bounding-polygon file=BeNeLux.poly. -- View this message in context: http://gis.638310.n2.nabble.com/some-questions-about-splitter-and-europe-ext... Sent from the Mkgmap Development mailing list archive at Nabble.com.
I have to edit my previous Mail: I use osmosis for cutting 9 maps in one pass. This takes not much more time then cutting just one map. Maybe splitter is faster if you need just one map. aighes -- View this message in context: http://gis.638310.n2.nabble.com/some-questions-about-splitter-and-europe-ext... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Do you need --write-xml or --write-pbf? I used the last one, but when I splitted the output file benelux.osm I got this error: "Error parsing from xml from file org.xmlpullv1.xmlpullparserexception: only whitespace content allowed before start tag and not /u0" aighes wrote: Hi I use the following command with osmosis v0.37. This Version can read the new binary format. set java=C:\Programme\Java\jre6\bin\java.exe -Xmx7000M set osmosis_file=D:\OpenStreetMap\mkgmap\bin\osmosis %java% -cp "%osmosis_file%\lib\default\plexus-classworlds-2.2.2.jar" -Dapp.home="%osmosis_file%" -Dclassworlds.conf="%osmosis_file%\config\plexus.conf" org.codehaus.classworlds.Launcher --read-pbf data\europe.osm.pbf --bounding-box bottom=49.4 left=2.4 top=54.9 right=7.4 clipIncompleteEntities=yes idTrackerType=BitSet --write-xml file=data\BeNeLux.osm %* Instead of --bounding-box bottom=49.4 left=2.4 top=54.9 right=7.4 clipIncompleteEntities=yes idTrackerType=BitSet you can alos use --bounding-polygon file=BeNeLux.poly. -- View this message in context: http://gis.638310.n2.nabble.com/some-questions-about-splitter-and-europe-ext... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Minko-2 wrote:
Do you need --write-xml or --write-pbf?
I used the last one, but when I splitted the output file benelux.osm I got this error: "Error parsing from xml from file org.xmlpullv1.xmlpullparserexception: only whitespace content allowed before start tag and not /u0"
Hi, splitter r123 only reads osm-xml format. So you have to use in osmosis --write-xml. I don't know, when a new version of splitter will be released, but I hope it will be soon. Because diskspace will be reduced enormously, is you can use the binary format. aighes -- View this message in context: http://gis.638310.n2.nabble.com/some-questions-about-splitter-and-europe-ext... Sent from the Mkgmap Development mailing list archive at Nabble.com.
Thanks Aighes, It now worked, splitting time was much faster (about 1 hr to extract the europe.pbf with osmosis and another hour to split the benelux.osm file). Cheers, Minko Aighes wrote: Hi, splitter r123 only reads osm-xml format. So you have to use in osmosis --write-xml. I don't know, when a new version of splitter will be released, but I hope it will be soon. Because diskspace will be reduced enormously, is you can use the binary format. aighes
Am 09.10.2010 10:27, schrieb Minko:
Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter?
There is already a branch of the splitter which is able to read the pbf format (branches/crosby_integration). I build and successfully tested it last week with the europe extract from Geofabrik. It is much faster than using the bz2 file, time went down from ~5 hours + time for extraction of the bz2 file to less than 4 hours. I hope there is soon an "official" build of the splitter with pbf support. Perhaps it was not merged because of additional dependencies.
Cheers, Minko
[1] http://sites.google.com/site/openfietsmap [2] http://mijndev.openstreetmap.nl/~ligfietser/openfietsmap/Scripts/areas_europ...
El 10/10/10 19:55, Ronny Klier escribió:
Am 09.10.2010 10:27, schrieb Minko:
Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter?
There is already a branch of the splitter which is able to read the pbf format (branches/crosby_integration). I build and successfully tested it last week with the europe extract from Geofabrik. It is much faster than using the bz2 file, time went down from ~5 hours + time for extraction of the bz2 file to less than 4 hours.
I hope there is soon an "official" build of the splitter with pbf support. Perhaps it was not merged because of additional dependencies.
Trying to compile pbf branch I get the error below. How can I get missing packages to be able to compile? Buildfile: /home/carlos/Paquetes/splitter_pbf/build.xml prepare: [mkdir] Created dir: /home/carlos/Paquetes/splitter_pbf/build/classes [mkdir] Created dir: /home/carlos/Paquetes/splitter_pbf/build/test-classes [mkdir] Created dir: /home/carlos/Paquetes/splitter_pbf/build/test-output compile: [javac] /home/carlos/Paquetes/splitter_pbf/build.xml:68: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds [javac] Compiling 56 source files to /home/carlos/Paquetes/splitter_pbf/build/classes [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:6: package crosby.binary does not exist [javac] import crosby.binary.BinaryParser; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:7: package crosby.binary does not exist [javac] import crosby.binary.Osmformat; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:9: cannot find symbol [javac] symbol: class BinaryParser [javac] public class BinaryMapParser extends BinaryParser { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:31: package Osmformat does not exist [javac] protected void parseDense(Osmformat.DenseNodes nodes) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:55: package Osmformat does not exist [javac] protected void parseNodes(List<Osmformat.Node> nodes) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:71: package Osmformat does not exist [javac] protected void parseWays(List<Osmformat.Way> ways) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:90: package Osmformat does not exist [javac] protected void parseRelations(List<Osmformat.Relation> rels) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:122: package Osmformat does not exist [javac] public void parse(Osmformat.HeaderBlock block) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Way.java:15: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.IntArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Way.java:21: cannot find symbol [javac] symbol : class IntArrayList [javac] location: class uk.me.parabola.splitter.Way [javac] private final IntArrayList refs = new IntArrayList(10); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Way.java:37: cannot find symbol [javac] symbol : class IntArrayList [javac] location: class uk.me.parabola.splitter.Way [javac] public IntArrayList getRefs() { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:5: package it.unimi.dsi.bits does not exist [javac] import it.unimi.dsi.bits.Fast; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:6: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortFunction; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:7: package it.unimi.dsi.fastutil.longs does not exist [javac] import it.unimi.dsi.fastutil.longs.LongArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:8: package it.unimi.dsi.fastutil.objects does not exist [javac] import it.unimi.dsi.fastutil.objects.ObjectArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:9: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:11: cannot find symbol [javac] symbol: class Int2ShortFunction [javac] public class DenseInt2ShortMap implements Int2ShortFunction { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:17: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.DenseInt2ShortMap [javac] ObjectArrayList<short[]> valschunks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Main.java:38: package crosby.binary.file does not exist [javac] import crosby.binary.file.BlockInputStream; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/OSMWriter.java:16: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.IntArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:3: package it.unimi.dsi.bits does not exist [javac] import it.unimi.dsi.bits.Fast; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:4: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortFunction; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:5: package it.unimi.dsi.fastutil.longs does not exist [javac] import it.unimi.dsi.fastutil.longs.LongArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:6: package it.unimi.dsi.fastutil.objects does not exist [javac] import it.unimi.dsi.fastutil.objects.ObjectArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:7: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:9: cannot find symbol [javac] symbol: class Int2ShortFunction [javac] public class SparseInt2ShortMap implements Int2ShortFunction { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:15: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] ObjectArrayList<ShortArrayList> valschunks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:15: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] ObjectArrayList<ShortArrayList> valschunks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:16: cannot find symbol [javac] symbol : class LongArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] LongArrayList bitmasks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:5: package it.unimi.dsi.bits does not exist [javac] import it.unimi.dsi.bits.Fast; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:6: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortFunction; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:7: package it.unimi.dsi.fastutil.longs does not exist [javac] import it.unimi.dsi.fastutil.longs.LongArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:8: package it.unimi.dsi.fastutil.objects does not exist [javac] import it.unimi.dsi.fastutil.objects.ObjectArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:9: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:11: cannot find symbol [javac] symbol: class Int2ShortFunction [javac] public class SparseInt2ShortMapInline implements Int2ShortFunction { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:17: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] ObjectArrayList<short []> valschunks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:18: cannot find symbol [javac] symbol : class LongArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] LongArrayList bitmasks; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:67: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] void chunkAdd(ShortArrayList arraylist, int index, short val) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:70: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] void chunkSet(ShortArrayList arraylist, int index, short val) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:73: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] short chunkGet(ShortArrayList arraylist, int index) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:76: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] void chunkRem(ShortArrayList arraylist, int index) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:3: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortFunction; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:4: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortMap; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:5: package it.unimi.dsi.fastutil.ints does not exist [javac] import it.unimi.dsi.fastutil.ints.Int2ShortOpenHashMap; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:6: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortArrayList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:7: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortList; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:8: package it.unimi.dsi.fastutil.shorts does not exist [javac] import it.unimi.dsi.fastutil.shorts.ShortLists; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:21: cannot find symbol [javac] symbol : class ShortList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMultiMap [javac] public ShortList get(int key) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:46: cannot find symbol [javac] symbol : class Int2ShortFunction [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMultiMap.Inner [javac] final Int2ShortFunction map; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:85: cannot find symbol [javac] symbol : class ShortList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMultiMap.Inner [javac] public ShortList get(int key) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMultiMap.java:102: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMultiMap.Inner [javac] public void addTo(int key, ShortArrayList out) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:36: operator + cannot be applied to Osmformat.DenseNodes.getLat,long [javac] long lat = nodes.getLat(i)+last_lat; last_lat = lat; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:36: incompatible types [javac] found : <nulltype> [javac] required: long [javac] long lat = nodes.getLat(i)+last_lat; last_lat = lat; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:37: operator + cannot be applied to Osmformat.DenseNodes.getLon,long [javac] long lon = nodes.getLon(i)+last_lon; last_lon = lon; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:37: incompatible types [javac] found : <nulltype> [javac] required: long [javac] long lon = nodes.getLon(i)+last_lon; last_lon = lon; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:38: operator + cannot be applied to Osmformat.DenseNodes.getId,long [javac] long id = nodes.getId(i)+last_id; last_id = id; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:38: incompatible types [javac] found : <nulltype> [javac] required: long [javac] long id = nodes.getId(i)+last_id; last_id = id; [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:39: cannot find symbol [javac] symbol : method parseLat(long) [javac] location: class uk.me.parabola.splitter.BinaryMapParser [javac] double latf = parseLat(lat), lonf = parseLon(lon); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:39: cannot find symbol [javac] symbol : method parseLon(long) [javac] location: class uk.me.parabola.splitter.BinaryMapParser [javac] double latf = parseLat(lat), lonf = parseLon(lon); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:46: cannot find symbol [javac] symbol : method getStringById(int) [javac] location: class uk.me.parabola.splitter.BinaryMapParser [javac] tmp.addTag(getStringById(keyid),getStringById(valid)); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:46: cannot find symbol [javac] symbol : method getStringById(int) [javac] location: class uk.me.parabola.splitter.BinaryMapParser [javac] tmp.addTag(getStringById(keyid),getStringById(valid)); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:56: package Osmformat does not exist [javac] for (Osmformat.Node i : nodes) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:72: package Osmformat does not exist [javac] for (Osmformat.Way i : ways) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:91: package Osmformat does not exist [javac] for (Osmformat.Relation i : rels) { [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:101: operator + cannot be applied to long,Osmformat.Relation.getMemids [javac] long mid = last_mid + i.getMemids(j); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:101: incompatible types [javac] found : <nulltype> [javac] required: long [javac] long mid = last_mid + i.getMemids(j); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:106: package Osmformat.Relation does not exist [javac] if (i.getTypes(j) == Osmformat.Relation.MemberType.NODE) [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:108: package Osmformat.Relation does not exist [javac] else if (i.getTypes(j) == Osmformat.Relation.MemberType.WAY) [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/BinaryMapParser.java:110: package Osmformat.Relation does not exist [javac] else if (i.getTypes(j) == Osmformat.Relation.MemberType.RELATION) [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Way.java:21: cannot find symbol [javac] symbol : class IntArrayList [javac] location: class uk.me.parabola.splitter.Way [javac] private final IntArrayList refs = new IntArrayList(10); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:99: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.DenseInt2ShortMap [javac] valschunks = new ObjectArrayList<short[]>(); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:97: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:104: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:109: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:114: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:119: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:124: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:129: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/DenseInt2ShortMap.java:134: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Main.java:324: cannot find symbol [javac] symbol : class BlockInputStream [javac] location: class uk.me.parabola.splitter.Main [javac] BlockInputStream blockinput = (new BlockInputStream( [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/Main.java:324: cannot find symbol [javac] symbol : class BlockInputStream [javac] location: class uk.me.parabola.splitter.Main [javac] BlockInputStream blockinput = (new BlockInputStream( [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/OSMWriter.java:117: cannot find symbol [javac] symbol : class IntArrayList [javac] location: class uk.me.parabola.splitter.OSMWriter [javac] IntArrayList refs = way.getRefs(); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:44: cannot find symbol [javac] symbol : variable Fast [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] return Fast.count(mask & ((1L << lowest) -1)); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:69: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] valschunks.set(chunkid, new ShortArrayList(1)); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:70: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] ShortArrayList chunk = valschunks.get(chunkid); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:95: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] ShortArrayList chunk = valschunks.get(chunkid); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:110: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] ShortArrayList chunk = valschunks.get(chunkid); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:130: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] valschunks = new ObjectArrayList<ShortArrayList>(); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:130: cannot find symbol [javac] symbol : class ShortArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] valschunks = new ObjectArrayList<ShortArrayList>(); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:131: cannot find symbol [javac] symbol : class LongArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMap [javac] bitmasks = new LongArrayList(); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:128: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:136: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:141: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:146: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:151: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:156: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:161: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMap.java:166: method does not override or implement a method from a supertype [javac] @Override [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:102: cannot find symbol [javac] symbol : variable Fast [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] return Fast.count(mask & ((1L << lowest) -1)); [javac] ^ [javac] /home/carlos/Paquetes/splitter_pbf/src/uk/me/parabola/splitter/SparseInt2ShortMapInline.java:188: cannot find symbol [javac] symbol : class ObjectArrayList [javac] location: class uk.me.parabola.splitter.SparseInt2ShortMapInline [javac] valschunks = new ObjectArrayList<short[]>(); [javac] ^ [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 100 errors
Am 12.10.2010 13:45, schrieb Carlos Dávila:
El 10/10/10 19:55, Ronny Klier escribió:
Am 09.10.2010 10:27, schrieb Minko:
Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter?
There is already a branch of the splitter which is able to read the pbf format (branches/crosby_integration). I build and successfully tested it last week with the europe extract from Geofabrik. It is much faster than using the bz2 file, time went down from ~5 hours + time for extraction of the bz2 file to less than 4 hours.
I hope there is soon an "official" build of the splitter with pbf support. Perhaps it was not merged because of additional dependencies.
Trying to compile pbf branch I get the error below. How can I get missing packages to be able to compile? Buildfile: /home/carlos/Paquetes/splitter_pbf/build.xml
I took them from actual osmosis package as Scott Crosby wrote some time ago. There are four required jars: - disutils - fastutil - osmbin - protobuf
El 13/10/10 00:23, Ronny Klier escribió:
Am 12.10.2010 13:45, schrieb Carlos Dávila:
El 10/10/10 19:55, Ronny Klier escribió:
Am 09.10.2010 10:27, schrieb Minko:
Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter?
There is already a branch of the splitter which is able to read the pbf format (branches/crosby_integration). I build and successfully tested it last week with the europe extract from Geofabrik. It is much faster than using the bz2 file, time went down from ~5 hours + time for extraction of the bz2 file to less than 4 hours.
I hope there is soon an "official" build of the splitter with pbf support. Perhaps it was not merged because of additional dependencies.
Trying to compile pbf branch I get the error below. How can I get missing packages to be able to compile? Buildfile: /home/carlos/Paquetes/splitter_pbf/build.xml
I took them from actual osmosis package as Scott Crosby wrote some time ago. There are four required jars: - disutils - fastutil - osmbin - protobuf
I downloaded current osmosis package (0.37) but could only find osmbin (osmbin-1.0-6d760534) and protobuf (protobuf-java-2.3.0) jars. Where should I place them to compile splitter successfully?
Am 18.10.2010 18:10, schrieb Carlos Dávila:
El 13/10/10 00:23, Ronny Klier escribió:
Am 12.10.2010 13:45, schrieb Carlos Dávila:
El 10/10/10 19:55, Ronny Klier escribió:
Am 09.10.2010 10:27, schrieb Minko:
Is there an explanation why the latest split runs are so slow? Could it be that the tiles in my areas.list are getting too big? How can I extract the pbf format? Can I use it with the splitter?
There is already a branch of the splitter which is able to read the pbf format (branches/crosby_integration). I build and successfully tested it last week with the europe extract from Geofabrik. It is much faster than using the bz2 file, time went down from ~5 hours + time for extraction of the bz2 file to less than 4 hours.
I hope there is soon an "official" build of the splitter with pbf support. Perhaps it was not merged because of additional dependencies.
Trying to compile pbf branch I get the error below. How can I get missing packages to be able to compile? Buildfile: /home/carlos/Paquetes/splitter_pbf/build.xml
I took them from actual osmosis package as Scott Crosby wrote some time ago. There are four required jars: - disutils - fastutil - osmbin - protobuf
I downloaded current osmosis package (0.37) but could only find osmbin (osmbin-1.0-6d760534) and protobuf (protobuf-java-2.3.0) jars. Where should I place them to compile splitter successfully?
I use eclipse as build environment. I copied those jars to a place where eclipse could find them and added them as dependency to the splitter project. Now that you told it I remember that I searched the web for the missing jars and found them at: http://dsiutils.dsi.unimi.it/ and http://fastutil.dsi.unimi.it/
participants (4)
-
aighes -
Carlos Dávila -
Minko -
Ronny Klier