splitter error using --cache option
I have used successfully new splitter when no --split_file is given, but if I use an existing areas.list splitter looks for cached files and they don't exist; see below: java -Xmx2500M -jar splitter.jar --cache=/160GB/cache_europe --split-file=areas_5.list /160GB/europe.osm Time started: Mon Aug 24 16:24:22 CEST 2009 cache = /160GB/cache_europe split-file = areas_5.list Areas read from file 63240001 (35.947265625,-9.66796875) to (41.044921875,-4.04296875) 63240002 (41.044921875,-9.580078125) to (43.9453125,-4.04296875) 63240003 (36.474609375,-4.04296875) to (39.7265625,1.845703125) 63240004 (39.7265625,-4.04296875) to (43.76953125,-1.142578125) 63240005 (39.7265625,-1.142578125) to (43.41796875,1.845703125) 63240006 (38.759765625,1.845703125) to (42.978515625,4.833984375) Writing out split osm files Mon Aug 24 16:24:22 CEST 2009 Processing 6 areas in a single pass Starting pass 1, processing 6 areas (63240001 to 63240006) Loading and processing nodes Error opening or reading file java.io.FileNotFoundException: /160GB/cache_europe/nodes.bin (No such file or directory) java.io.FileNotFoundException: /160GB/cache_europe/nodes.bin (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(Unknown Source) at uk.me.parabola.splitter.disk.NodeStoreReader.<init>(NodeStoreReader.java:19) at uk.me.parabola.splitter.BinaryMapLoader.processNodes(BinaryMapLoader.java:92) at uk.me.parabola.splitter.BinaryMapLoader.load(BinaryMapLoader.java:82) at uk.me.parabola.splitter.Main.processMap(Main.java:281) at uk.me.parabola.splitter.Main.writeAreas(Main.java:271) at uk.me.parabola.splitter.Main.split(Main.java:118) at uk.me.parabola.splitter.Main.main(Main.java:92) Time finished: Mon Aug 24 16:24:23 CEST 2009 Total time taken: 0s Regards Carlos
Hi Carlos, This is something I'm aware of but haven't yet had time to address. It's because (currently) the cache can only be generated during the first stage when the areas.list file is also being generated. If you supply areas.list and use --cache at the same time, the splitter assumes the cache has already been generated, hence the reason you're seeing this error. It's on my todo list to allow cache generation to occur during the second stage too if need be, I just didn't think anyone would encounter this situation before I had a solution in place. Guess I was wrong! :) To be honest though if you already have an areas.list file and are only making one pass over the XML file during the splitting stage, there's no benefit to using the --cache parameter at all unless you perform multiple runs of the splitter (even with different areas.list files - all that matters is the original .osm file being split is the same). Is that what you want to do? For now your options are to either generate the cache by running the splitter once with no --split-file parameter, or running without the --cache option. Sorry for any inconvenience! Chris CD> I have used successfully new splitter when no --split_file is given, CD> but CD> if I use an existing areas.list splitter looks for cached files and CD> they CD> don't exist; see below:
Chris Miller escribió:
Hi Carlos,
This is something I'm aware of but haven't yet had time to address. It's because (currently) the cache can only be generated during the first stage when the areas.list file is also being generated. If you supply areas.list and use --cache at the same time, the splitter assumes the cache has already been generated, hence the reason you're seeing this error. It's on my todo list to allow cache generation to occur during the second stage too if need be, I just didn't think anyone would encounter this situation before I had a solution in place. Guess I was wrong! :) To be honest though if you already have an areas.list file and are only making one pass over the XML file during the splitting stage, there's no benefit to using the --cache parameter at all unless you perform multiple runs of the splitter (even with different areas.list files - all that matters is the original .osm file being split is the same). Is that what you want to do?
Currently I only run splitter once on europe.osm to extract Spain, Portugal and South of France, but depending on the performance with the cache may be I run it later to extract other countries. Thanks for the explanation.
For now your options are to either generate the cache by running the splitter once with no --split-file parameter, or running without the --cache option.
Sorry for any inconvenience!
No problem Carlos
Chris
CD> I have used successfully new splitter when no --split_file is given, CD> but CD> if I use an existing areas.list splitter looks for cached files and CD> they CD> don't exist; see below:
Hi Carlos, Try splitter r77, it should now behave the way you expect with caching and --split-file. I decided to make it print a (perhaps too threatening?) warning message if it detects that you want to build a cache even though it only needs to make a single pass over the .osm file. I figured it's better to make people aware of the performance impact of doing this than for the splitter to just silently take longer than it should without the user realising something might be amiss. Chris CD> Currently I only run splitter once on europe.osm to extract Spain, CD> Portugal and South of France, but depending on the performance with CD> the cache may be I run it later to extract other countries. Thanks CD> for the explanation.
participants (2)
-
Carlos Dávila -
Chris Miller