okay - well the only working solution to this problem is to use o5m output format with mkgmap, and then use osmconvert to join the files before splitting them again.
Would it be possible for mkgmap to have a mode to split the files one by one without the detour via osmconvert?

o5m uses a lot more hdd space, and mkgmap.jar is only marginally faster with 05m input vs osm.pbf (splitter works quite a lot faster on big o5m files vs osm.pbf - but small input files make not much time difference)...

On Mon, 30 Aug 2021 at 02:25, Felix Hartmann <extremecarver@gmail.com> wrote:
I split france.osm.pbf with D:\openmtbmap\maps>if france == legend start /belownormal /b /wait java -jar -XX:+AggressiveHeap -Xms1000m -Xmx3800m C:\openmtbmap\splitter.jar --max-nodes=9600000  --max-threads=12 --search-limit=1000000 --output=pbf "--keep-complete" --max-areas=1024  --geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip --description=france --mapid=83910000 france.osm  1>NUL

Then I compiled the map using mgkamp.jar - and want to resplit all remaining files that did not compile with smaller max-nodes value. However splitter.jar chokes if I pass several files... Is there any way to have splitter just working with several files without trying to merge them? I just need the files split to the new max-nodes value and incrementing the map id. If I pass only one file at a time, it is a PITA to script this (is already hard enough figuring out the new map-id and creating a list of the tiles).

D:\openmtbmap\maps>if yes EQU yes start /belownormal /b /wait java -XX:+AggressiveHeap -Xms5000M -Xmx54000M -jar C:\openmtbmap\splitter.jar --max-nodes=4800000 --max-threads=12 --search-limit=1000000 --output=pbf  --geonames-file=D:\openmtbmap\osmpbf_geofabrik\cities5000.zip  --description=france --mapid=83910046  "83910015.osm.pbf" "83910025.osm.pbf" "83910026.osm.pbf" "83910042.osm.pbf"    1>NUL
Warning: --keep-complete is only used for the first input file. Further files must use higher ids.
Error: Node ids are not sorted. Use e.g. osmosis to sort the input data.
This is not supported with keep-complete=true or --problem-list
uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted
        at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:497)
        at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:126)
        at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:82)
        at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)
        at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
        at uk.me.parabola.splitter.Main.useProblemLists(Main.java:503)
        at uk.me.parabola.splitter.Main.start(Main.java:127)
        at uk.me.parabola.splitter.Main.main(Main.java:81)


The option to first join the files with osmconvert - is a PITA - as you cannot parse several osm.pbf files to osmconvert (it only supports joining osm or o5m files).
The option to supply splitter.jar with each file on it's own - is also a PITA that would require horrendous for loops because after each split you need to check which new map-id to use for the next file!

So it would be great to have a keep-complete option that for each input file, but not for all input files, uses keep-complete. Actually I think this should be the default mode when supplying several input files. Even if mkgmap actually cannot run this in parallel / multithreading it would be way better than parsing each file on it's own...
Or am I dumbstruck in finding a way to resplit too big input files?

--
Felix Hartman - Openmtbmap.org & VeloMap.org



--
Felix Hartman - Openmtbmap.org & VeloMap.org