
Hi all, if you are testing the --keep-complete or the --problem-list option, please update to r232. The older versions in the branch are known to have stupid bugs causing extreme memory needs (and possibly long run times). Example: the DACH extract is now handled on a 32bit machine with -Xmx1600m without problems. Ciao, Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Sounds like a release candidate ... I will run some tests with it. Klaus -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735825.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Hi Klaus,
Sounds like a release candidate ... I will run some tests with it.
not yet, but test results regarding the output are welcome. I am working on an improvement regarding memory usage in the MultiTileAnalyzer, so performance tests can wait. I plan to add a restriction regarding "--mixed" data. Splitter r202 doesn't recognize mixed data in pbf files and it doesn't matter for r202. The new functions in the problem-list branch require sorted data, so I plan to stop with an error message if OSM ids are not sorted by type and id (ascending). I hope this will not cause problems because osmosis has a sort function. Ciao, Gerd

Hi, On Wed, Nov 14, GerdP wrote:
Hi all,
if you are testing the --keep-complete or the --problem-list option, please update to r232. The older versions in the branch are known to have stupid bugs causing extreme memory needs (and possibly long run times).
Example: the DACH extract is now handled on a 32bit machine with -Xmx1600m without problems.
Yes, my DACH is now splitted fine again, really great. But for europe, 6000M are no longer enough, I'm currently trying it with more memory. Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)

Hello Thorsten, Thorsten Kukuk wrote
But for europe, 6000M are no longer enough, I'm currently trying it with more memory.
please send me both logs. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735831.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

flabot@googlemail.com wrote
What about reading o5m Files an writing pbf with r232 ? _______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
Maybe I don't understand the question. This works fine with r232. The default output format is pbf. If you input file ends with .o5m, splitter assumes that it is in o5m format and reads it without problems. You can also write o5m with --output=o5m, but mkgmap doesn't yet support o5m Ciao, Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735863.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

On Wed, Nov 14, GerdP wrote:
Hello Thorsten,
Thorsten Kukuk wrote
But for europe, 6000M are no longer enough, I'm currently trying it with more memory.
please send me both logs.
Which logs exactly do you need? The output of splitter.jar? Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)

Thorsten Kukuk wrote
Which logs exactly do you need?
The output of splitter.jar?
yes, and the JVM parameters that you use. thanks, Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735949.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Hi Henning, Henning Scholland wrote
http:/www.aighes.de/data/mkgmap.zip
thanks, so it is already 20 minutes faster, and requires less memory, but I also see that we still have some bottlenecks. I also found a small error in the handling of parent rels. By mistake, splitter added the ways and nodes of parent rels to the problem lists. This will be corrected with the next version. I've also found some places to save heap and CPU. By the way: I am searching for tools that help me testing the results. If you have to large OSM files that differ only in a few nodes and ways, which one is correct? Is their a simple way to load all the differences into e.g. JOSM and add the *.kml file with the tile boundaries? Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735989.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Hi Gerd, I discovered a problem with missing objects and to much objects. I think the first was almost wrong in earlier versions. 11000004: 1800192,1406976 to 1968128,1681408 In this tile (north-western Turkey) there are missing two ferry-routes (Yalta-Istanbul and Sivastopol-Istanbul). As I opened the tile in josm, I discovered also, that there are several objects in that tile, which shouldn't be in it. Eg. boundary of USA, Russia Also josm reports several errors because not all nodes of some ways are in that tile. Maybe this could cause the problem of missing ferry-routes. Henning

Hi Henning, thanlks, I will try to find similar errors in smaller input files. My PC is far too slow for planet. Gerd Henning Scholland wrote
Hi Gerd, I discovered a problem with missing objects and to much objects. I think the first was almost wrong in earlier versions.
11000004: 1800192,1406976 to 1968128,1681408
In this tile (north-western Turkey) there are missing two ferry-routes (Yalta-Istanbul and Sivastopol-Istanbul). As I opened the tile in josm, I discovered also, that there are several objects in that tile, which shouldn't be in it. Eg. boundary of USA, Russia
Also josm reports several errors because not all nodes of some ways are in that tile. Maybe this could cause the problem of missing ferry-routes.
Henning
_______________________________________________ mkgmap-dev mailing list
mkgmap-dev@.org
-- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735993.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

GerdP wrote
thanlks, I will try to find similar errors in smaller input files. My PC is far too slow for planet.
I can reproduce the broken ways errors. They are caused by the wrong parent relation handling. I hope the other problems have the same reason. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735997.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Hi all,
11000004: 1800192,1406976 to 1968128,1681408
In this tile (north-western Turkey) there are missing two ferry-routes (Yalta-Istanbul and Sivastopol-Istanbul). As I opened the tile in josm, I discovered also, that there are several objects in that tile, which shouldn't be in it. Eg. boundary of USA, Russia
reg. the ferry-routes: I can't say for sure, but I'd say these ferry lines really don't cross the tile. I think it is a good idea to write the tile bordes in gpx format, this will help to answer such questions. reg. "objects that shouldn't be in it": I would accept Russia for now, but USA really sounds strange. I think the reason is the very simple that is used now: For a multipolygon (or boundary), splitter simply calculates the bounding box of all ways. The result is a rectangle. Every tile that intersects this rectangle contain the complete relation. The advantage of this method is that splitter doesn't have to calculate the real MP (which invvolves many special cases like incomplete data, holes etc). I wanted to leave this to mkgmap. For most boundaries, this simple alg. works quite well, because the bbox of the boundary doesn't contain too much other stuff. The bbox for Russia is obviously very big and contains a large area which doesn't belong to Russia. A real problem is USA (and probably other countries with external territories: Because of Hawai and some other islands the bbox for USA covers almost half of the planet. I hope I can find an alternative algorithm that is still simple and fast, but doesn't make this error.
Also josm reports several errors because not all nodes of some ways are in that tile. Maybe this could cause the problem of missing ferry-routes.
This is caused by an error in splitter and wiil be fixed soon. Gerd

Am 16.11.2012 09:09, schrieb Gerd Petermann:
Hi all,
11000004: 1800192,1406976 to 1968128,1681408
In this tile (north-western Turkey) there are missing two ferry-routes (Yalta-Istanbul and Sivastopol-Istanbul). As I opened the tile in josm, I discovered also, that there are several objects in that tile, which shouldn't be in it. Eg. boundary of USA, Russia
reg. the ferry-routes: I can't say for sure, but I'd say these ferry lines really don't cross the tile. I think it is a good idea to write the tile bordes in gpx format, this will help to answer such questions. An easy check is to display the map in MapSource and choose one tile, so this tile will be highlighted. I created an osm-file with 11000004-boundary and the two ferry-routes. http://www.aighes.de/data/11000004.zip
With open-data-PlugIn you can also open kml-files to josm ;)
reg. "objects that shouldn't be in it": I would accept Russia for now, but USA really sounds strange. I think the reason is the very simple that is used now: For a multipolygon (or boundary), splitter simply calculates the bounding box of all ways. The result is a rectangle. Every tile that intersects this rectangle contain the complete relation. The advantage of this method is that splitter doesn't have to calculate the real MP (which invvolves many special cases like incomplete data, holes etc). I wanted to leave this to mkgmap. Ah..thats logical. Russia is crossing 180°/-180° For most boundaries, this simple alg. works quite well, because the bbox of the boundary doesn't contain too much other stuff. The bbox for Russia is obviously very big and contains a large area which doesn't belong to Russia. A real problem is USA (and probably other countries with external territories: Because of Hawai and some other islands the bbox for USA covers almost half of the planet. I hope I can find an alternative algorithm that is still simple and fast, but doesn't make this error. I think, this would be acceptable. Splitter would save data with overlap=0, so there shouldn't be a real problem, if these to boundaries were copied to a lot of tiles.
Henning

Henning Scholland wrote
An easy check is to display the map in MapSource and choose one tile, so this tile will be highlighted. I created an osm-file with 11000004-boundary and the two ferry-routes. http://www.aighes.de/data/11000004.zip
With open-data-PlugIn you can also open kml-files to josm ;)
Thanks, that helps a lot! I was looking for tools to convert from kml to gpx, but that is much easier :-) I will find out why splitter doesn't think that the ferry routes cross the tile. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5736050.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

Am 14.11.2012 15:03, schrieb GerdP:
if you are testing the --keep-complete or the --problem-list option, please update to r232. The older versions in the branch are known to have stupid bugs causing extreme memory needs (and possibly long run times).
Hi, I didn't follow the splitter improvements threads in detail. So, what is now the suggested value for --overlap, when using the --keep-complete option? Is 0 okay ? Chris

Am 15.11.2012 11:52, schrieb Chris66:
Am 14.11.2012 15:03, schrieb GerdP:
if you are testing the --keep-complete or the --problem-list option, please update to r232. The older versions in the branch are known to have stupid bugs causing extreme memory needs (and possibly long run times). Hi, I didn't follow the splitter improvements threads in detail.
So, what is now the suggested value for --overlap, when using the --keep-complete option? Is 0 okay ? Yes, you could use --overlap=0
Henning

Chris66 wrote
So, what is now the suggested value for --overlap, when using the --keep-complete option? Is 0 okay ?
Yes, it is. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-r232-tp5735799p5735915.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
participants (8)
-
aighes
-
Chris66
-
flabot@googlemail.com
-
Gerd Petermann
-
GerdP
-
Henning Scholland
-
Thorsten Kukuk
-
toc-rox