Hi all, some time ago we discussed whether splitter could somehow exchange information with mkgmap. I think the idea was that mkgmap (or the style system) can remove many or all nodes from a tile which leads to problems. I assume this problem is solved with the new split algorithm? If not: Splitter writes the file densities-out.txt, which contains the node counters. It can also read that file, and it would not be very difficult to manipulate the counters somehow. I stll see one option to improve splitter throughput: Before reading any osm file, mkgmap creates a list of all tag keys which are needed (hard coded keys and those from the style files). All other tags are simply ignored when reading OSM data to save memory. If we would give this list to splitter we could probably save a lot of processing time when splitter ignores these tags as well. On the other hand, you have to execute splitter again if you change a style. What do you think? Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-fine-tuning-tp5741064.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
On 19.12.2012 17:15, GerdP wrote:
Hi all,
some time ago we discussed whether splitter could somehow exchange information with mkgmap. I think the idea was that mkgmap (or the style system) can remove many or all nodes from a tile which leads to problems. I assume this problem is solved with the new split algorithm? If not: Splitter writes the file densities-out.txt, which contains the node counters. It can also read that file, and it would not be very difficult to manipulate the counters somehow. yes, I didn't actually check it on the device, but the tiles in France are now never empty, so I think it's highly unlikely to run into the bug again (only if people added loads of open sea map data).
I stll see one option to improve splitter throughput: Before reading any osm file, mkgmap creates a list of all tag keys which are needed (hard coded keys and those from the style files). All other tags are simply ignored when reading OSM data to save memory. If we would give this list to splitter we could probably save a lot of processing time when splitter ignores these tags as well. On the other hand, you have to execute splitter again if you change a style.
What do you think?
Gerd I think that sounds very cool as an option. It could be switched off if you play a lot - but then in generall, splitter is much faster than mkgmap anyhow, so it's rather seldom that it would matter to me...
-- View this message in context: http://gis.19327.n5.nabble.com/splitter-fine-tuning-tp5741064.html Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- keep on biking and discovering new trails Felix openmtbmap.org & www.velomap.org
Am 19.12.2012 17:15, schrieb GerdP:
I stll see one option to improve splitter throughput: Before reading any osm file, mkgmap creates a list of all tag keys which are needed (hard coded keys and those from the style files). All other tags are simply ignored when reading OSM data to save memory. If we would give this list to splitter we could probably save a lot of processing time when splitter ignores these tags as well. On the other hand, you have to execute splitter again if you change a style.
What do you think? Sounds good. Typical the tag-list wont change very often, only if you change your style or then mkgmap changes default tags, so it should be possible to generate such a list manually. First point isn't hard, because the user did this active and can create a new list. If mkgmap-behavior changes, the user doesn't mind it and this could lead to problems. So it would be best, to did this everytime splitter runs. But this could also be managed seperate in script.
First call mkgmap.jar --create-tag-list, then splitter and mkgmap as usual. Henning
Henning Scholland wrote
What do you think? Sounds good. Typical the tag-list wont change very often, only if you change your style or then mkgmap changes default tags, so it should be possible to generate such a list manually. First point isn't hard, because the user did this active and can create a new list. If mkgmap-behavior changes, the user doesn't mind it and this could lead to problems. So it would be best, to did this everytime splitter runs. But this could also be managed seperate in script.
First call mkgmap.jar --create-tag-list, then splitter and mkgmap as usual.
I just tried a hard coded version of this and the effect is not promising: files are only 4 % smaller, but runtime increased with o5m and decreased with pbf output. All changes are small, so forget that idea. Gerd -- View this message in context: http://gis.19327.n5.nabble.com/splitter-fine-tuning-tp5741064p5741083.html Sent from the Mkgmap Development mailing list archive at Nabble.com.
Am 19.12.2012 20:52, schrieb GerdP:
Henning Scholland wrote
What do you think? Sounds good. Typical the tag-list wont change very often, only if you change your style or then mkgmap changes default tags, so it should be possible to generate such a list manually. First point isn't hard, because the user did this active and can create a new list. If mkgmap-behavior changes, the user doesn't mind it and this could lead to problems. So it would be best, to did this everytime splitter runs. But this could also be managed seperate in script.
First call mkgmap.jar --create-tag-list, then splitter and mkgmap as usual. I just tried a hard coded version of this and the effect is not promising: files are only 4 % smaller, but runtime increased with o5m and decreased with pbf output. All changes are small, so forget that idea. I think it depends on how it should be coded. If you just skip tags, the benefit isn't huge in compressed output. There would be a benefit, if everything is skiped, if it contains a tag. Eg. remove complete landuse=*, natural=* and so on.
But this could also be done with osmfilter or osmosis. So I would agree to forget that idea. Henning
I think it depends on how it should be coded. If you just skip tags, the benefit isn't huge in compressed output. There would be a benefit, if everything is skiped, if it contains a tag. Eg. remove complete landuse=*, natural=* and so on.
But this could also be done with osmfilter or osmosis. So I would agree to forget that idea.
Well, you are right, I've simply discarded the tag info. It would be much work to delete the whole way or relation depending on the (non-)existence of tags and that is handled well by the other tools. So, yes, forget that idea. Gerd
participants (4)
-
Felix Hartmann -
Gerd Petermann -
GerdP
-
Henning Scholland