My internal repository has several independent development threads. Only one of them is the binary format. The other development threads included patches that are already in and some have not been benchmarked/tested thoroughly. * The binary format. * Improved double writing for XML output. * Alternate threading design. * Avoid sending each node to each osmwriter to do a bbox check. (Depends on some of the binary format refactoring.) * too-many-areas. (already in) * Tag representation (partially in; not benchmarked) Git knows which patch is in which thread. The two critical ones for the binary format are the binary format patches and the ULP patch in the double-writing thread. The patches where I avoid sending each node to each OSM writer improved the Big-O from O(n) to O(sqrt(n))), but the benchmarks seem to be the identical performance. I think things are bottlenecked updating the large shared arrays, which may also be why there's no need to worry about the threading design making one thread per writer; they're all idle waiting for the bottleneck to dribble out stuff to write. Scott On Sun, Sep 12, 2010 at 8:25 AM, Chris Miller <chris_overseas@hotmail.com> wrote:
Hello Jeffrey,
Thanks very much for this. It's likely to be a while before I get a chance to look at and apply these sorry, but hopefully will find some time next weekend. Perhaps in the meantime if you have a current build of the splitter that includes these patches you could make it available for others to test and provide some initial feedback on?
The one thing I'm not sure about is the last one with the new thread design - I know it provides a decent performance boost, but I'm a little uneasy with the way it achieves this by using more threads than CPU cores. AFAIK that patch is incidental to the rest of the binary file support though so shouldn't affect this transition.
Cheers, Chris
Here's a quick rebasing of the patches in Scott's splitter repository to the current splitter trunk. My Java dev system is down for some upgrades at the moment so I haven't even compiled these yet but hopefully I didn't do too bad of a job of resolving the conflicts.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev