May 2012 - mkgmap-dev - The mkgmap lists

[PATCH v1] Precompiled sea
by WanMil 12 May '12

12 May '12

Hi, attached patch uses precompiled sea files to generate the sea areas. Generating sea areas have always been a source of problems. Little errors in the coastline data could flood whole tiles. A second problem is that generating sea areas often take long CPU time. After having used a precompilation step for the address assignment I wondered if it would be possible to do something similar for sea areas. After having played around with compiling sea areas myself I remembered that Mapnik also precompiles sea areas. This precompilation can be downloaded at http://wiki.openstreetmap.org/wiki/Coastline#Main_Mapnik_Layer and uses the ESRI shape format. That makes things very easy :-) I used the geotools lib and converted the sea information to our well known boundary format and added a small code addition to the mkgmap SeaGenerator. Attached patch does not include the ESRI shape conversion (I have to think about how to put that into the mkgmap code base - lots of geotools libs have to be included). But I have uploaded the whole world to http://www.navmaps.eu/wanmil/sea_20120331.zip. The new mkgmap parameter --precomp-sea=<dir with sea files> must be added. So unzip the sea_20120331.zip file to a directory called sea. Call the patched mkgmap with java -jar mkgmap.jar <.. your mkgmap options..> --precomp-sea=sea *.osm.pbf I am sure there are some problems left and the code patch should be cleaned up (better integration of the BoundaryXXX classes). But the first results looked so well that I want to give you the early chance to play with it. Now I hope it should also be possible to create a tile in the atlantic/pacific ocean without any data but full of sea. WanMil

5 18

Commit: r2277: Adjust TRE size of test
by svn commit 08 May '12

08 May '12

Version 2277 was commited by wanmil on 2012-05-06 20:16:39 +0100 (Sun, 06 May 2012) Adjust TRE size of test Commit 2276 changed the size of the TRE file by using a different subdivision algorithm.

4 7

[Patch] Overflow in splitter node counters
by GerdP 08 May '12

08 May '12

Hi Thorsten, attached is a patch for the problem reported here: http://gis.19327.n5.nabble.com/Bug-in-splitter-tp5610856p5610856.html I don't have the data nor the machine to test it, so I tried to find all places were nodes are counted and changed the variables to long. I did not find a good explanation for the small difference in the numbers: osmconvert reported 2518069911 nodes, which is ~ 2.518.069.911 -(2^31 - 1) = 370.586.264 - 2^31 + 370.586.264 =* -1.776.897.384* but splitter r200 reported * -1.776.905.107* nodes The formatting routine is not the reason. Any ideas? Gerd http://gis.19327.n5.nabble.com/file/n5611808/nodes_int_overflow_v1.patch nodes_int_overflow_v1.patch -- View this message in context: http://gis.19327.n5.nabble.com/Patch-Overflow-in-splitter-node-counters-tp5… Sent from the Mkgmap Development mailing list archive at Nabble.com.

4 9

[Patch] Improve boundary splitting
by GerdP 07 May '12

07 May '12

Hi WanMil, attached is a patch that uses an optimized implementation of the Sutherland-Hodgman algorithm http://en.wikipedia.org/wiki/Sutherland-Hodgeman wikipedia which is much faster than the area.intersect. It is only used to create the precompiled boundaries. The improvements depend on the continent, for africa I see 130 secs with r2272 and 84 secs with the patched version. For asia, I see only a small improvement of ~ 10% On my machine, the precompilation of the boundaries is now quite often waiting for the disk. http://gis.19327.n5.nabble.com/file/n5670294/BoundarySplitter.patch BoundarySplitter.patch Gerd -- View this message in context: http://gis.19327.n5.nabble.com/Patch-Improve-boundary-splitting-tp5670294p5… Sent from the Mkgmap Development mailing list archive at Nabble.com.

3 5

[TeamCity, SUCCESSFUL] Build mkgmap::trunk build #2277-1226
by teamcity＠mkgmap.org.uk 06 May '12

06 May '12

Build mkgmap::trunk build #2277-1226 successful (tests passed: 235). Agent: Iocaste Build results: http://teamcity.mkgmap.org.uk/viewLog.html?buildId=4024&buildTypeId=bt2 Changes included (1 change) ==================================================== Change 2277 by wanmil (1 file): Adjust TRE size of test Commit 2276 changed the size of the TRE file by using a different subdivision algorithm. see more information about changed files: http://teamcity.mkgmap.org.uk/viewLog.html?tab=buildChangesDiv&buildId=4024… ============================================================================ Configure email notifications: http://teamcity.mkgmap.org.uk/profile.html?init=1&tab=userNotifications

1 0

[TeamCity, FAILED] Build mkgmap::trunk build #2276-1225
by teamcity＠mkgmap.org.uk 06 May '12

06 May '12

Build mkgmap::trunk build #2276-1225 failed (tests failed: 1 (1 new), passed: 234). Agent: Iocaste Build results: http://teamcity.mkgmap.org.uk/viewLog.html?buildId=4023&buildTypeId=bt2 Failed Tests Summary: Newly failed tests (1 test, alphabetically ordered) ================================================================== func.route.SimpleRouteTest.testSize Newly failed tests details (naturally ordered) ================================================================== func.route.SimpleRouteTest.testSize (new) ================= junit.framework.AssertionFailedError: TRE size expected:<1343> but was:<1445> at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.failNotEquals(Assert.java:618) at org.junit.Assert.assertEquals(Assert.java:126) at org.junit.Assert.assertEquals(Assert.java:443) at func.route.SimpleRouteTest.testSize(SimpleRouteTest.java:60) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:220) at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906) ------- Stdout: ------- Time started: Sun May 06 19:54:26 BST 2012 Time finished: Sun May 06 19:54:29 BST 2012 Total time taken: 2477ms Changes included (1 change) ==================================================== Change 2276 by wanmil (3 files): Split subparts of lines that exceed the maximum size of a subdivision If two subsequent points of a line exceed that maximum size of a subdivision some points are added inbetween to fulfill the subdivision rules. --GerdP see more information about changed files: http://teamcity.mkgmap.org.uk/viewLog.html?tab=buildChangesDiv&buildId=4023… ============================================================================ Configure email notifications: http://teamcity.mkgmap.org.uk/profile.html?init=1&tab=userNotifications

1 0

Commit: r2276: Split subparts of lines that exceed the maximum size of a subdivision
by svn commit 06 May '12

06 May '12

Version 2276 was commited by wanmil on 2012-05-06 19:44:28 +0100 (Sun, 06 May 2012) Split subparts of lines that exceed the maximum size of a subdivision If two subsequent points of a line exceed that maximum size of a subdivision some points are added inbetween to fulfill the subdivision rules. --GerdP

1 0

Commit: r2275: Allow only flat file structure in boundary zip file to speed up detection of missing boundary files.
by svn commit 06 May '12

06 May '12

Version 2275 was commited by wanmil on 2012-05-06 19:32:32 +0100 (Sun, 06 May 2012) Allow only flat file structure in boundary zip file to speed up detection of missing boundary files.

1 0

[Patch V1] Subdivision width is 36627 at 3230916/1236133
by GerdP 04 May '12

04 May '12

Hi all, attached is a patch that fixes a few problems with r2272 1) When the initial area is created, the area bounds were only updated when a point was added. Lines and shapes were added without updating the area bounds. The patch changes this. 2) The splitMaxSize() method did used the calculated bounds, not the bounds of the elements. Therefore, it did not split the area when the number of elements was small. 3) The LineSizeSplitterFilter did not split a line to parts that are smaller than maxSize, it always added the point that made the bbox of the line too large. If this last point is far away from the others, the bbox is far too large. The patch makes sure that a line is split into parts that are small enough. Regarding the problem described by WanMils picture: This is not yet solved with this patch. I have to create some test cases to analyse this also. Gerd http://gis.19327.n5.nabble.com/file/n5672934/subdivision_width_v1.patch subdivision_width_v1.patch -- View this message in context: http://gis.19327.n5.nabble.com/Patch-V1-Subdivision-width-is-36627-at-32309… Sent from the Mkgmap Development mailing list archive at Nabble.com.

5 11

[Patch] Faster BoundaryGrid
by GerdP 04 May '12

04 May '12

Hi WanMil, attached is a small patch which drastically improves BoundaryGrid for tiles that contain mostly sea. I found such a tile in south-america, and the patched version is > 10 times faster. It contais a large part of the pacific, and BoudaryGrid searches the boundary zip file for > 5000 different *.bnd files, most of them do not exist in your > bounds_20120401.zip , so the code in BoundaryUtil searches through all elements of the zip file to find out that it doesn't exist. The patch creates a HashSet first with the available files and calls the load routine only for those that exist. Gerd http://gis.19327.n5.nabble.com/file/n5680180/BoundaryGrid.java.patch BoundaryGrid.java.patch -- View this message in context: http://gis.19327.n5.nabble.com/Patch-Faster-BoundaryGrid-tp5680180.html Sent from the Mkgmap Development mailing list archive at Nabble.com.

2 4