Hi WanMil, I've analysed the problem and found two different reasons. 1) delta coding for doubles sometimes involves rounding errors: delta = lat-oldLat; if (oldLat + delta != lat) System.out.println("delta coding causes rounding"); This happens more often when the delta is near 0 (e.g. 60000-59999.99999999999999), I thinks this also explains why you saw more holes around the Greenwich meridian. This case is now signaled with a Double.POSITIVE_INFINITY and handled by the read routine (it uses the next value without applying the delta). 2) Conversion between Area and Path2D.Double is not reversible. Sample: Path2D.Double path = new Path2D.Double(area); Area testArea = new Area(path); if area contains only those holes (one or more), testArea might be empty. I've used this knowledge to re-implement some tests in BoundaryQuadTree.makeDistinct() With my test data these changes produce a result that has fewer holes than trunk and no holes that I could not explain with OSM data. (I've tested latest africa.osm.pbf) I've also changed the routines that read and write the varying double format to avoid the Long.reverse() operation. One has to create boundary files again. Gerd http://gis.19327.n5.nabble.com/file/n5607312/boundary_double_v2.patch boundary_double_v2.patch WanMil wrote
Hi Gerd,
thanks a lot. Unfortunately the new patched code creates more holes than the current SVN code. The BoundaryCoverageUtil creates 1000 separate polygons for admin_level=2 instead of 640.
I have uploaded the results of the BoundaryCoverageUtil to http://www.navmaps.eu/wanmil/coverage_2.zip.
It seems that there are many holes around the Greenwich meridian. Can you have a look on it and check if old or new code is better?
Anyhow I aggree that the holes are more a cosmetic problem. But maybe it's worthy to invest the time to know exactly what happens there to be sure that there are no other problems.
Thanks! WanMil
Hi WanMil,
attached is a patch that changes all boundary routines to use double precision. The file format has changed again, so one has to create new boundaries.
http://gis.19327.n5.nabble.com/file/n5604371/boundary_double_v1.patch boundary_double_v1.patch
Although it uses double precision, the new format produces smaller files because it uses delta coding for the boundary coordinates.
I found a strange error in Area.subtract(). For two given areas a1 and a2 lying within a bbox, a1.subtract(a2) may result in an area lying outside of the bbox. I've added a test for that. The performance is almost equal, the newer version may be a bit faster.
Gerd
GerdP wrote
Hi WanMil,
I can try writing the RAW format with double precision. Do you have a sample input that creates a hole? I tried africa and asia and found none, the europe file is too large for my machine. We may also try to save the quadtree data with doubles without getting much larger files if we use a compressed format (e.g. the delta coding used in pbf or o5m).
Gerd
WanMil wrote
Hi Gerd,
your conclusion sound very reasonable. The number of holes has been reduced quite a lot after your last patch and if your example a) b) and c) are not equal all the time there is no chance to avoid such holes completely.
Just a quick idea: Would it improve (and reduce the number of holes) if the intermediate bounds format which is written first uses double instead of float precision? I think it doesn't really matter if the intermediate format uses more space on disk?!
I will try to create updated bounds files during the weekend and then we can start a one week test phase with more users before merging back to trunk.
WanMil
Hi WanMil,
I found three reasons for these holes: errors in BoundaryQuadTree, "errors" in java.awt.geom.Area and rounding erros.
My last patches fixed the errors that I found in BoundaryQuadTree, I also tried to avoid the problem with the rounding errors (we are doing the calculations with double precision, but we save with float precision).
I stopped searching when I saw fewer holes in the result of the performance branch than in the result of trunk.
The errors in java.awt.geom.Area are complex. I think one has different options to calculate the intersection of two areas a1 and a2: a) Area x = new Area(a1); x.intersect(a2) b) Area x = new Area(a2); x.intersect(a1) c) Area x = new Area(a1); x.add(a2);x.subtract(a1);x.subtract(a2);
In an ideal world I would expect to get exactly the same result for these three calculations, but sometimes the real results are not equal. The same problem occurs when we add the areas again in the BoundaryCoverageUtil.
Maybe we can get rid of a few more of the holes if we manage to do all calculations with doubles, but probably even doubles will not solve all problems.
The question is if we should invest time for that. I think the holes are purely cosmetic, if you really manage to query BoundaryQuadTree for a point that lies within such a hole, it will return the result for a point that is next to it, I think this is good enough.
What do you mean?
Gerd
WanMil wrote
Hi Gerd,
thanks for the patch. I have commited it although I still see the problem. I am not sure if the holes are at the same place but there are some of them which I cannot explain with wrong boundary data.
Can you please recheck that?
Thanks! WanMil
> Hi WanMil, > > attached is a fix for this problem. > > Please, can you review if the UnusedElementsRemoverHook is still > useful? > With my test data, it is slowing down mkgmap a little bit and I also > see > a different result for one tile in the UK when I disable it. > > Gerd > > > Date: Thu, 15 Mar 2012 21:11:02 +0100 > > From: wmgcnfg@ > > To: mkgmap-dev@.org > > Subject: [mkgmap-dev] Small holes in boundary coverage > > > > Hi, > > > > I have compiled world bounds using the performance branch. > They > can be > > downloaded from > http://www.navmaps.eu/wanmil/bounds_perf_20120313.zip. > > > > I have checked the coverage of different admin_levels with > the > > BoundaryCoverageUtil. The result for admin_level=2 can be > downloaded > > from http://files.mkgmap.org.uk/detail/61. > > > > Some countries are missing (USA, Canada, some african > countries). > I > > assume that their boundaries were/are corrupt. > > > > The more interesting thing is that there are small holes > throughout > the > > world in admin_level=2. They seem to be created by the bounds > > preparation and should not be there. Gerd, can you please > have a > look > on it? > > > > Thanks! > > WanMil > > _______________________________________________ > > mkgmap-dev mailing list > > mkgmap-dev@.org > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > > > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev@.org > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p55... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p56... Sent from the Mkgmap Development mailing list archive at Nabble.com. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@.org http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- View this message in context: http://gis.19327.n5.nabble.com/Small-holes-in-boundary-coverage-tp5569161p56... Sent from the Mkgmap Development mailing list archive at Nabble.com.