Hello, in some areas, gaps may appear (see attachment). In OSM, however, the surfaces are connected and the nodes have the same coordinates. There should be no rounding errors. It will therefore be a bug in mkgmap. I ask for correction. Thank you! Best regards, Michael
Hi Michael, the Garmin format always requires rounding. Please share a link to one of the OSM objects, so that I can have a closer look. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Donnerstag, 14. September 2023 12:20 An: 'Development list for mkgmap' Betreff: [mkgmap-dev] Gaps in surfaces Hello, in some areas, gaps may appear (see attachment). In OSM, however, the surfaces are connected and the nodes have the same coordinates. There should be no rounding errors. It will therefore be a bug in mkgmap. I ask for correction. Thank you! Best regards, Michael
Hello Gerd, here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin. Thank you! Best regards, Michael
Hi Michael, okay, I can reproduce the problem. I assume it is caused by the code which subtracts the inner area from the multipolygon area. No idea if this can be fixed easily. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Dienstag, 26. September 2023 19:44 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Gaps in surfaces Hello Gerd, here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin. Thank you! Best regards, Michael _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi all, I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values. @Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead. ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 26. September 2023 19:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Michael, okay, I can reproduce the problem. I assume it is caused by the code which subtracts the inner area from the multipolygon area. No idea if this can be fixed easily. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Dienstag, 26. September 2023 19:44 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Gaps in surfaces Hello Gerd, here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin. Thank you! Best regards, Michael _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi I was thinking that gaps like this could often happen when adjacent polygons share the points along the common boundary, but, during the various point-removal optimisations, esp. Douglas Peucker, different points along this boundary are chosen to represent the 2 polygons. Regarding LineClipperTest, I don't have the facilities to look at this until next week. Ticker On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 26. September 2023 19:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Michael,
okay, I can reproduce the problem. I assume it is caused by the code which subtracts the inner area from the multipolygon area. No idea if this can be fixed easily.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Dienstag, 26. September 2023 19:44 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hello Gerd,
here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin.
Thank you!
Best regards, Michael
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Ticker, no problem, I think this is a very old error ;) I think the remaining gaps (at res 24) are produced by the MultipolygonCutter. It sometimes cuts the outer and produces new points and these new points are not added to the inner rings, only to the outer. I have to find out if this can be fixed by avoiding these cuts or by modifying the inner rings. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 29. September 2023 11:03 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi I was thinking that gaps like this could often happen when adjacent polygons share the points along the common boundary, but, during the various point-removal optimisations, esp. Douglas Peucker, different points along this boundary are chosen to represent the 2 polygons. Regarding LineClipperTest, I don't have the facilities to look at this until next week. Ticker On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 26. September 2023 19:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Michael,
okay, I can reproduce the problem. I assume it is caused by the code which subtracts the inner area from the multipolygon area. No idea if this can be fixed easily.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Dienstag, 26. September 2023 19:44 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hello Gerd,
here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin.
Thank you!
Best regards, Michael
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd Considering no_hp-overflow_alpha.patch: It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion. The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value. The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally. getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here? Haven't looked at LineClipperTest yet. Ticker On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
Hi Ticker, the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Gerd Considering no_hp-overflow_alpha.patch: It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion. The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value. The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally. getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here? Haven't looked at LineClipperTest yet. Ticker On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Ticker, I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Ticker, the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Gerd Considering no_hp-overflow_alpha.patch: It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion. The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value. The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally. getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here? Haven't looked at LineClipperTest yet. Ticker On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd I've been thinking about how to stop the small ranges of decimal degrees from generating MapUnit/delta=+32 and MapUnit+1/delta=-32. Without changing the MapUnit value, but truncating the highPrec calc, eg, in Coord.java private static int toHighPrec(double degrees) { final double CVT_HP = ((double)(1 << HIGH_PREC_BITS)) / 360; return (int)Math.floor(degrees * CVT_HP); } this problem almost goes away, with deltas between -31 .. +32 except for 2 instances of delta of -32 I get while building the Britain-and-ireland.osm.pbf. I need to work out why these are happening. Although it is unnatural to have this range rather than -32..+31, it doesn't matter. It probably could be fixed by using Math.ceil instead or reversing where the delta goes. getAlternativePositions() will generated delta values approx -48..+48. I don't get failures from LineClipperTest with this change. I do get failures from GmapsuppTest, SimpleRouteTest & SortTest, but I think these are not significant to this issue. As far as the gap between areas is concerned, I haven't looked at this yet but I'll see if my change has similar effects to your patch to Coord. Ticker On Wed, 2023-10-04 at 08:16 +0000, Gerd Petermann wrote:
Hi Ticker,
I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Ticker,
the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
Considering no_hp-overflow_alpha.patch:
It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion.
The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value.
The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally.
getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here?
Haven't looked at LineClipperTest yet.
Ticker
On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Ticker, reg. the change in toHighPrec() : I must confess that I never doubted why positive values should be rounded differently to negative values. This is also done in Utils.toMapUnit() and I just adapted that code. If you still want to change that I think Math.rint() would get you closer to the original implementation. Question is if or why this difference in rounding is/was needed. There should be visible effects near equator and at Greenwhich or at 180°. Anyhow, I don't think this is the solution for the gap problem which is caused by further conversions from highprec int to double and back. An alternative solution for that problem would be the attached patch which makes use of the coord pool. No idea which one has more effects on performance. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 4. Oktober 2023 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Gerd I've been thinking about how to stop the small ranges of decimal degrees from generating MapUnit/delta=+32 and MapUnit+1/delta=-32. Without changing the MapUnit value, but truncating the highPrec calc, eg, in Coord.java private static int toHighPrec(double degrees) { final double CVT_HP = ((double)(1 << HIGH_PREC_BITS)) / 360; return (int)Math.floor(degrees * CVT_HP); } this problem almost goes away, with deltas between -31 .. +32 except for 2 instances of delta of -32 I get while building the Britain-and-ireland.osm.pbf. I need to work out why these are happening. Although it is unnatural to have this range rather than -32..+31, it doesn't matter. It probably could be fixed by using Math.ceil instead or reversing where the delta goes. getAlternativePositions() will generated delta values approx -48..+48. I don't get failures from LineClipperTest with this change. I do get failures from GmapsuppTest, SimpleRouteTest & SortTest, but I think these are not significant to this issue. As far as the gap between areas is concerned, I haven't looked at this yet but I'll see if my change has similar effects to your patch to Coord. Ticker On Wed, 2023-10-04 at 08:16 +0000, Gerd Petermann wrote:
Hi Ticker,
I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Ticker,
the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
Considering no_hp-overflow_alpha.patch:
It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion.
The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value.
The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally.
getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here?
Haven't looked at LineClipperTest yet.
Ticker
On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd I've changed the way mapUnit and highPrec Coords are calculated slightly so that the exact 1/2 unit boundary values are consistent between negative and positive. This gets rid of my couple of abnormalities with a -32 delta. The old method added or subtracted 1/2 to make ((int)double) do rounding, which assigns the exact 1/2 to the larger abs value. toHighPrec(), as in yesterdays change, keeps its bounds consistent with the mapUnit it is splitting up but is now coded in a similar way to toMapUnit. I think rounding to get MapUnits is a mistake but changing it now could have drastic consequences, it should have been floorToNegInfinity, giving the nice cyclic scale -128..+127 << 24. As it is, we get -128..+128 << 24. Equator & Grenwhich Meridian behave well, but I've always had doubts about behaviour around 128 Hope this makes sense - patch attached Sorry, haven't thought about the actual problem with the gaps yet. Ticker On Thu, 2023-10-05 at 07:30 +0000, Gerd Petermann wrote:
Hi Ticker,
reg. the change in toHighPrec() : I must confess that I never doubted why positive values should be rounded differently to negative values. This is also done in Utils.toMapUnit() and I just adapted that code. If you still want to change that I think Math.rint() would get you closer to the original implementation. Question is if or why this difference in rounding is/was needed. There should be visible effects near equator and at Greenwhich or at 180°.
Anyhow, I don't think this is the solution for the gap problem which is caused by further conversions from highprec int to double and back. An alternative solution for that problem would be the attached patch which makes use of the coord pool. No idea which one has more effects on performance.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 4. Oktober 2023 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
I've been thinking about how to stop the small ranges of decimal degrees from generating MapUnit/delta=+32 and MapUnit+1/delta=-32.
Without changing the MapUnit value, but truncating the highPrec calc, eg, in Coord.java
private static int toHighPrec(double degrees) { final double CVT_HP = ((double)(1 << HIGH_PREC_BITS)) / 360; return (int)Math.floor(degrees * CVT_HP); }
this problem almost goes away, with deltas between -31 .. +32 except for 2 instances of delta of -32 I get while building the Britain-and-ireland.osm.pbf. I need to work out why these are happening.
Although it is unnatural to have this range rather than -32..+31, it doesn't matter. It probably could be fixed by using Math.ceil instead or reversing where the delta goes. getAlternativePositions() will generated delta values approx -48..+48.
I don't get failures from LineClipperTest with this change. I do get failures from GmapsuppTest, SimpleRouteTest & SortTest, but I think these are not significant to this issue.
As far as the gap between areas is concerned, I haven't looked at this yet but I'll see if my change has similar effects to your patch to Coord.
Ticker
On Wed, 2023-10-04 at 08:16 +0000, Gerd Petermann wrote:
Hi Ticker,
I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Ticker,
the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
Considering no_hp-overflow_alpha.patch:
It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion.
The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value.
The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally.
getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here?
Haven't looked at LineClipperTest yet.
Ticker
On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd I've now looked at your example (mp-gap2.osm). With my toMapUnit/toHighPrec patch the problem almost disappears. I don't see any significant difference when I also apply your mp-cut-coord- pool.patch. I can't see a way of fixing these gaps/overlaps without disabling a lot of filter point-removal, maybe by setting Coord.preserved() and ensuring added points for polygon cutting are on both the inner and outer polygon, which I thought they were, with the algorithm cutting through the complete structure and then bits being merged later if possible. Ticker On Thu, 2023-10-05 at 16:37 +0100, Ticker Berkin wrote:
Hi Gerd
I've changed the way mapUnit and highPrec Coords are calculated slightly so that the exact 1/2 unit boundary values are consistent between negative and positive. This gets rid of my couple of abnormalities with a -32 delta. The old method added or subtracted 1/2 to make ((int)double) do rounding, which assigns the exact 1/2 to the larger abs value.
toHighPrec(), as in yesterdays change, keeps its bounds consistent with the mapUnit it is splitting up but is now coded in a similar way to toMapUnit.
I think rounding to get MapUnits is a mistake but changing it now could have drastic consequences, it should have been floorToNegInfinity, giving the nice cyclic scale -128..+127 << 24. As it is, we get -128..+128 << 24.
Equator & Grenwhich Meridian behave well, but I've always had doubts about behaviour around 128
Hope this makes sense - patch attached
Sorry, haven't thought about the actual problem with the gaps yet.
Ticker
On Thu, 2023-10-05 at 07:30 +0000, Gerd Petermann wrote:
Hi Ticker,
reg. the change in toHighPrec() : I must confess that I never doubted why positive values should be rounded differently to negative values. This is also done in Utils.toMapUnit() and I just adapted that code. If you still want to change that I think Math.rint() would get you closer to the original implementation. Question is if or why this difference in rounding is/was needed. There should be visible effects near equator and at Greenwhich or at 180°.
Anyhow, I don't think this is the solution for the gap problem which is caused by further conversions from highprec int to double and back. An alternative solution for that problem would be the attached patch which makes use of the coord pool. No idea which one has more effects on performance.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 4. Oktober 2023 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
I've been thinking about how to stop the small ranges of decimal degrees from generating MapUnit/delta=+32 and MapUnit+1/delta=-32.
Without changing the MapUnit value, but truncating the highPrec calc, eg, in Coord.java
private static int toHighPrec(double degrees) { final double CVT_HP = ((double)(1 << HIGH_PREC_BITS)) / 360; return (int)Math.floor(degrees * CVT_HP); }
this problem almost goes away, with deltas between -31 .. +32 except for 2 instances of delta of -32 I get while building the Britain-and-ireland.osm.pbf. I need to work out why these are happening.
Although it is unnatural to have this range rather than -32..+31, it doesn't matter. It probably could be fixed by using Math.ceil instead or reversing where the delta goes. getAlternativePositions() will generated delta values approx -48..+48.
I don't get failures from LineClipperTest with this change. I do get failures from GmapsuppTest, SimpleRouteTest & SortTest, but I think these are not significant to this issue.
As far as the gap between areas is concerned, I haven't looked at this yet but I'll see if my change has similar effects to your patch to Coord.
Ticker
On Wed, 2023-10-04 at 08:16 +0000, Gerd Petermann wrote:
Hi Ticker,
I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Ticker,
the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
Considering no_hp-overflow_alpha.patch:
It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion.
The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value.
The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally.
getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here?
Haven't looked at LineClipperTest yet.
Ticker
On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Ticker, your patch rounds the problematic point to a different position, the same that is assigned to the new Coord instance that is derived from the cutOutInnerPolygons() method which involves singularAreaToPoints() and thus another place where rounding happens: int latHp = (int)Math.round(res[1] * (1<<Coord.DELTA_SHIFT)); int lonHp = (int)Math.round(res[0] * (1<<Coord.DELTA_SHIFT)); I think we have to check all nodes which are part of an inner and change position because of your different rounding method, right? I wouldn't be surprised to find cases where a gap occurs with (just) your patch. I'll try to find an example... If I can't we may just use your patch, although it's much harder to understand compared to the original code. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 6. Oktober 2023 16:19 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Gerd I've now looked at your example (mp-gap2.osm). With my toMapUnit/toHighPrec patch the problem almost disappears. I don't see any significant difference when I also apply your mp-cut-coord- pool.patch. I can't see a way of fixing these gaps/overlaps without disabling a lot of filter point-removal, maybe by setting Coord.preserved() and ensuring added points for polygon cutting are on both the inner and outer polygon, which I thought they were, with the algorithm cutting through the complete structure and then bits being merged later if possible. Ticker On Thu, 2023-10-05 at 16:37 +0100, Ticker Berkin wrote:
Hi Gerd
I've changed the way mapUnit and highPrec Coords are calculated slightly so that the exact 1/2 unit boundary values are consistent between negative and positive. This gets rid of my couple of abnormalities with a -32 delta. The old method added or subtracted 1/2 to make ((int)double) do rounding, which assigns the exact 1/2 to the larger abs value.
toHighPrec(), as in yesterdays change, keeps its bounds consistent with the mapUnit it is splitting up but is now coded in a similar way to toMapUnit.
I think rounding to get MapUnits is a mistake but changing it now could have drastic consequences, it should have been floorToNegInfinity, giving the nice cyclic scale -128..+127 << 24. As it is, we get -128..+128 << 24.
Equator & Grenwhich Meridian behave well, but I've always had doubts about behaviour around 128
Hope this makes sense - patch attached
Sorry, haven't thought about the actual problem with the gaps yet.
Ticker
On Thu, 2023-10-05 at 07:30 +0000, Gerd Petermann wrote:
Hi Ticker,
reg. the change in toHighPrec() : I must confess that I never doubted why positive values should be rounded differently to negative values. This is also done in Utils.toMapUnit() and I just adapted that code. If you still want to change that I think Math.rint() would get you closer to the original implementation. Question is if or why this difference in rounding is/was needed. There should be visible effects near equator and at Greenwhich or at 180°.
Anyhow, I don't think this is the solution for the gap problem which is caused by further conversions from highprec int to double and back. An alternative solution for that problem would be the attached patch which makes use of the coord pool. No idea which one has more effects on performance.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 4. Oktober 2023 15:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
I've been thinking about how to stop the small ranges of decimal degrees from generating MapUnit/delta=+32 and MapUnit+1/delta=-32.
Without changing the MapUnit value, but truncating the highPrec calc, eg, in Coord.java
private static int toHighPrec(double degrees) { final double CVT_HP = ((double)(1 << HIGH_PREC_BITS)) / 360; return (int)Math.floor(degrees * CVT_HP); }
this problem almost goes away, with deltas between -31 .. +32 except for 2 instances of delta of -32 I get while building the Britain-and-ireland.osm.pbf. I need to work out why these are happening.
Although it is unnatural to have this range rather than -32..+31, it doesn't matter. It probably could be fixed by using Math.ceil instead or reversing where the delta goes. getAlternativePositions() will generated delta values approx -48..+48.
I don't get failures from LineClipperTest with this change. I do get failures from GmapsuppTest, SimpleRouteTest & SortTest, but I think these are not significant to this issue.
As far as the gap between areas is concerned, I haven't looked at this yet but I'll see if my change has similar effects to your patch to Coord.
Ticker
On Wed, 2023-10-04 at 08:16 +0000, Gerd Petermann wrote:
Hi Ticker,
I've uploaded the test case: https://files.mkgmap.org.uk/detail/564 I had to modify the data a bit since the default style doesn't render natural=heath
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 2. Oktober 2023 16:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Ticker,
the word overflow might be confusing. The problem is that we want to use only 6 bits for the delta, but we store values from -32 .. 32. The special case with Michael example is that one coord with such an extreme delta is used (after converting to double) in Area.subtract() and the returned coord is converted back but get's the other extreme. In the end, only the 24 bit value is written to the map, and that causes the gap.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 2. Oktober 2023 15:55 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces
Hi Gerd
Considering no_hp-overflow_alpha.patch:
It seems wrong to change a 24bit coordinate. Utils.toMapUnit() is well defined to do the expected rounding with the conversion.
The actual deltas are local to Coord.java and, apart from their use by getAlternativePositions, are just used to get back to the highPrec coord value.
The deltas are stored as byte, so the value of +32 causes no arithmetic problems generally.
getAlternativePositions(), however, should handle any complications with the +32, but it looks like it mostly does, bumping up or down the 24bit coord if the abs(deltas) are > 16. It it really possible that modLxxDelta can overflow a byte here?
Haven't looked at LineClipperTest yet.
Ticker
On Fri, 2023-09-29 at 06:57 +0000, Gerd Petermann wrote:
Hi all,
I've found out that this problem is caused by a flaw in the "high precision" code. Under special conditions, two points with almost identical coordinates are internally represented with very different values. There is a possible overflow in the delta values, the value +32 should not occur as it cannot be represented with 6 bits, but the calculations produce this value. I think I have an ugly fix for this but the resulting map still shows (smaller) gaps and a unit test needs corrections, so there is more to do. Attached is a patch that checks for this overflow. Work in progress and maybe causes trouble in other areas, e.g. in South America where we have negative lat/lon values.
@Ticker: The unit test für LineClipperTest fails. I am not sure if only the test has to be changed (how) or if the code in LineClipper can be improved. I seem to remember that you suggested to use the code in ShapeSplitter instead.
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd My change to toMapUnit should be almost undetectable. Imagining mapUnits [x] as degrees over the real range -2.5 .. +2.5, the original implementation resulted in: -2.5 < [-2] <= -1.5 < [-1] <= -0.5 < [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5 I've changed it to give: -2.5 <= [-2] < -1.5 <- [-1] < -0.5 <= [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5 The change to highPrecCoord is so that 64 highPrecCoords are exactly within the appropriate mapUnit, with deltas of -31..+32 according to the existing delta calculations. Ticker On Fri, 2023-10-06 at 14:34 +0000, Gerd Petermann wrote:
Hi Ticker,
your patch rounds the problematic point to a different position, the same that is assigned to the new Coord instance that is derived from the cutOutInnerPolygons() method which involves singularAreaToPoints() and thus another place where rounding happens: int latHp = (int)Math.round(res[1] * (1<<Coord.DELTA_SHIFT)); int lonHp = (int)Math.round(res[0] * (1<<Coord.DELTA_SHIFT));
I think we have to check all nodes which are part of an inner and change position because of your different rounding method, right? I wouldn't be surprised to find cases where a gap occurs with (just) your patch. I'll try to find an example... If I can't we may just use your patch, although it's much harder to understand compared to the original code.
Gerd
Hi Gerd Sorry - bit hasty in my reply I need to look at the other places that use DELTA_SHIFT and check their arithmetic/rounding etc is consistent with the corrected highPrecCoord and uniform in rounding behaviour. Ticker On Fri, 2023-10-06 at 16:15 +0100, Ticker Berkin wrote:
Hi Gerd
My change to toMapUnit should be almost undetectable. Imagining mapUnits [x] as degrees over the real range -2.5 .. +2.5, the original implementation resulted in:
-2.5 < [-2] <= -1.5 < [-1] <= -0.5 < [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5
I've changed it to give:
-2.5 <= [-2] < -1.5 <- [-1] < -0.5 <= [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5
The change to highPrecCoord is so that 64 highPrecCoords are exactly within the appropriate mapUnit, with deltas of -31..+32 according to the existing delta calculations.
Ticker
On Fri, 2023-10-06 at 14:34 +0000, Gerd Petermann wrote:
Hi Ticker,
your patch rounds the problematic point to a different position, the same that is assigned to the new Coord instance that is derived from the cutOutInnerPolygons() method which involves singularAreaToPoints() and thus another place where rounding happens: int latHp = (int)Math.round(res[1] * (1<<Coord.DELTA_SHIFT)); int lonHp = (int)Math.round(res[0] * (1<<Coord.DELTA_SHIFT));
I think we have to check all nodes which are part of an inner and change position because of your different rounding method, right? I wouldn't be surprised to find cases where a gap occurs with (just) your patch. I'll try to find an example... If I can't we may just use your patch, although it's much harder to understand compared to the original code.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Ticker, I did some tests and your patch really seems to solve the problem without introducing new ones :) Just one remark: Please try to avoid terms like "+ve" in javadoc. I had to read this two times to understand that it means positive ;) So, if you don't find furher things to change I would be happy to commit the patch, although I don't yet understand why it also fixes the MP gaps issue. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 6. Oktober 2023 17:22 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Gaps in surfaces Hi Gerd Sorry - bit hasty in my reply I need to look at the other places that use DELTA_SHIFT and check their arithmetic/rounding etc is consistent with the corrected highPrecCoord and uniform in rounding behaviour. Ticker On Fri, 2023-10-06 at 16:15 +0100, Ticker Berkin wrote:
Hi Gerd
My change to toMapUnit should be almost undetectable. Imagining mapUnits [x] as degrees over the real range -2.5 .. +2.5, the original implementation resulted in:
-2.5 < [-2] <= -1.5 < [-1] <= -0.5 < [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5
I've changed it to give:
-2.5 <= [-2] < -1.5 <- [-1] < -0.5 <= [0] < -0.5 <= [1] < +1.5 <= [2] < +2.5
The change to highPrecCoord is so that 64 highPrecCoords are exactly within the appropriate mapUnit, with deltas of -31..+32 according to the existing delta calculations.
Ticker
On Fri, 2023-10-06 at 14:34 +0000, Gerd Petermann wrote:
Hi Ticker,
your patch rounds the problematic point to a different position, the same that is assigned to the new Coord instance that is derived from the cutOutInnerPolygons() method which involves singularAreaToPoints() and thus another place where rounding happens: int latHp = (int)Math.round(res[1] * (1<<Coord.DELTA_SHIFT)); int lonHp = (int)Math.round(res[0] * (1<<Coord.DELTA_SHIFT));
I think we have to check all nodes which are part of an inner and change position because of your different rounding method, right? I wouldn't be surprised to find cases where a gap occurs with (just) your patch. I'll try to find an example... If I can't we may just use your patch, although it's much harder to understand compared to the original code.
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd I've looked through all the places I could find that took [highPrec]coord vals, manipulated them and then converted back and they all seem safe with my patch. There might be slight differences where, using Java2DConverter or similar, a calculated value converts to a different/adjacent highPrec value from the old code; I don't think this matters. Original values will be restored to the same highPrec value. I'll try and remember not to use +ve and -ve; it is very common shorthand in English Ticker On Sat, 2023-10-07 at 07:07 +0000, Gerd Petermann wrote:
Hi Ticker,
I did some tests and your patch really seems to solve the problem without introducing new ones :)
Just one remark: Please try to avoid terms like "+ve" in javadoc. I had to read this two times to understand that it means positive ;)
So, if you don't find furher things to change I would be happy to commit the patch, although I don't yet understand why it also fixes the MP gaps issue.
Gerd
Hi Michael, please try new version r4914 which fixes the large gap in your example and probably some more issues like that. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Forstner Michael <forstner-m@a1.net> Gesendet: Dienstag, 26. September 2023 19:44 An: 'Development list for mkgmap' Betreff: Re: [mkgmap-dev] Gaps in surfaces Hello Gerd, here is the area: https://www.openstreetmap.org/#map=19/48.57117/13.96920 I understand that it is necessary to make rounding. If a point of two surfaces is equal in OSM, the point should also be equal in Garmin. Thank you! Best regards, Michael _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (3)
-
Forstner Michael -
Gerd Petermann -
Ticker Berkin