
Hello list I'm getting the error below processing a tile from Vietnam: Exception in thread "main" java.lang.AssertionError at uk.me.parabola.imgfmt.app.net.RoadIndex.write(RoadIndex.java:45) at uk.me.parabola.imgfmt.app.net.RoadDef.writeLevelDivs(RoadDef.java:356) at uk.me.parabola.imgfmt.app.net.RoadDef.writeNet1(RoadDef.java:237) at uk.me.parabola.imgfmt.app.net.NETFile.write(NETFile.java:68) at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:355) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:114) at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:70) at uk.me.parabola.mkgmap.main.Main.lambda$processFilename$1(Main.java:291) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor .java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecuto r.java:628) at java.base/java.lang.Thread.run(Thread.java:829) The simplest command triggering the error is: java -jar mkgmap.jar --route 51120021.o5m Tile can be downloaded from https://alternativaslibres.org/tmp/51120021.o5m My mkgmap version is 4907

Hi Carlos I've tried this on my latest build which is r4907 + quite a few other changes and I don't get the problem, but using an older, downloaded, version I can reproduce it. My changes don't look relevant to this issue; mainly character set issues and some line/polygon filter reordering. The other difference I have is building with Java 17 on a unix machine. I'll investigate further. The problem is related to a road with >255 line segments. NB: need option -ea for the exception to happen. Ticker

Thanks for investigating it. This is my java environment: openjdk version "11.0.18" 2023-01-17 OpenJDK Runtime Environment (build 11.0.18+10-post-Debian-1deb11u1) OpenJDK 64-Bit Server VM (build 11.0.18+10-post-Debian-1deb11u1, mixed mode, sharing) El 1/6/23 a las 11:28, Ticker Berkin escribió:
Hi Carlos
I've tried this on my latest build which is r4907 + quite a few other changes and I don't get the problem, but using an older, downloaded, version I can reproduce it.
My changes don't look relevant to this issue; mainly character set issues and some line/polygon filter reordering. The other difference I have is building with Java 17 on a unix machine.
I'll investigate further. The problem is related to a road with >255 line segments.
NB: need option -ea for the exception to happen.
Ticker
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd & Carlos I've found the problem and attach a patch which reduces the Subdivision line limit to 254. Subdivisions/areas have a set of limits, one of which was <= 255 lines. These lines can be of any type including roads. When --route, a reference to each line/road part is created, but these are numbered from 1, so the 255th line in a SubDivision is numbered 256 as a road segment and this can't be represented as a byte in the NET structures. It looks like this problem has always been there, but is most likely to manifest itself in areas with mostly roads. Ticker

Hi Ticker, thanks for the patch. Some unit tests need changes. Please check if my changes are plausible. Something might be wrong on my system because SimpleRouteTest failed also without your patch. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 2. Juni 2023 17:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd & Carlos I've found the problem and attach a patch which reduces the Subdivision line limit to 254. Subdivisions/areas have a set of limits, one of which was <= 255 lines. These lines can be of any type including roads. When --route, a reference to each line/road part is created, but these are numbered from 1, so the 255th line in a SubDivision is numbered 256 as a road segment and this can't be represented as a byte in the NET structures. It looks like this problem has always been there, but is most likely to manifest itself in areas with mostly roads. Ticker

Hi Gerd Our messages crossed. I'll investigate However SimpleRouteTest just checks on the sizes of TRE and RGN; likewise GmapsuppTest, but SimpleTest is more interesting. Ticker On Sat, 2023-06-03 at 09:09 +0000, Gerd Petermann wrote:
Hi Ticker,
thanks for the patch. Some unit tests need changes. Please check if my changes are plausible. Something might be wrong on my system because SimpleRouteTest failed also without your patch.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 2. Juni 2023 17:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile
Hi Gerd & Carlos
I've found the problem and attach a patch which reduces the Subdivision line limit to 254.
Subdivisions/areas have a set of limits, one of which was <= 255 lines. These lines can be of any type including roads. When --route, a reference to each line/road part is created, but these are numbered from 1, so the 255th line in a SubDivision is numbered 256 as a road segment and this can't be represented as a byte in the NET structures.
It looks like this problem has always been there, but is most likely to manifest itself in areas with mostly roads.
Ticker
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Sorry - this isn't the correct explanation. Reducing MapSplitter.MAX_NUM_LINES fixes the problem but there must be something else going on that adds 1 more line. I'll investigate a bit more. Ticker On Fri, 2023-06-02 at 16:47 +0100, Ticker Berkin wrote:
Hi Gerd & Carlos
I've found the problem and attach a patch which reduces the Subdivision line limit to 254.
Subdivisions/areas have a set of limits, one of which was <= 255 lines. These lines can be of any type including roads. When --route, a reference to each line/road part is created, but these are numbered from 1, so the 255th line in a SubDivision is numbered 256 as a road segment and this can't be represented as a byte in the NET structures.
It looks like this problem has always been there, but is most likely to manifest itself in areas with mostly roads.
Ticker
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email. The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work. Attached is patch of the code I've been using appropriate to trunk. Ticker -------- Forwarded Message -------- From: Ticker Berkin <rwb-mkgmap@jagit.co.uk> To: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100 Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved

Hi Ticker, The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email. The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work. Attached is patch of the code I've been using appropriate to trunk. Ticker -------- Forwarded Message -------- From: Ticker Berkin <rwb-mkgmap@jagit.co.uk> To: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100 Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved

Hi Ticker, okay, after some more debugging I think I understand. The problem is/was that the code in PredictFilterPoints assumes that the obsolete point removal happens before line splitting, but the actual order in MapBuilder is/was different (and wrong). Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 5. Juni 2023 08:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Ticker, The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email. The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work. Attached is patch of the code I've been using appropriate to trunk. Ticker -------- Forwarded Message -------- From: Ticker Berkin <rwb-mkgmap@jagit.co.uk> To: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100 Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd Yes. MapSplitter enforces all the subDivision limits (#items and #pointsInItems) before the filtering has happened. A Polyline might need to be split into many lines if it has a large number of points, but, at a given resolution, many points might be discarded. Using this fact it can estimate the worst-case number of line splits to use in the subDivision line count limit. This logic makes the most significant difference at low resolutions but, in Carlos's data, there must have been roads with multiple points in res24 squares. Ticker On Mon, 2023-06-05 at 06:48 +0000, Gerd Petermann wrote:
Hi Ticker,
okay, after some more debugging I think I understand. The problem is/was that the code in PredictFilterPoints assumes that the obsolete point removal happens before line splitting, but the actual order in MapBuilder is/was different (and wrong).
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 5. Juni 2023 08:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile
Hi Ticker,
The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile
Hi Gerd
Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email.
The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work.
Attached is patch of the code I've been using appropriate to trunk.
Ticker
-------- Forwarded Message -------- From: Ticker Berkin <rwb-mkgmap@jagit.co.uk> To: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100
Hi Gerd
I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes.
My conclusions are:
Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these.
Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter.
I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant.
For the contourFilters, I've left DP as the first filter but moved _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, thanks, I've committed the patch with r4909. What do you think about the attached patch to improve readability? Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 5. Juni 2023 09:54 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd Yes. MapSplitter enforces all the subDivision limits (#items and #pointsInItems) before the filtering has happened. A Polyline might need to be split into many lines if it has a large number of points, but, at a given resolution, many points might be discarded. Using this fact it can estimate the worst-case number of line splits to use in the subDivision line count limit. This logic makes the most significant difference at low resolutions but, in Carlos's data, there must have been roads with multiple points in res24 squares. Ticker On Mon, 2023-06-05 at 06:48 +0000, Gerd Petermann wrote:
Hi Ticker,
okay, after some more debugging I think I understand. The problem is/was that the code in PredictFilterPoints assumes that the obsolete point removal happens before line splitting, but the actual order in MapBuilder is/was different (and wrong).
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Montag, 5. Juni 2023 08:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile
Hi Ticker,
The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile
Hi Gerd
Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email.
The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work.
Attached is patch of the code I've been using appropriate to trunk.
Ticker
-------- Forwarded Message -------- From: Ticker Berkin <rwb-mkgmap@jagit.co.uk> To: Development list for mkgmap <mkgmap-dev@lists.mkgmap.org.uk> Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100
Hi Gerd
I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes.
My conclusions are:
Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these.
Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter.
I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant.
For the contourFilters, I've left DP as the first filter but moved _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd The added comments are helpful but overall I think the readability is worse; it takes careful reading to see where the dp filter is placed. If other tweaks to different orderings between normal and parallel chains are necessary it will become even more confusing. Ticker On Mon, 2023-06-05 at 09:41 +0000, Gerd Petermann wrote:
Hi Ticker,
thanks, I've committed the patch with r4909. What do you think about the attached patch to improve readability?
Gerd

Hi Ticker, okay, it will probably always be hard to understand that part of the code. So let's see what comes next... Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 5. Juni 2023 12:39 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd The added comments are helpful but overall I think the readability is worse; it takes careful reading to see where the dp filter is placed. If other tweaks to different orderings between normal and parallel chains are necessary it will become even more confusing. Ticker On Mon, 2023-06-05 at 09:41 +0000, Gerd Petermann wrote:
Hi Ticker,
thanks, I've committed the patch with r4909. What do you think about the attached patch to improve readability?
Gerd
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
participants (3)
-
Carlos Dávila
-
Gerd Petermann
-
Ticker Berkin