Hi Ticker, see http://www.mkgmap.org.uk/download/mkgmap-NET-no-NOD-r4301.zip You just have scroll down on the normal download page to see the branch builds. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 13:57 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd Is there a build available for this that I can download, rather than creating a branch. Thanks Ticker On Sat, 2019-10-12 at 10:17 +0000, Gerd Petermann wrote:

Hi all,

I've tried to work out under what conditions we need routing nodes when using OSM input (not polish *mp): See http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4300 for details.

Please try the new version with a few tiles (maybe 4). The version r4301 produces some messages like SCHWERWIEGEND (StyledConverter): d:\mkgmap\test\resources\in\osm\uk -test-1.osm.gz: check: road without connection is not written to NOD Wildwood Rise (OSM id 4013934) 51.570805,-0.178385 SCHWERWIEGEND (StyledConverter): f:\osm\wildeshausen.o5m: check: road without connection is not written to NOD (OSM id 243082453) 52.903136,8.414926

These are not errors, they just help to find the places where the result of the branch version is (very) different.

Please compare a map produced with the trunk version (r4299) with a new map produced with the branch version and check address search and routing at some places printed with these error messages. Please try also new experimental option --x-all-roads-to-nod . If given, mkgmap will add at least one routing node to each routable line so that it is added to NOD.

I hope the program is stable, so you should not see crashes, else please let me know details. Pleaase let me know which result you prefer. 1) trunk r4299 2) branch r4301 WITHOUT option --x-all-roads-to-nod 3) branch r4301 WITH option --x-all-roads-to-nod

1) should give the largest file, 2) the smallest, 3) between

Gerd

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, I just committed r4302, please try with that. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 17:07 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd I'm just starting to test this, but looking through the log lines (I get 1260 of them for 2 tiles) like: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210001.osm.pbf: check: road without connection is not written to NOD Newport to Cowes cycleway (OSM id 562475661) 50.749482,-1.294847 there are some, eg the above, that I don't expect to see as it is joined at both ends. I'll continue to investigate and I'm loading onto my device to test. Ticker On Sat, 2019-10-12 at 10:17 +0000, Gerd Petermann wrote:

Hi all,

I've tried to work out under what conditions we need routing nodes when using OSM input (not polish *mp): See http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4300 for details.

Please try the new version with a few tiles (maybe 4). The version r4301 produces some messages like SCHWERWIEGEND (StyledConverter): d:\mkgmap\test\resources\in\osm\uk -test-1.osm.gz: check: road without connection is not written to NOD Wildwood Rise (OSM id 4013934) 51.570805,-0.178385 SCHWERWIEGEND (StyledConverter): f:\osm\wildeshausen.o5m: check: road without connection is not written to NOD (OSM id 243082453) 52.903136,8.414926

These are not errors, they just help to find the places where the result of the branch version is (very) different.

Please compare a map produced with the trunk version (r4299) with a new map produced with the branch version and check address search and routing at some places printed with these error messages. Please try also new experimental option --x-all-roads-to-nod . If given, mkgmap will add at least one routing node to each routable line so that it is added to NOD.

I hope the program is stable, so you should not see crashes, else please let me know details. Pleaase let me know which result you prefer. 1) trunk r4299 2) branch r4301 WITHOUT option --x-all-roads-to-nod 3) branch r4301 WITH option --x-all-roads-to-nod

1) should give the largest file, 2) the smallest, 3) between

Gerd

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Ticker Berkin wrote

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about? Gerd -- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html

Hi Ticker, please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here? A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc. The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD. The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line. Ticker On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, thanks for testing. I'll work on a patch without recursive calls. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 12:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd I've applied the patch to my 'current' version and tried running it but it gives: java.lang.StackOverflowError at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:938) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) ... 1020 lines like this ... at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) Exiting - if you want to carry on regardless, use the --keep-going option My source had the patches "avoid-to-split-via-ways.patch" and "only_with_via_ways.patch". I can remove these and try again if you think there might be an interaction. I don't think there is a need to try and check on islands of different access modes; the apparent behaviour of my device is that it finds the closest highway of any type to get into or out of the main road network. ie, if here is a footpath closer to the destination that any motor-vehicle road, car route planning will direct me onto it. Ticker On Fri, 2019-10-18 at 09:25 +0000, Gerd Petermann wrote:

Hi Ticker,

please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here?

A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc.

The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD.

The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line.

Ticker

On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

...

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd That runs OK. As before, I get lots of 2 node roads - often bridges over streams in open land, and quite a few (about 1/10 the number of the 2 noders) are 3 and 4 nodes - frequently paths between buildings in schools / campuses and short bits of path at either end of the bridges in open land. I upped the test from 5 to 10 and got more of the same + networks of short paths on golf courses, walkways on piers (which should have been connected to something), paths in an enclosed quadrangle, etc, etc All of these I would consider to be a hindrance to route calculations. Having an option, defaulting to, say, 10, to stop these road-islands being added to NOD must be a good idea. Setting the value to 0 would give the current behaviour Ticker On Fri, 2019-10-18 at 12:35 +0000, Gerd Petermann wrote:

Hi Ticker,

here is the patch without recursive call.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 12:46 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

thanks for testing. I'll work on a patch without recursive calls.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 12:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I've applied the patch to my 'current' version and tried running it but it gives:

java.lang.StackOverflowError at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:938) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) ... 1020 lines like this ... at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) Exiting - if you want to carry on regardless, use the --keep-going option

My source had the patches "avoid-to-split-via-ways.patch" and "only_with_via_ways.patch". I can remove these and try again if you think there might be an interaction.

I don't think there is a need to try and check on islands of different access modes; the apparent behaviour of my device is that it finds the closest highway of any type to get into or out of the main road network. ie, if here is a footpath closer to the destination that any motor-vehicle road, car route planning will direct me onto it.

Ticker

On Fri, 2019-10-18 at 09:25 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here?

A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc.

The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD.

The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line.

Ticker

On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

...

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, I think it would be better to measure the sum of road lengths instead of the number of nodes. In some areas you may have unconnected roads with many km length, e.g. in Canada or Australia. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 17:02 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Ticker, okay, I am working on the part that removes those islands from NOD. I am developing it with the NET-no-NOD branch. With trunk you get higher counts as each end of an unconnected way is also a routing node. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 16:28 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd That runs OK. As before, I get lots of 2 node roads - often bridges over streams in open land, and quite a few (about 1/10 the number of the 2 noders) are 3 and 4 nodes - frequently paths between buildings in schools / campuses and short bits of path at either end of the bridges in open land. I upped the test from 5 to 10 and got more of the same + networks of short paths on golf courses, walkways on piers (which should have been connected to something), paths in an enclosed quadrangle, etc, etc All of these I would consider to be a hindrance to route calculations. Having an option, defaulting to, say, 10, to stop these road-islands being added to NOD must be a good idea. Setting the value to 0 would give the current behaviour Ticker On Fri, 2019-10-18 at 12:35 +0000, Gerd Petermann wrote:

Hi Ticker,

here is the patch without recursive call.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 12:46 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

thanks for testing. I'll work on a patch without recursive calls.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 12:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I've applied the patch to my 'current' version and tried running it but it gives:

java.lang.StackOverflowError at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:938) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) ... 1020 lines like this ... at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) Exiting - if you want to carry on regardless, use the --keep-going option

My source had the patches "avoid-to-split-via-ways.patch" and "only_with_via_ways.patch". I can remove these and try again if you think there might be an interaction.

I don't think there is a need to try and check on islands of different access modes; the apparent behaviour of my device is that it finds the closest highway of any type to get into or out of the main road network. ie, if here is a footpath closer to the destination that any motor-vehicle road, car route planning will direct me onto it.

Ticker

On Fri, 2019-10-18 at 09:25 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here?

A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc.

The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD.

The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line.

Ticker

On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

...

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, yes, it is already quite complicated. In Australia I see islands of private tracks with a total length of > 50.000 m. I've also not yet decided where the checks should be done (StyledConverter or RoadNetwork). StyledConverter still knows the OSM way, RoadNetwork doesn't. On the other hand RoadNetwork does all the network calculations and therefore it seems stupid to code them also in StyledConverter. Please check r4304: http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4304 http://www.mkgmap.org.uk/download/mkgmap-NET-no-NOD-r4304.zip Sample output: SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: check for routing islands found 5 islands SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island with 1 routing node(s) at 51.554362,-0.197136 has length 156 m SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island with 2 routing node(s) at 51.623528,-0.143968 has length 263 m SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island with 2 routing node(s) at 51.623368,-0.146458 has length 1063 m SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island with 3 routing node(s) at 51.640080,-0.143457 has length 282 m SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island with 4 routing node(s) at 51.583387,-0.157835 has length 2250 m SEVERE (RoadNetwork): test/resources/in/osm/uk-test-1.osm.gz: routing island check took 20 ms All islands are reported, no matter how many nodes there are. There is no particular order for now. Sample output when option --x-check-routing-island-len=500 is used: ... SEVERE (RoadNetwork): e:\osm_out_work\wales\63240001.osm.pbf: routing island with 2 routing node(s) at 53.326519,-3.453854 has length 697 m SEVERE (RoadNetwork): e:\osm_out_work\wales\63240001.osm.pbf: routing island with 2 routing node(s) at 53.314260,-3.502073 has length 106 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855016) is removed from NOD, length: 105 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855016) is removed from NOD, length: 1 m SEVERE (RoadNetwork): e:\osm_out_work\wales\63240001.osm.pbf: routing island with 2 routing node(s) at 53.313371,-3.501693 has length 109 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855022) is removed from NOD, length: 107 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855022) is removed from NOD, length: 2 m SEVERE (RoadNetwork): e:\osm_out_work\wales\63240001.osm.pbf: routing island with 2 routing node(s) at 53.313692,-3.502430 has length 25 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855021) is removed from NOD, length: 22 m SEVERE (RoadDef): e:\osm_out_work\wales\63240001.osm.pbf: road (http://www.openstreetmap.org/browse/way/511855021) is removed from NOD, length: 3 m ... It shows that many islands are in fact closed Ways which were split for routing purposes, ofter areas. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 19. Oktober 2019 12:40 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd Maybe yes, but a long unconnected road is no use for route generation except to somewhere else near the same road, which doesn't need route generation anyway. Trying to get to anywhere else will probably give a "route calculation error" It occurs to me that almost the examples I've found are small networks of paths where motor-vehicle=no. These are often the ones that get picked up as the closest for starting/ending a car journey (wrong in itself) and then cause "route calculation error" because of the isolation from the main network. Could there be some options to control the cut-off based on number of nodes, total length, and non-motor vehicle? I realise this is getting complicated. Ticker On Fri, 2019-10-18 at 15:51 +0000, Gerd Petermann wrote:

Hi Ticker,

I think it would be better to measure the sum of road lengths instead of the number of nodes. In some areas you may have unconnected roads with many km length, e.g. in Canada or Australia.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 17:02 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

okay, I am working on the part that removes those islands from NOD. I am developing it with the NET-no-NOD branch. With trunk you get higher counts as each end of an unconnected way is also a routing node.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 16:28 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

That runs OK.

As before, I get lots of 2 node roads - often bridges over streams in open land, and quite a few (about 1/10 the number of the 2 noders) are 3 and 4 nodes - frequently paths between buildings in schools / campuses and short bits of path at either end of the bridges in open land.

I upped the test from 5 to 10 and got more of the same + networks of short paths on golf courses, walkways on piers (which should have been connected to something), paths in an enclosed quadrangle, etc, etc

All of these I would consider to be a hindrance to route calculations.

Having an option, defaulting to, say, 10, to stop these road-islands being added to NOD must be a good idea. Setting the value to 0 would give the current behaviour

Ticker

On Fri, 2019-10-18 at 12:35 +0000, Gerd Petermann wrote:

...

Hi Ticker,

here is the patch without recursive call.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 12:46 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

thanks for testing. I'll work on a patch without recursive calls.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 12:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I've applied the patch to my 'current' version and tried running it but it gives:

java.lang.StackOverflowError at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:938 ) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941 ) ... 1020 lines like this ... at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941 ) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941 ) Exiting - if you want to carry on regardless, use the --keep-going option

My source had the patches "avoid-to-split-via-ways.patch" and "only_with_via_ways.patch". I can remove these and try again if you think there might be an interaction.

I don't think there is a need to try and check on islands of different access modes; the apparent behaviour of my device is that it finds the closest highway of any type to get into or out of the main road network. ie, if here is a footpath closer to the destination that any motor-vehicle road, car route planning will direct me onto it.

Ticker

On Fri, 2019-10-18 at 09:25 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here?

A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc.

The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD.

The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line.

Ticker

On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

...

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road?

Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700 Some build statistics: SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912 For comparison, testing with r4295 (I get similar figures for trunk+): Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912 So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style: It adds quite a lot of processing time (increased by 66%) Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%. It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles. I vote for keeping something like this provided the routing-island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len) The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic. Ticker

Hi Gerd These are with default style. mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700 SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms Total time taken: 1 minute 12 seconds gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912 With r4295: Total time taken: 1 minute 3 seconds gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912 Ticker On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing-island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd split.list contains: 74210001: 2362592,-73088 to 2376288,-50304 Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them. splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml >splitter.log 2>&1 # significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0 Command line is: java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x-check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt I'm going to be out for the next few hours, but can do the log stuff later if you need it. Ticker On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing-island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4317 Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Ticker, okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds Most of the additional time is used to remove the nodes from NOD. This can be improved... Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd split.list contains: 74210001: 2362592,-73088 to 2376288,-50304 Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them. splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml >splitter.log 2>&1 # significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0 Command line is: java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x-check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt I'm going to be out for the next few hours, but can do the log stuff later if you need it. Ticker On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing-island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, so you get nearly no improvement. Strange. What do you see with default style? Regarding the messages from StyledConverter: That's what I am working on. Question is wheter we should add those obvious routing islands to the road network and let island removal take care of it or if StyledConverter should also calculate the island length given by --x-check-routing-island-len before removing it. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 12:38 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd with mkgmap-NET-no-NOD-r4317: SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: search for routing islands found 293 islands in 65 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island removal took 1953 ms EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: search for routing islands found 2820 islands in 62 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island removal took 17696 ms Total time taken: 1 minute 20 seconds 13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap-NET-no -NOD-r4304) It still give lots (691) of these messages: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210002.osm.pbf: check: road without connection is not written to NOD (OSM id 40586008) 51.280882,-1.096959 Ticker On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:

Hi Ticker,

fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4317

Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds

Most of the additional time is used to remove the nodes from NOD. This can be improved...

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

split.list contains: 74210001: 2362592,-73088 to 2376288,-50304

Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them.

splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml

...

splitter.log 2>&1

# significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0

Command line is:

java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x -check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt

I'm going to be out for the next few hours, but can do the log stuff later if you need it.

Ticker

On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing -island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd The last test was with my style. The routing island check was faster: r4317 r4309 ms 74210002: 593 5252 74310001: 17696 45755 The style I'm testing with does add a lot of footways (around car parks) and has the highway/barrier point > highway splitting. I've got to be out for the rest of the day now, but I can send you the data later tonight or tomorrow morning. Ticker On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:

Hi Ticker,

if I got that right the new version is slower for you. I see two possible reasons: - bottleneck in heap memory when processing multiple files at the same time (max-jobs) - your style adds a lot more roads than the default style. For my tests I've already used a modified default style with the highway=traffic_signals | highway=crossing {set mkgmap:road-speed=1} rule added to points.

Maybe you can analyse this on your side? Or you can send me your complete test data (the two input files created by splitter and your style files) to gptermann_muenchen <at> hotmail.com?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 23. Oktober 2019 12:49 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

so you get nearly no improvement. Strange. What do you see with default style?

Regarding the messages from StyledConverter: That's what I am working on. Question is wheter we should add those obvious routing islands to the road network and let island removal take care of it or if StyledConverter should also calculate the island length given by --x -check-routing-island-len before removing it.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 12:38 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

with mkgmap-NET-no-NOD-r4317:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: search for routing islands found 293 islands in 65 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island removal took 1953 ms EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: search for routing islands found 2820 islands in 62 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island removal took 17696 ms Total time taken: 1 minute 20 seconds

13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap-NET -no -NOD-r4304)

It still give lots (691) of these messages: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210002.osm.pbf: check: road without connection is not written to NOD (OSM id 40586008) 51.280882,-1.096959

Ticker

On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:

...

Hi Ticker,

fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=431 7

Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds

Most of the additional time is used to remove the nodes from NOD. This can be improved...

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

split.list contains: 74210001: 2362592,-73088 to 2376288,-50304

Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them.

splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml

...

splitter.log 2>&1

# significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0

Command line is:

java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x -check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt

I'm going to be out for the next few hours, but can do the log stuff later if you need it.

Ticker

On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing -island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd $ java -version openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-b14) OpenJDK Server VM (build 25.91-b14, mixed mode) Ticker On Wed, 2019-10-23 at 16:21 +0000, Gerd Petermann wrote:

Hi Ticker,

okay, please let me also know the output of command java -version

Mine is: e:\ld>java -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 13:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

The last test was with my style. The routing island check was faster: r4317 r4309 ms 74210002: 593 5252 74310001: 17696 45755

The style I'm testing with does add a lot of footways (around car parks) and has the highway/barrier point > highway splitting.

I've got to be out for the rest of the day now, but I can send you the data later tonight or tomorrow morning.

Ticker

On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:

...

Hi Ticker,

if I got that right the new version is slower for you. I see two possible reasons: - bottleneck in heap memory when processing multiple files at the same time (max-jobs) - your style adds a lot more roads than the default style. For my tests I've already used a modified default style with the highway=traffic_signals | highway=crossing {set mkgmap:road -speed=1} rule added to points.

Maybe you can analyse this on your side? Or you can send me your complete test data (the two input files created by splitter and your style files) to gptermann_muenchen <at> hotmail.com?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 23. Oktober 2019 12:49 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

so you get nearly no improvement. Strange. What do you see with default style?

Regarding the messages from StyledConverter: That's what I am working on. Question is wheter we should add those obvious routing islands to the road network and let island removal take care of it or if StyledConverter should also calculate the island length given by - -x -check-routing-island-len before removing it.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 12:38 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

with mkgmap-NET-no-NOD-r4317:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: search for routing islands found 293 islands in 65 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island removal took 1953 ms EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: search for routing islands found 2820 islands in 62 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island removal took 17696 ms Total time taken: 1 minute 20 seconds

13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap -NET -no -NOD-r4304)

It still give lots (691) of these messages: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210002.osm.pbf: check: road without connection is not written to NOD (OSM id 40586008) 51.280882,-1.096959

Ticker

On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:

...

Hi Ticker,

fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4 31 7

Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds

Most of the additional time is used to remove the nodes from NOD. This can be improved...

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

split.list contains: 74210001: 2362592,-73088 to 2376288,-50304

Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them.

splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml

...

splitter.log 2>&1

# significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0

Command line is:

java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x -check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt

I'm going to be out for the next few hours, but can do the log stuff later if you need it.

Ticker

On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing -island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker sorry, just noticed that I had a typo in the mail address. First part is gpetermann_muenchen Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 23. Oktober 2019 18:57 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Ticker, wow, that is even older than mine ;) Might be worth to try an update to the latest 1.8 release Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 18:43 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd $ java -version openjdk version "1.8.0_91" OpenJDK Runtime Environment (build 1.8.0_91-b14) OpenJDK Server VM (build 25.91-b14, mixed mode) Ticker On Wed, 2019-10-23 at 16:21 +0000, Gerd Petermann wrote:

Hi Ticker,

okay, please let me also know the output of command java -version

Mine is: e:\ld>java -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 13:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

The last test was with my style. The routing island check was faster: r4317 r4309 ms 74210002: 593 5252 74310001: 17696 45755

The style I'm testing with does add a lot of footways (around car parks) and has the highway/barrier point > highway splitting.

I've got to be out for the rest of the day now, but I can send you the data later tonight or tomorrow morning.

Ticker

On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:

...

Hi Ticker,

if I got that right the new version is slower for you. I see two possible reasons: - bottleneck in heap memory when processing multiple files at the same time (max-jobs) - your style adds a lot more roads than the default style. For my tests I've already used a modified default style with the highway=traffic_signals | highway=crossing {set mkgmap:road -speed=1} rule added to points.

Maybe you can analyse this on your side? Or you can send me your complete test data (the two input files created by splitter and your style files) to gptermann_muenchen <at> hotmail.com?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 23. Oktober 2019 12:49 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

so you get nearly no improvement. Strange. What do you see with default style?

Regarding the messages from StyledConverter: That's what I am working on. Question is wheter we should add those obvious routing islands to the road network and let island removal take care of it or if StyledConverter should also calculate the island length given by - -x -check-routing-island-len before removing it.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 12:38 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

with mkgmap-NET-no-NOD-r4317:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: search for routing islands found 293 islands in 65 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island removal took 1953 ms EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: search for routing islands found 2820 islands in 62 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island removal took 17696 ms Total time taken: 1 minute 20 seconds

13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap -NET -no -NOD-r4304)

It still give lots (691) of these messages: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210002.osm.pbf: check: road without connection is not written to NOD (OSM id 40586008) 51.280882,-1.096959

Ticker

On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:

...

Hi Ticker,

fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4 31 7

Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds

Most of the additional time is used to remove the nodes from NOD. This can be improved...

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

split.list contains: 74210001: 2362592,-73088 to 2376288,-50304

Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them.

splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml

...

splitter.log 2>&1

# significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0

Command line is:

java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x -check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt

I'm going to be out for the next few hours, but can do the log stuff later if you need it.

Ticker

On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

These are with default style.

mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms

Total time taken: 1 minute 12 seconds

gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912

With r4295:

Total time taken: 1 minute 3 seconds

gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912

Ticker

On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:

...

Hi Ticker,

strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700

Some build statistics:

SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms

Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912

For comparison, testing with r4295 (I get similar figures for trunk+):

Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912

So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:

It adds quite a lot of processing time (increased by 66%)

Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.

It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.

I vote for keeping something like this provided the routing -island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)

The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.

Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Randolph, I don't see anything special in this area. Please describe more detailed what you are trying to do. Tell us the input for splitter and the options in use. What error message do you see? It seems this is more related to splitter, not the NET-no-NOD branch? If yes, please open a new thread for this. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Randolph J. Herber <army.bronze.star@gmail.com> Gesendet: Samstag, 19. Oktober 2019 15:51 An: Development list for mkgmap; Ticker Berkin Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD There is an area like that is the older part of Port Said near Cairo, Egypt, that blows up map making in mkgmap. Examine the area in the vicinity of 31.0755N 32.2661E. There are many single block streets. This causes problems as there is no parameter setting a limit on the number of ways similar to the parameter setting a limit on the number of nodes in a tile. In this area, there are almost half as many ways as there are nodes. Perhaps, after the initial tile split in splitter, if the number of ways is excessive, just that tile could be split until the number of ways becomes "reasonable." Randolph J. Herber On 10/18/2019 9:28 AM, Ticker Berkin wrote:

Hi Gerd

That runs OK.

As before, I get lots of 2 node roads - often bridges over streams in open land, and quite a few (about 1/10 the number of the 2 noders) are 3 and 4 nodes - frequently paths between buildings in schools / campuses and short bits of path at either end of the bridges in open land.

I upped the test from 5 to 10 and got more of the same + networks of short paths on golf courses, walkways on piers (which should have been connected to something), paths in an enclosed quadrangle, etc, etc

All of these I would consider to be a hindrance to route calculations.

Having an option, defaulting to, say, 10, to stop these road-islands being added to NOD must be a good idea. Setting the value to 0 would give the current behaviour

Ticker

On Fri, 2019-10-18 at 12:35 +0000, Gerd Petermann wrote:

...

Hi Ticker,

here is the patch without recursive call.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Freitag, 18. Oktober 2019 12:46 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

thanks for testing. I'll work on a patch without recursive calls.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Freitag, 18. Oktober 2019 12:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I've applied the patch to my 'current' version and tried running it but it gives:

java.lang.StackOverflowError at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:938) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) ... 1020 lines like this ... at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) at uk.me.parabola.imgfmt.app.net.RouteNode.visitNet(RouteNode.java:941) Exiting - if you want to carry on regardless, use the --keep-going option

My source had the patches "avoid-to-split-via-ways.patch" and "only_with_via_ways.patch". I can remove these and try again if you think there might be an interaction.

I don't think there is a need to try and check on islands of different access modes; the apparent behaviour of my device is that it finds the closest highway of any type to get into or out of the main road network. ie, if here is a footpath closer to the destination that any motor-vehicle road, car route planning will direct me onto it.

Ticker

On Fri, 2019-10-18 at 09:25 +0000, Gerd Petermann wrote:

...

Hi Ticker,

please check: attached is a simple patch that implements the calculation of routing islands. It just reports islands with less than 5 routing nodes and the position of one of the nodes. It ignores such islands which have at least one node that is a boundary node. Remember that we also create nodes oncountry borders. Maybe those should be ignored here?

A more detailed test might also check the access attributes, so that we report islands for pedestrian, bicycle etc.

The patch doesn't change the data written to the img files. Please play with it and let me know how it works for you.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 12. Oktober 2019 19:34 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

I was thinking of a threshold (maybe < 5) and then not adding any of them to NOD.

The reason is that a while ago I found many instances where tracks lead up to the edge of car-parks but didn't join to each other or the car -park access road and so walking routing, where one was expected to cross the car-park, didn't work. I tried adding a footway around the edge of the car-park and this helped in a lot of cases but I got driving route-calculation-error in or out of the car-park if the access road wasn't correctly specified. Your latest change will help in a lot of instances but sometimes there car-park was defined by more than 1 line.

Ticker

On Sat, 2019-10-12 at 10:10 -0700, Gerd Petermann wrote:

...

Ticker Berkin wrote

...

Do you attempt to isolate small road networks that are not connected to the rest of the system or just a single road? Not yet. Do you think about some kind of threshold value giving the minimal number of connections or maybe a bbox size or a sum of road lengths? if the isolated network is "small" we might not add any of its roads to NOD. Is that what you think about?

Gerd

-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

---

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Ticker, hmm, the restriction is not correct as the role for way https://www.openstreetmap.org/way/153273366 is empty. Do you think that mkgmap should start guessing what the role of the way is? In this case that would be possible, but I'd still prefer that someone fixes the data in OSM. Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 21. Oktober 2019 09:39 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd Just doing more experimenting with mkgmap-NET-no-NOD-r4304 and options: --no-route --net I get lots and lots of messages like: WARN: uk.me.parabola.mkgmap.reader.osm.ElementSaver 74210002.osm.pbf: ignoring unspecified/unsupported restriction http://www.openstreetmap.org/relation/3931753 in my mkgmap.log file. No need to do anything about these at the moment, but sometime maybe. Thanks Ticker _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Hi Gerd This was just an example I got with this branch and options --no-route --net. It looked like all the restriction relations cause the error, but I chose a bad example that has other things wrong with it. A previous run with --route gave: WARN: uk.me.parabola.mkgmap.reader.osm.RestrictionRelation 74210002.osm.pbf: Turn restriction (no_right_turn) http://www.openstreetmap.org/relation/3931753 (at http://www.openstreetmap.org/?mlat=51.113802&mlon=-1.489513&zoom=17) unknown member role '' Sorry - should have checked. A better example is WARN: uk.me.parabola.mkgmap.reader.osm.ElementSaver 74210002.osm.pbf: ignoring unspecified/unsupported restriction http://www.openstreetmap.org/relation/5143835 Ticker On Mon, 2019-10-21 at 07:52 +0000, Gerd Petermann wrote:

Hi Ticker,

hmm, the restriction is not correct as the role for way https://www.openstreetmap.org/way/153273366 is empty. Do you think that mkgmap should start guessing what the role of the way is? In this case that would be possible, but I'd still prefer that someone fixes the data in OSM.

Gerd

________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 21. Oktober 2019 09:39 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

Just doing more experimenting with mkgmap-NET-no-NOD-r4304 and options: --no-route --net

I get lots and lots of messages like:

WARN: uk.me.parabola.mkgmap.reader.osm.ElementSaver 74210002.osm.pbf: ignoring unspecified/unsupported restriction http://www.openstreetmap.org/relation/3931753

in my mkgmap.log file.

No need to do anything about these at the moment, but sometime maybe.

Thanks Ticker

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev