Hi Ticker, okay, please let me also know the output of command java -version Mine is: e:\ld>java -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode) Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 13:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD Hi Gerd The last test was with my style. The routing island check was faster: r4317 r4309 ms 74210002: 593 5252 74310001: 17696 45755 The style I'm testing with does add a lot of footways (around car parks) and has the highway/barrier point > highway splitting. I've got to be out for the rest of the day now, but I can send you the data later tonight or tomorrow morning. Ticker On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:
Hi Ticker,
if I got that right the new version is slower for you. I see two possible reasons: - bottleneck in heap memory when processing multiple files at the same time (max-jobs) - your style adds a lot more roads than the default style. For my tests I've already used a modified default style with the highway=traffic_signals | highway=crossing {set mkgmap:road-speed=1} rule added to points.
Maybe you can analyse this on your side? Or you can send me your complete test data (the two input files created by splitter and your style files) to gptermann_muenchen <at> hotmail.com?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Mittwoch, 23. Oktober 2019 12:49 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Ticker,
so you get nearly no improvement. Strange. What do you see with default style?
Regarding the messages from StyledConverter: That's what I am working on. Question is wheter we should add those obvious routing islands to the road network and let island removal take care of it or if StyledConverter should also calculate the island length given by --x -check-routing-island-len before removing it.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Mittwoch, 23. Oktober 2019 12:38 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Gerd
with mkgmap-NET-no-NOD-r4317:
SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: search for routing islands found 293 islands in 65 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island removal took 1953 ms EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: search for routing islands found 2820 islands in 62 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island removal took 17696 ms Total time taken: 1 minute 20 seconds
13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap-NET -no -NOD-r4304)
It still give lots (691) of these messages: SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter 74210002.osm.pbf: check: road without connection is not written to NOD (OSM id 40586008) 51.280882,-1.096959
Ticker
On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:
Hi Ticker,
fixed, see http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=431 7
Working on some more code improvements now, so look for the latest binary for the branch at http://www.mkgmap.org.uk/download/mkgmap.html
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen@hotmail.com> Gesendet: Dienstag, 22. Oktober 2019 20:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Ticker,
okay, I can reproduce the poor performance in this area. My timings: SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 5281 ms Total time taken: 35 seconds
Most of the additional time is used to remove the nodes from NOD. This can be improved...
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 19:24 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Gerd
split.list contains: 74210001: 2362592,-73088 to 2376288,-50304
Note feed in a lot of points (postcodes) into the splitter, but there is nothing in the default style that will process them.
splitter command line is: java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames -file=../cities15000.zip --mapid=74210001 --split-file=split.list ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml
splitter.log 2>&1
# significant lines in ../mkgmap.opt max-jobs gmapsupp code-page=1252 index bounds=../bounds.zip location-autofill=is_in,nearest no-housenumbers name-tag-list=name:en,int_name,name,place_name,loc_name family-name=osm series-name=OpenStreetMap area-name=berkin x-mapset-name=OSM ticker route drive-on=detect,left preserve-element-order add-pois-to-areas generate-sea=multipolygon,extend-sea-sectors,close-gaps=350 link-pois-to-ways process-destination process-exits remove-ovm-work-files poi-address verbose order-by-decreasing-area add-boundary-nodes-at-admin-boundaries=0
Command line is:
java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x -check -routing-island-len=700 --family-id=7421 --family-name=Hants -c template.args ../mytyp.txt
I'm going to be out for the next few hours, but can do the log stuff later if you need it.
Ticker
On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:
Hi Ticker,
please let me know the mkgmap options and the area for 74210001. Maybe try with commented the log statements (except the one for the timing )
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 18:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Gerd
These are with default style.
mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700
SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 38 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 689 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 554 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 8279 ms
Total time taken: 1 minute 12 seconds
gmapsupp.img 15126528 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1347117 74210001.NOD 2384384 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 592062 74210002.NOD 1124864 00007421.SRT 912
With r4295:
Total time taken: 1 minute 3 seconds
gmapsupp.img 15943680 bytes Block size: 2048 . 23040 MAKEGMAP.MPS 138 00007421.MDR 728896 000MYTYP.TYP 1580 74210001.RGN 5870951 74210001.TRE 45927 74210001.LBL 624175 74210001.NET 1354678 74210001.NOD 2975539 74210002.RGN 2120704 74210002.TRE 17330 74210002.LBL 229542 74210002.NET 593443 74210002.NOD 1343157 00007421.SRT 912
Ticker
On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:
Hi Ticker,
strange, in my tests the test for routing islands performed in < 50 ms. So, there seems to be a special case. Also, I did not see that big effect on img siz. Please can you test with default style for comparison?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 22. Oktober 2019 17:51 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
Hi Gerd
Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with option --x-check-routing-island-len=700
Some build statistics:
SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: check for routing islands found 293 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf: routing island check took 5252 ms SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: check for routing islands found 2820 islands SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf: routing island check took 45755 ms
Total time taken: 1 minute 53 seconds gmapsupp.img 13719552 bytes block size: 2048 . 20992 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802657 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1536856 74210001.NOD 2556416 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 718899 74210002.NOD 1241600 00007421.SRT 912
For comparison, testing with r4295 (I get similar figures for trunk+):
Total time taken: 1 minute 8 seconds gmapsupp.img 14710784 bytes Block size: 2048 . 22016 MAKEGMAP.MPS 138 00007421.MDR 1121108 000MYTYP.TYP 1580 74210001.RGN 3802683 74210001.TRE 17874 74210001.LBL 758810 74210001.NET 1562592 74210001.NOD 3285327 74210002.RGN 1645275 74210002.TRE 8820 74210002.LBL 268782 74210002.NET 722291 74210002.NOD 1477491 00007421.SRT 912
So, for my test case, with my style, which probably creates quite a few more isolated networks than the default style:
It adds quite a lot of processing time (increased by 66%)
Reduces the NOD size significantly (22%) and full gmapsupp.img by 6%.
It fixes a lot of "Route calculation errors" where the nearest road to the start/end point is an isolated network. However the behaviour of this changes in a way not apparent to the map user if what should have been an isolated network overlaps tiles.
I vote for keeping something like this provided the routing -island detection only happens of the length option is specified (needs a better name than --x-check-routing-island-len)
The unconnected road handling should be moved out of StyledConverter and handled as part of the length controlled general logic.
Ticker
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev