Hi Gerd I decided to both fix the code and put out a warning message if there are structures that switch to 3 byte pointers (zips, highways and exit facilities as well as cities) Address search now works when there are > 65535 cities. The code is a lot tidier - I've done it properly and it gets rid of lots of messy bits of in-line testing against sizes > 255 etc. Ticker On Mon, 2017-07-31 at 13:12 +0000, Gerd Petermann wrote:
Hi Tcker,
yes, good enough. I also thought about this alternative.
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Montag, 31. Juli 2017 13:09:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Find Address - No cities
Hi Gerd
I started fixing all the places where there is an assumption about the number of cities (and zips) being < 65535 and I'm only part way through my list of suspect files and have make a lot of changes so far.
I'm thinking it is not worth while because:
The overall MDR size jumps because all city entries change to 3 bytes. I get a smaller GMAPSUPP by increasing the splitting so that it stays in the 2 byte limit.
I might miss some subtle coding of the header flags and/or the input/output.
It might not work anyway after I've made all the changes.
The changes are a bit messy because I've just hacked each problem area with a fix. Really there should be some utility functions, enhancements to the readers/writers and pointerSize variables in the relevant classes.
I propose just to signal an error in individual maps when the exceed the limit
What do you think?
Regards Ticker
On Sun, 2017-07-30 at 10:16 +0100, Ticker Berkin wrote:
Hi Gerd
I'll just done a bit more investigation and see what seems to be the the same assumption in imgfmt/app/net/RoadDef.java around line 296 if(numCities > 255)
writer.putChar(cityIndex); else
writer.put((byte)cityIndex);
I'll fix and test tomorrow and if there are still have problems I'll assemble the problem map, style etc for you
Thanks Ticker
On Sun, 2017-07-30 at 06:33 +0000, Gerd Petermann wrote:
Hi Ticker,
hmm, I think the patch looks good and I have no idea why the search doesn't work for you. Yesterday I thought the patch handles the overall number of cities in the gmapsupp, but it is about the number of cities in a single map tile. Maybe there is another problem caused by the high number of cities. Maybe you can post a link to the tile with that high number?
Gerd
________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 29. Juli 2017 21:12:27 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Find Address - No cities
Hi Gerd
Quite possibly, or maybe there are limits in the device.
More testing reveals that although my lists of cities and streets within them appear correctly in Find>Address, it doesn't find anything.
Ticker
On Sat, 2017-07-29 at 17:22 +0000, Gerd Petermann wrote:
Hi Tcker,
thanks for the patch. I seem to remember that the number of entries in mdr5 is also used elsewhere. Will have a closer look tomorrow.
Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Samstag, 29. Juli 2017 19:14:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Find Address - No cities
Hi Gerd
It looks as if can simply write 1/2/3 bytes as have indicated in the magic field in the MDR header and I've changed it to do this. All seems OK.
Yes - I do misuse a City POI. But now, after having worked out how the MDR20 index works, I might be able to do what I want in a better way.
Attached is patch to write the correct number of bytes.
Regards Ticker
On Sat, 2017-07-29 at 16:37 +0000, Gerd Petermann wrote:
Hi Ticker,
yes, sounds plausible. The comment seems to say that we don't know how to handle so many cities. Why do you have so many cities. Do you "abuse" the corresponding POI for something?
Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <ticker@jagIT.co.uk> Gesendet: Samstag, 29. Juli 2017 18:13:20 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Find Address - No cities
Hi Gerd
I think I've found the problem:
/src/uk/me/parabola/imgfmt/app/mdr/Mdr5.java line 227
private void putLocalCityIndex(ImgFileWriter writer, int cityIndex) { if (localCitySize == 2) // 3 probably not possible in actual maps. writer.putChar((char) cityIndex); else writer.put((byte) cityIndex); }
There were more than 63335 cities in the failing map.
Ticker
On Sat, 2017-07-29 at 16:09 +0100, Ticker Berkin wrote:
Hi Gerd
I've done a bit of investigation and debugging. It seems to be building the indexes with enough cities and streets. MDR20 is the suspect index.
I'm now experimenting with mkgmap / Display
With just the suspect map built in gmapsupp:
$ ... test.ExtractFile gmapsupp.img filter[] Copying . 65024 Copying MAKEGMAP.MPS 78 Copying 74440044.RGN 4154927 Copying 74440044.TRE 18772 Copying 74440044.LBL 2174485 Copying 74440044.NET 1626350 Copying 74440044.NOD 3566829 Copying 00007444.MDR 2979295 Copying 00007444.SRT 912
$ ... test.display.MdrSummary 00007444.MDR initial values 7 2 e MDR 1 NR=1 (000001) RS=4 magic=0x0 MDR 4 NR=101 (000065) RS=3 magic=0x0 MDR 5 NR=54456 (00d4b8) RS=9 magic=0x152 MDR 6 NR=4039 (000fc7) RS=3 magic=0x1 MDR 7 NR=17674 (00450a) RS=6 magic=0x66 MDR 9 NR=11 (00000b) RS=4 magic=0x0 MDR 10 DataSize=427616 (00068660) magic=0x0 MDR 11 NR=106904(01a198) RS=10 magic=0x95 MDR 12 NR=11 (00000b) RS=7 magic=0x40a MDR 17 DataSize=111036 (0001b1bc) magic=0x0 MDR 18 NR=102 (000066) RS=5 magic=0x6 MDR 19 NR=106904(01a198) RS=3 magic=0x2 MDR 20 NR=20389 (004fa5) RS=6 magic=0xe MDR 22 NR=18117 (0046c5) RS=6 magic=0xc000e MDR 24 NR=2 (000002) RS=6 magic=0x0 MDR 25 NR=70016 (011180) RS=3 magic=0x0 MDR 29 NR=2 (000002) RS=8 magic=0x26
seems fine but
$ ... test.display.MdrCheck 00007444.MDR ---------- 00007444.MDR -------------------- EXTRA remaining {} # reading 74440044 mdr5 check Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 308736, Size: 70018 at java.util.ArrayList.rangeCheck(ArrayList.java:653) at java.util.ArrayList.get(ArrayList.java:429) at test.display.check.MapDetails.getCity(MapDetails.java:188) at test.display.MdrCheck.check5(MdrCheck.java:282) at test.display.MdrCheck.print(MdrCheck.java:87) at test.display.CommonDisplay.display(CommonDisplay.java:171) at test.display.CommonDisplay.display(CommonDisplay.java:196) at test.check.CommonCheck.runMain(CommonCheck.java:145) at test.display.MdrCheck.main(MdrCheck.java:1789) + exit
On another map from the split this runs OK, producing masses of output:
$ ... test.display.MdrCheck 00007429.MDR ---------- 00007429.MDR -------------------- EXTRA remaining {} # reading 74290047 ERROR: map 1: city 1: no name; index 1 mdr5 check 1 map1; ABBEY WOOD mapCity=2 reg=0 (LONDON) country=0 (UNITED KINGDOM^]GBR) ind20=0 rep=false 2 map1; ABRIDGE mapCity=3 reg=0 (ESSEX) country=0 (UNITED KINGDOM^]GBR) ind20=0 rep=false ... 87000 more lines ...
Do you think mkgmap:Display is finding a real problem and I should pursue this line, or could it be unreliable and this exception is due to it rather than the format of the index?
Thanks Ticker
On Wed, 2017-07-26 at 23:42 -0700, Gerd Petermann wrote: > Ticker Berkin wrote > > With a particular split of 'british-isles', one map > > from > > the > > split > > causes the list of cities in the Find>Address function > > to > > be > > empty. > > Maybe you have the same problem here which I described > for > road > name search when road names contain special characters. > In > your > case > this would mean special characters in the citiy names. > I suggest to add debug code in > uk.me.parabola.imgfmt.app.mdr.Mdr5.writeSectData(ImgFileW > ri > te > r > writer) > to print all city names which are written to the global > index. > > Gerd > > > > -- > View this message in context: > http://gis.19327.n8.nabble.com/Find-Add > ress-No-cities-tp5899775p5899880.html > Sent from the Mkgmap Development mailing list archive at > Nabble.com. > _______________________________________________ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev