Hi Gerd A bit more: Are the boundary conditions for tab1Width (ie the space for the shifted minCode values) correct? eg if depth is 16, there is no minCode for this level, so the maximum minCode appears at level 15, the range being 1..2^15-1 (I think) I see a couple of problems with unicode: 1/ To fit the char into prefixTable, must be either 1 byte in utf8 representation or <= 8 bits in absolute value (or v0 is made 16 bits instead). It could be that lookupBits is reduced until this isn't a problem. What did example unicode mdrs have? 2/ If >256 chars not in prefix table, the offset per level in tab1 needs to be bigger than a byte. I'm still surprised that it didn't appear to work when all the chars fit into the prefix table. Maybe the top nibble of lookupBits is a flag saying that tab1 is present, although the tab1length would do that equally well. The 8 after 1stLevWithStruct and rowsIn1stTable is still troubling, as I'd have expected it to be associated with the prefixTable Ticker On Sun, 2022-01-16 at 12:37 +0000, Ticker Berkin wrote:
Hi Gerd
I'm just starting to look at this, but won't be able to do much today, but some initial findings:
If just making gmapi from .img, unless --code-page is given it doesn't do compression, however the Mdr generated with the expected charset.
MdrCheck and MdrDisplay, including string dump, look good
Ticker
On Sat, 2022-01-15 at 15:59 +0000, Gerd Petermann wrote:
Hi all,
during the last weeks Ticker and I decoded Garmin's String compression of the global index for the PC (the *_mdr.img) . In the mdr2 branch I've coded support for that in mkgmap and I think its ready for a first beta test.
It works with codepages 1250 ..1258 or 65001. I've no idea how the 2-byte pages work, but Garmin seems to support that, too. Maybe I'll work on this later.
The branch version produces around 200 lines of diagnostic output to show what happens when compression is enabled. This will be changed to debug level later.
The encoding is enabled by default, special option --x-no-mdr15- compression can be used to turn it off.
Note that the size of the *_mdr.img file will be smaller (which is esp. intersting for those who get close to the 2G limit) Note also that a zipped *_mdr.img can and will be larger since zip cannot remove as much redundancy. Note also that this branch includes the changes from the faster-mp branch and introduces a few changes which are relavant for --lower-case maps.
How to test? Install the map in MapSource and check if you see any unexpected characters in the result lists or maybe even crashes while searching. If so, please try the same input files with the --x-no-mdr15- compression to verify if this is caused by compression or maybe other changes.
Please don't yet publish the maps, but let me know if you see anything unexpected. Some changes reg. lower-case in this branch are still experimental.
The binary is here: https://www.mkgmap.org.uk/download/mkgmap-mdr2-r4865.zip
If you see errors which are caused by compression, please attach the diagnostic output.
@Ticker: Please review the source.
Gerd _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev