Re: [mkgmap-dev] Errors converting "mapnik.txt" into "mapnik.typ" (information only)
Hi Eric There is a lot here! I don't want to spend a lot of time going through it point by point here are some comments and some changes, helpful to TYPViewer users, that could be made to mapnik.txt and mkgmap. 1/ DONE: remove a comma from a STRING default line. This is because mkgmap and TYPViewer use different parsing methods for: String[#]=[lang#,]description where [] indicates optional. mkgmap spots there is no language# 2/ PATCH in progress: Change a character used to represent a pixel colour. These characters are arbitrary, but something, either the . or the 1, causes TYPViewer to write an incorrect .typ file 3/ pixel characters can be chosen by the user; it is NOT an error to use different characters per icon. TYPViewer changes them to the characters it would use if it was reading a binary .typ file. 4/ Implement the [_comments] ... [end] section in mkgmap. These comments wouldn't put it into the .typ file unlike TYPViewer, which does. It is good for copyright and version information but not actual comments. Having experimented with this I find that TYPViewer is inconsistent when reading it back and sometimes reports a different header length and doesn't pick up the comments. 5/ What I meant by "supposed Win 1252 Typ file" is that it is very easy to get TYPViewer to generate a file where the Typ header says the CodePage is 1252, but the strings are encoded as utf8 hence chars>127 will show incorrectly. I was going try list some of the ways to get it correct and some that get it wrong, but there are many variables that might have an effect on this I'm finding this a time-consuming and pointless process. 6/ BOM and coding line. These lines are there so that there is a higher chance of tools (editors, compilers...) getting the source coding correct. Without these lines, some tools will scan the complete file checking that all chars >127 are part of a valid utf-8 sequence, some will just make assumptions and possibly get it wrong. 7/ TYPViewer recognises strings in mapnik.txt as being encoded as utf8 regardless of the method used to open the file or BOM/coding line (good - see above). On forcing a change to be saved, it converts to the specified CodePage, quietly mapping chars not in this CodePage to their more generic form. 8/ mkgmap TypCompiler does check for BOM. Up until January 2020, just in the utf8 encoding, but since then in 16LE/16BE/32LE/32BE as well, also looking for "-*- coding:" near the start of the file. 9/ Change mkgmap so that the message severity for missing numbers is downgraded for: ProductCode= FID= as these will be generated from other sources 10/ as per your recent mail, TYPViewer does change the Type hex numbers in the _draworder section 11/ utf8 should be the standard for all source files. You suggest it is not needed for 99% of usage, but you are ruling out most countries of the world. There is little extra effort required to support utf8 until we meet tools like TYPViewer. 12/ "Correct procedure" - most of this is fine for you, but I'd suggest always using TYPViewer in utf8/65001 mode and never using it to generate the .typ file. Then in mkgmap, use your normal --code -page=1252 because 65001 makes the map bigger and isn't supported on many Garmin devices. 13/ Actual translations in mapnik.txt: this is an ongoing effort by anyone who wants to improve it. In the spreadsheet you've highlighted quite a few default strings - I don't see what is wrong with them; they were specified as described in the posting: http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q1/030254.html You've also highlighted quite Dutch ones - could you do a patch for these based on either the file I sent you or the latest svn source file, or, failing that, just edit it and and sent it back to me. Best wishes - hope everyone is healthy Ticker On Mon, 2020-03-30 at 21:59 +0200, eric_internet@casema.nl wrote:
Hi Ticker, Gerd,
I have spent "some" time in investigating "mapnik.txt" vs. "TypViewer". Understatement...
Things got out of hand...
Therefore: two attachments...
Best regards,
Eric (AnkEric)
Hi Ticker/Eric, I didn't try to understand the issues. Please let me know if I should commit mapnik-TYPViewer-2.patch Gerd ________________________________________ Von: mkgmap-dev <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap@jagit.co.uk> Gesendet: Dienstag, 31. März 2020 18:11 An: mkgmap development Betreff: Re: [mkgmap-dev] Errors converting "mapnik.txt" into "mapnik.typ" (information only) Hi Eric There is a lot here! I don't want to spend a lot of time going through it point by point here are some comments and some changes, helpful to TYPViewer users, that could be made to mapnik.txt and mkgmap. 1/ DONE: remove a comma from a STRING default line. This is because mkgmap and TYPViewer use different parsing methods for: String[#]=[lang#,]description where [] indicates optional. mkgmap spots there is no language# 2/ PATCH in progress: Change a character used to represent a pixel colour. These characters are arbitrary, but something, either the . or the 1, causes TYPViewer to write an incorrect .typ file 3/ pixel characters can be chosen by the user; it is NOT an error to use different characters per icon. TYPViewer changes them to the characters it would use if it was reading a binary .typ file. 4/ Implement the [_comments] ... [end] section in mkgmap. These comments wouldn't put it into the .typ file unlike TYPViewer, which does. It is good for copyright and version information but not actual comments. Having experimented with this I find that TYPViewer is inconsistent when reading it back and sometimes reports a different header length and doesn't pick up the comments. 5/ What I meant by "supposed Win 1252 Typ file" is that it is very easy to get TYPViewer to generate a file where the Typ header says the CodePage is 1252, but the strings are encoded as utf8 hence chars>127 will show incorrectly. I was going try list some of the ways to get it correct and some that get it wrong, but there are many variables that might have an effect on this I'm finding this a time-consuming and pointless process. 6/ BOM and coding line. These lines are there so that there is a higher chance of tools (editors, compilers...) getting the source coding correct. Without these lines, some tools will scan the complete file checking that all chars >127 are part of a valid utf-8 sequence, some will just make assumptions and possibly get it wrong. 7/ TYPViewer recognises strings in mapnik.txt as being encoded as utf8 regardless of the method used to open the file or BOM/coding line (good - see above). On forcing a change to be saved, it converts to the specified CodePage, quietly mapping chars not in this CodePage to their more generic form. 8/ mkgmap TypCompiler does check for BOM. Up until January 2020, just in the utf8 encoding, but since then in 16LE/16BE/32LE/32BE as well, also looking for "-*- coding:" near the start of the file. 9/ Change mkgmap so that the message severity for missing numbers is downgraded for: ProductCode= FID= as these will be generated from other sources 10/ as per your recent mail, TYPViewer does change the Type hex numbers in the _draworder section 11/ utf8 should be the standard for all source files. You suggest it is not needed for 99% of usage, but you are ruling out most countries of the world. There is little extra effort required to support utf8 until we meet tools like TYPViewer. 12/ "Correct procedure" - most of this is fine for you, but I'd suggest always using TYPViewer in utf8/65001 mode and never using it to generate the .typ file. Then in mkgmap, use your normal --code -page=1252 because 65001 makes the map bigger and isn't supported on many Garmin devices. 13/ Actual translations in mapnik.txt: this is an ongoing effort by anyone who wants to improve it. In the spreadsheet you've highlighted quite a few default strings - I don't see what is wrong with them; they were specified as described in the posting: http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q1/030254.html You've also highlighted quite Dutch ones - could you do a patch for these based on either the file I sent you or the latest svn source file, or, failing that, just edit it and and sent it back to me. Best wishes - hope everyone is healthy Ticker On Mon, 2020-03-30 at 21:59 +0200, eric_internet@casema.nl wrote:
Hi Ticker, Gerd,
I have spent "some" time in investigating "mapnik.txt" vs. "TypViewer". Understatement...
Things got out of hand...
Therefore: two attachments...
Best regards,
Eric (AnkEric)
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, This is not for me to decide. I'm just a User... IMO: the mapnik-TYPViewer-2.patch is okay. So to be committed. IMO! I will - again - verify (by TypViewer) if "mapnik.txt" is okay after committed patch. If not: I will let you know. BTW: Did I gave you the impression I did understand??? ROFL... I don't understand any of this, including my own document; -)) Eric -- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html
Hi Gerd Yes, please commit. I'm going to send another patch later for the mkgmap typ compiler and it will include other changes to mapnik.txt so Eric, you might want to hold off your testing until this is also committed. Ticker On Wed, 2020-04-01 at 01:35 -0700, AnkEric wrote:
Hi Gerd,
This is not for me to decide. I'm just a User...
IMO: the mapnik-TYPViewer-2.patch is okay. So to be committed. IMO!
I will - again - verify (by TypViewer) if "mapnik.txt" is okay after committed patch. If not: I will let you know.
BTW: Did I gave you the impression I did understand??? ROFL... I don't understand any of this, including my own document; -))
Eric
-- Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Gerd, Eric Here is another patch to help TYPViewer users - mkgmap TYP compiler skips [_comments] ... [End] sections - for TYP lines in the [_id] section like: ProductCode= FID= CodePage= ie. without a value, mkgmap now writes a WARN message to the log instead of a SEVE: MapFailedException. These values will be supplied from the main mkgmap --options anyway. - put the mapnik.txt copyright message etc in the new [_comments] secti on - a few minor layout tweaks to mapnik.txt Ticker
participants (3)
-
AnkEric -
Gerd Petermann -
Ticker Berkin