Missing cities in mapsource search entry field
Hi Steve, I don't really use the search facility in mapsource much so I can't say if it's always been like this (since the MDX stuff arrived) or whether it's due to the latest changes but I notice that when searching for cities in the UK map (no funky accents in the names to confuse the issue) that great swathes of names are not recognised in the name entry box (mapsource just goes 'bong' and ignores your input) even though the names are in the map. For example, if I type 'm' it shows a list of names and the first entry is 'Mod Abbey Wood' and the next entry is 'Mabe' which can't be right if they are meant to be in alphabetic order. If you then type 'o' it shows a list of names with the 'mo' prefix but, interestingly 'mod abbey wood' isn't among them. So that's weird. If you type 'a' after the 'm', it goes bong and you don't get any names. It's as if the 'mod abbey wood' is in the wrong position in the list? If I type 'ba' and then find, it pops up a "mapsource app error dialog". Mark
Hi Mark,
For example, if I type 'm' it shows a list of names and the first entry is 'Mod Abbey Wood' and the next entry is 'Mabe' which can't be right if they are meant to be in alphabetic order. If you then type 'o' it shows a list of names with the 'mo' prefix but, interestingly 'mod abbey wood' isn't among them. So that's weird. If you type 'a' after the 'm', it goes bong and you don't get any names. It's as if the 'mod abbey wood' is in the wrong position in the list?
I don't get any of this wierdness and I just tried again after the recent check-in-fest. Thats the sort of thing that happens however if your index files are not generated from the exact same .img files in the map, like your first Baltic map you posted. ..Steve
Hi Steve,
For example, if I type 'm' it shows a list of names and the first entry is 'Mod Abbey Wood' and the next entry is 'Mabe' which can't be right if they are meant to be in alphabetic order. If you then type 'o' it shows a list of names with the 'mo' prefix but, interestingly 'mod abbey wood' isn't among them. So that's weird. If you type 'a' after the 'm', it goes bong and you don't get any names. It's as if the 'mod abbey wood' is in the wrong position in the list?
I don't get any of this wierdness and I just tried again after the recent check-in-fest.
Thats the sort of thing that happens however if your index files are not generated from the exact same .img files in the map, like your first Baltic map you posted.
Yes, at some point in the Baltic map work, I turned off index generation and so it was using a previous index. Fair enough, but with the UK map I am sure the index is being generated from the right files. This is the command line: java -Xmx2000m -Dlog.config=/home/markb/OSM/logging.properties -ea -jar /home/markb/OSM/mkgmap.jar --adjust-turn-headings --country-abbr=GBR --country-name=GBR --code-page=1250 --check-roundabouts --check-roundabout-flares --description=A fine map --draw-priority=28 --family-id=909 --family-name=Burto Maps --gmapsupp --generate-sea=polygons,no-sea-sectors,close-gaps=2000 --ignore-maxspeeds --index --latin1 --link-pois-to-ways --make-all-cycleways --max-jobs=1 --nsis --product-id=6324 --region-abbr=UK --remove-bogus-nodes --report-dead-ends=2 --region-name=UK --route --remove-short-arcs --series-name=OSM map --style-file=/home/markb/OSM/mkgmap-burto-style --tdbfile -c template.args /home/markb/OSM/M000038d.TYP and template.args is: mapname: 63240001 description: IE-Cork input-file: 63240001.osm.gz mapname: 63240002 description: GB-Plymouth input-file: 63240002.osm.gz mapname: 63240003 description: GB-Cardiff input-file: 63240003.osm.gz mapname: 63240004 description: GB-Southampton input-file: 63240004.osm.gz mapname: 63240005 description: GB-Bristol input-file: 63240005.osm.gz mapname: 63240006 description: GB-Wolverhampton input-file: 63240006.osm.gz mapname: 63240007 description: GB-Birmingham input-file: 63240007.osm.gz mapname: 63240008 description: GB-Portsmouth input-file: 63240008.osm.gz mapname: 63240009 description: GB-Reading input-file: 63240009.osm.gz mapname: 63240010 description: GB-Slough input-file: 63240010.osm.gz mapname: 63240011 description: GB-London input-file: 63240011.osm.gz mapname: 63240012 description: GB-Southend-on-Sea input-file: 63240012.osm.gz mapname: 63240013 description: GB-Luton input-file: 63240013.osm.gz mapname: 63240014 description: GB-Leicester input-file: 63240014.osm.gz mapname: 63240015 description: GB-Cambridge input-file: 63240015.osm.gz mapname: 63240016 description: GB-Norwich input-file: 63240016.osm.gz mapname: 63240017 description: IE-Dublin input-file: 63240017.osm.gz mapname: 63240018 description: GB-Stoke-on-Trent input-file: 63240018.osm.gz mapname: 63240019 description: GB-Liverpool input-file: 63240019.osm.gz mapname: 63240020 description: GB-Preston input-file: 63240020.osm.gz mapname: 63240021 description: GB-Sheffield input-file: 63240021.osm.gz mapname: 63240022 description: GB-Leeds input-file: 63240022.osm.gz mapname: 63240023 description: GB-Nottingham input-file: 63240023.osm.gz mapname: 63240024 description: GB-Kingston upon Hull input-file: 63240024.osm.gz mapname: 63240025 description: GB-Glasgow input-file: 63240025.osm.gz mapname: 63240026 description: GB-Newcastle upon Tyne input-file: 63240026.osm.gz mapname: 63240027 description: GB-Fort William input-file: 63240027.osm.gz mapname: 63240028 description: GB-Aberdeen input-file: 63240028.osm.gz ---------- Its generating the .img files and the index all in one run - is that OK? Cheers, Mark
Hi Mark
Its generating the .img files and the index all in one run - is that OK?
Yes that is OK. It generates the .img first and then goes back and does the index, so there shouldn't be any difference either way. I didn't regenerate the UK files, I just built the index from existing ones (which have sea, so they are not that old). I am now doing all from scratch. I don't use all those options though so perhaps one of those is to blame somehow. ..Steve
This is the command line:
java -Xmx2000m -Dlog.config=/home/markb/OSM/logging.properties -ea -jar /home/markb/OSM/mkgmap.jar --adjust-turn-headings --country-abbr=GBR --country-name=GBR --code-page=1250 --check-roundabouts --check-roundabout-flares --description=A fine map --draw-priority=28 --family-id=909 --family-name=Burto Maps --gmapsupp --generate-sea=polygons,no-sea-sectors,close-gaps=2000 --ignore-maxspeeds --index --latin1 --link-pois-to-ways --make-all-cycleways --max-jobs=1 --nsis --product-id=6324 --region-abbr=UK --remove-bogus-nodes --report-dead-ends=2 --region-name=UK --route --remove-short-arcs --series-name=OSM map --style-file=/home/markb/OSM/mkgmap-burto-style --tdbfile -c template.args /home/markb/OSM/M000038d.TYP
Try to build without --code-page=1250. I bet it will work then. In my eyes the problems are currently all related to using --latin1 or another codepage. Without Codepage everything seems to work (inside Mapsource, not on GPS of course).
Hi Felix,
Try to build without --code-page=1250. I bet it will work then. In my eyes the problems are currently all related to using --latin1 or another codepage. Without Codepage everything seems to work (inside Mapsource, not on GPS of course).
Are you saying don't use any code-page option and don't use latin1? i.e. let them be default values. Also, is that just for making the index or for generating the tiles as well? Thanks, Mark
On 28.01.2010 16:21, Mark Burton wrote:
Hi Felix,
Try to build without --code-page=1250. I bet it will work then. In my eyes the problems are currently all related to using --latin1 or another codepage. Without Codepage everything seems to work (inside Mapsource, not on GPS of course).
Are you saying don't use any code-page option and don't use latin1? i.e. let them be default values.
Also, is that just for making the index or for generating the tiles as well?
Did not try it separately. If done in one run, no codepage, no latin1, then index works fine. In my eyes something goes wrong once one leaves the default (plain ASCI) value. I'll give it a go with two runs to see where we get, but I fear it wont work.
Thanks,
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi Felix,
Did not try it separately. If done in one run, no codepage, no latin1, then index works fine. In my eyes something goes wrong once one leaves the default (plain ASCI) value.
Yes, you're right, I have made both the UK map and the Baltic map without those options and the index is much better. It sometimes still pops up a mapsource error dialog when you type an incomplete name and press the find button (but, weirdly, doesn't always do that). So, great, the search index works reliably for cities now. The downside, of course, is that all of the accented characters have been replaced with their "dumbed down" versions. Thanks, Mark
On 28.01.2010 16:56, Mark Burton wrote:
Hi Felix,
Did not try it separately. If done in one run, no codepage, no latin1, then index works fine. In my eyes something goes wrong once one leaves the default (plain ASCI) value.
Yes, you're right, I have made both the UK map and the Baltic map without those options and the index is much better. It sometimes still pops up a mapsource error dialog when you type an incomplete name and press the find button (but, weirdly, doesn't always do that).
So, great, the search index works reliably for cities now. The downside, of course, is that all of the accented characters have been replaced with their "dumbed down" versions.
Well the more I played with it, the more it becomes apparent that the whole process of creating addresses and the index is flawed. Very often a street will be inside a city that is spelled differently. E.g. many streets in "Mödling, Austria" will be found in "Moedling, Austria", but "Moedling ist not available in the autocomplete. This happens whether we use a codepage or not, some other streets are correctly registered in Modling/Mödling (depending on whether we use a codepage or not): This address below cannot be found when entering a city - as inside the city autocomplete list only Modling exists, but the street somehow got registered in not existing city Moedling....
Thanks,
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Hi, Taking Felix openmtbmap de(29.01.2010) and searching e.g. for the street "Am Hof" in Mapsource, i got some results, two of them in Köln.. First found as "Am Hof, Koeln, DEU" Second as "Am Hof, Koeln, Koln, DEU. None of them could be found by entering Koln or Koeln in the citys field of adresssearch because KOELN is not in the city index and Koln in the address is only the county part in one of the streets. So in both cases addresssearch included city information can't work. The city itself only could be found ( map is compiled without -latin1 as Felix told) only by entering "Koln". On the GPS(60csx) entering "Am Hof" won't find the street directly. I have to scroll down the whole list, which itself is really messed up. Entering the city doesn't work at all. Where does that "KOELN" is coming from in the example?. Why is there no COUNTY information for one street.? _*Where does that "KOELN" is coming from?*_ I have downloaded the osm-data which covers these particular Also, although i'm not a skilled java coder, i had a look in the locator stuff(locator.java., locatorconfig.java, locatorconfig.xml) I played around with a the data by setting the location-autofill parameter 1: location-autofill=0 One founded like "Am Hof, Koeln, DEU" Second as "Am Hof, Junkersdorf, Koln, DEU In this case simply the content of the name tag from the citypoi "Junkersdorf", which is close to the street is taken for the city-part of the adress. 2: location-autofill=2 gaves me the same result as in Felix map. One founded like "Am Hof, Koeln, DEU" Second as "Am Hof, Koeln, Koln, DEU 3: location-autofill=1 gaves me also the same result as in Felix map. location-autofill=2 or location-autofill=1 is analyzing the is_in tag of that citypoi "Junkersdorf" where "Köln" is found. This leads to the citypoi "Köln". And there the "openGeoDB:sort_name" seems to be taken in absence of tag addr:city as value for the city-part of the streets address. In this case the value of openGeoDB:sort_name is "KOELN". And so the cityname in cityindex is "Koln" (without -latin1) but the city in the street address is "KOELN" and will never match. _*Why is there no COUNTY information for one street*?_ The easiest would be if there is always correct adress information at the object . So there would be no imperative trying to gather necessary information for the address search on another way like with the location-autofill. But thats not the situation. The COUNTY part is coming out of is_in tag. cityPoi "Köln" <tag k='is_in' v='Regierungsbezirk Köln,Nordrhein-Westfalen,Bundesrepublik Deutschland,Europe' /> Locatorconfig.xml regionOffset="3" code snippet out of LOCATOR.JAVA: if(p.getIsIn() != null) { String[] cityList = p.getIsIn().split(","); //System.out.println(p.getIsIn()); // is_in content is not well defined so we try our best to get some info out of it // Format 1 popular in Germany: "County,State,Country,Continent" if(cityList.length > 1 && isContinent(cityList[cityList.length-1])) // Is last a continent ? { // The one before contient should be the country p.setCountry(fixCountryString(cityList[cityList.length-2].trim())); // aks the config which info to use for region info int offset = locConfig.getRegionOffset(p.getCountry()) + 1; if(cityList.length > offset) p.setRegion(cityList[cityList.length-(offset+1)].trim()); As far as i understood and tested, i never will get here any usable result for county if i have only 4 field parts in is_in tag and regionOffset="3" in Locatorconfig.xml. Of cource you actually really not know how many field parts you will found in is_in tag, sometimes there are 5 parts and the code will work in a way as the example partially shows(citypoi "Junkersdorf has 5 field parts in his is_in tag). Nevertheless imo the code doesn't work as i would expect it from the given example in code // Format 1 popular in Germany: "County,State,Country,Continent". I think either you reduce regionOffset inLocatorconfig.xml or change the code. But maybe, because i have a really small understanding of JAVA i'm totally wrong. cheers Gert
On 28.01.2010 16:21, Mark Burton wrote:
Hi Felix,
Try to build without --code-page=1250. I bet it will work then. In my eyes the problems are currently all related to using --latin1 or another codepage. Without Codepage everything seems to work (inside Mapsource, not on GPS of course).
Are you saying don't use any code-page option and don't use latin1? i.e. let them be default values.
Also, is that just for making the index or for generating the tiles as well?
I just tried out with only the index having no codepage, but the maps being --latin1. This lead to problems like city madau showed up in the dropdownlist, but auto complete would not accept it (there is no non ACSI character in the city name of "Madau". So currently you're not allowed to give a codepage for city-search/address-search to work correctly (remaining the bug that it does not work correctly on GPS).
Thanks,
Mark _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
recreated it one more time, then "Madau" became searchable. Not able to enter any street starting with "Ma" this time however. Maybe there is some "leak" when using a codepage and entries get dropped? If no codepage is given at all it works. BTW there is another miserable failure. I have both AT Österreich, and AUT Austria in my map. Some addresses are in "Österreich", some are in "Austria". When specifying a country via commandline, it should be made sure that the country gets associated to any address/city inside the map. Maybe if this gets fixed, then the problem with sending to GPS and impossible to search for address because being asked for region instead of country would be solved too??? Also very problematic are cities that are regions by themselves. Like Berlin, or Vienna. Common sense would be to search for a street inside Vienna as city, but street can only be found by searching for region Vienna and entering for city one of the 23 district names (people give the postal code with their address so one could identify the district, but that means knowing which postal code any district has). However as long as it does more or less not work at all on the GPS, such problems can be solved later (if solvable, as addresses are a mess inside OSM).
Mark Burton wrote:
Hi Felix,
Try to build without --code-page=1250. I bet it will work then. In my eyes the problems are currently all related to using --latin1 or another codepage. Without Codepage everything seems to work (inside Mapsource, not on GPS of course).
Are you saying don't use any code-page option and don't use latin1? i.e. let them be default values.
I've just noticed something that might be relevant. When 'mapsource' uploads a set of searchable mkgmap to your GPS unit, it seems that it doesn't include a ".SRT" file. Now, I've only just become aware of the .SRT file - it typically has the same basename as the .MDR file, but all it contains is a 256-entry lookup table which categorises the character set. Garmin (being American) quite probably assume that if there is no such file, then assume the character set is plain ASCII! If this is the case then I can well believe that all the European accented characters will fail to sort correctly. The lookup table in the .SRT file for instance has an entry for code-page 1250's character 0xC5 (which is 'Aring' - Å) noting that is is to be sorted equal to plain 'A'. Character 0xC6 on the other hand (Icelandic Æ) has an entry that get it sorted below 'A' but above 'B'. As a note primarily for Steve Ratcliffe(*): it might be an idea to include a boilerplate .SRT file in the .IMG files generated by mkgmap. If this is done it is possible that the .SRT file will get loaded to the GPS units and help clear up this sorting issue. Steve (*) And thanks to SR for making me aware of what the .SRT file was about anyway!
On Tue, Feb 02, 2010 at 09:39:56AM +0000, Steve Hosgood wrote:
As a note primarily for Steve Ratcliffe(*): it might be an idea to include a boilerplate .SRT file in the .IMG files generated by mkgmap. If this is done it is possible that the .SRT file will get loaded to the GPS units and help clear up this sorting issue.
We might want to include several .SRT files, for different collations. While you can collate A=Ä in English and German, Swedish uses the sorting XYZÅÄÖ for instance. On the other hand, for the purpose of data entry, it might be easiest to identify all "funny" letters with their A-Z cousins, so that one can search for places without having to go through extra clicks to find the "funny" letters on the virtual keyboard. If you wish to pursue this idea, the 8-bit collations implemented in MySQL could be a useful starting point. Marko
Marko Mäkelä wrote:
On Tue, Feb 02, 2010 at 09:39:56AM +0000, Steve Hosgood wrote:
As a note primarily for Steve Ratcliffe(*): it might be an idea to include a boilerplate .SRT file in the .IMG files generated by mkgmap. If this is done it is possible that the .SRT file will get loaded to the GPS units and help clear up this sorting issue.
We might want to include several .SRT files, for different collations. While you can collate A=Ä in English and German, Swedish uses the sorting XYZÅÄÖ for instance.
I suspect that a given .IMG file can only contain one .SRT file, but yes - if Steve Ratcliffe agreed to do it, I guess that a user could request a given collation sequence. Then again, should it just be tied to the codepage that is already specified on the command-line to 'mkgmap'. Would a Swedish user want to use codepage 1252 like Western European users would? It seems that the codepage number to be used with a given .SRT file appears inside that .SRT file. I would guess that the actual lookup tables in the .MDR file have to be created using the sort-rules taken from the matching .SRT file if any of this is going to work. BTW - I just created a placeholder page on the Wiki to describe the .SRT file: http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/SRT_Subfile_Format Please contribute if you know anything more about that file format. Steve
Hi Steve, On Tue, Feb 02, 2010 at 02:53:45PM +0000, Steve Hosgood wrote:
Would a Swedish user want to use codepage 1252 like Western European users would?
Yes. Collation rules and character encodings are two different things. For what it is worth, MySQL implements a number of collations for Code Page 1252, which it calls "latin1". (cp1252 is a superset of ISO 8859-1 a.k.a. ISO Latin-1.) Some MySQL collations are documented at http://dev.mysql.com/doc/refman/5.1/en/charset-mysql.html For cp1252, the MySQL collations include German (DIN-1 and DIN-2), Swedish/Finnish, Danish/Norwegian, Spanish, general, and binary (according to code point value). But like I said in my previous message, we could at least initially use just one generic collation (case insensitive, ignoring any diacritical marks and accents). Best regards, Marko
participants (6)
-
Felix Hartmann -
Gert Münzel -
Mark Burton -
Marko Mäkelä -
Steve Hosgood -
Steve Ratcliffe