Mark Burton wrote:

Hi Maning,

...

Several problems with Search by Address in Garmin Mobile XT reported by some of my users.

This is a known issue. Until the global index stuff is implemented I don't think it will be fixed.

Detectives wanted! Some or maybe all of the global index is in the .MDR file. Information on the this file is starting to appear on: http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/MDR_Subfile_Format There's a global sorted city-list appearing in the MDR5 and MDR6 sections. Great chunks of tables containing short bits of plaintext strings can be seen in MDR30/31 and MDR32/33, but as yet no tables have been found which glues those bits together to form postcodes, addresses or city-names. Contributors familiar with text-compression systems would be very welcome to look over it. You can rip an .MDR file out of an official Garmin .IMG (it isn't encrypted as such). More eyes on the job will increase the chance that someone gets lucky and spots crucial clues into how it all works. Join in please! Steve

Hello, Being quite new here I would like to understand what is missing. My GPS is an etrex Vista. I generated a map of Michigan from OSM using mkgmap. If I go to Find > Addresses, I am able to search for adresses. I first search the street name (for example Jefferson Ave), then I go to street number and just hit OK. From that point I have a list of streets with cities that shows up. The things I have noticed: * If some ways are named similarly (Jeffersin Ave and Jefferson Blvd), all streets will show up if you select the first one. I f you select any other, the list will shoe "NONE FOUND". * Some streets do not show up in the correct city (maybe this is due to some bad OSM data) N. On Fri, Aug 28, 2009 at 1:08 PM, Felix Hartmann < extremecarver@googlemail.com> wrote:

[...]

Address search is really the last "big" thing we're missing.

On Fri, Aug 28, 2009 at 03:45:28PM +0100, Steve Hosgood wrote:

Detectives wanted!

Some or maybe all of the global index is in the .MDR file. Information on the this file is starting to appear on:

http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/MDR_Subfile_Format

There's a global sorted city-list appearing in the MDR5 and MDR6 sections.

Great chunks of tables containing short bits of plaintext strings can be seen in MDR30/31 and MDR32/33, but as yet no tables have been found which glues those bits together to form postcodes, addresses or city-names. Contributors familiar with text-compression systems would be very welcome to look over it.

You can rip an .MDR file out of an official Garmin .IMG (it isn't encrypted as such).

More eyes on the job will increase the chance that someone gets lucky and spots crucial clues into how it all works. Join in please!

Is there any way of creating an MDR file ourselves? I mean probably with commercial software? It might be interesting to have an MDR with a single adresse, 2 adresses same city, 2 adresses samt street+city etc ... Guessing by difference is more efficient than to wade through half a gigabyte of data in an unknown format. Flo -- Florian Lohoff flo@rfc822.org "Es ist ein grobes Missverständnis und eine Fehlwahrnehmung, dem Staat im Internet Zensur- und Überwachungsabsichten zu unterstellen." - - Bundesminister Dr. Wolfgang Schäuble -- 10. Juli in Berlin

Florian and Gert wrote in reply to my appeal:

On Fri, Aug 28, 2009 at 03:45:28PM +0100, Steve Hosgood wrote:

...

Detectives wanted!

Some or maybe all of the global index is in the .MDR file. All suggestions are welcome, but I don't have cGPSmapper nor a workable copy of MapSource.

So if you guys can try out your ideas and post observations here, or add to the information on the wiki it will all be very welcome. There is some information embedded in Steve Ratcliffe's well-known (and ancient) .java program that partly decodes .mdr files, not all of this is in the wiki yet. Specifically, Ratcliffe seems to have deciphered some way into the MDR10/11 blocks that allegedly contain POI lists and indexes - that stuff isn't on the wiki yet. Someone better than I at reading .java files might do it better and quicker (hint, hint!). I'm a C/C++ programmer myself. Other attack vectors would include decompiling MapSource and/or running it under a debugger to see what it does. To be sure we will expect to be playing "Mastermind" with .MDR files for quite a while I suspect, tweaking known-good Garmin versions and seeing what happens when you try and load Garmin GPS units with the tampered versions. It's all good fun. But seriously - the more the merrier. There's a *lot* of work here. Steve

Gert Münzel wrote:

MAPSOURCE: There are 2 Ways to install Mapsource without having the Original-CD to yours fingertips:

We could really use a wiki page on stuff like this. I do have an apparently-OK copy of 'mapsource' but it won't start because I don't have an "installed map". I don't/didn't know how to get one (though I have a genuine Garmin map in .IMG format with my Streetpilot of course). Gert - you do offer a couple of clues how to get past this stage in your mail. Can I install my genuine .IMG file as an "installed map"? Steve

Hi Steve, At Sun, Aug 30, 2009 at 11:29:16PM +0100, Steve Hosgood wrote:

We could really use a wiki page on stuff like this. I do have an apparently-OK copy of 'mapsource' but it won't start because I don't have an "installed map".

Yeah, that's a bit weird: I have one MapSource that does have the base map, the other doesn't. While both were installed through the trick with the training software... I guess it has something to do with the version for older OS'es and/or the "regular" version. (I'm running the former, because I'm running it under emulation software: Wine). Anyway, here's how to install a map: 1. Download mapsettoolkit from http://cypherman1.googlepages.com/ 2. start it. Now do the following: 3. Press "Install" 4. Choose your TDB file 5. For IMG file, choose the same name as your TDB file (if you have mkgmap.jar defaults, both would be 63240000). Now IIRC you press "install" and then you're done: your generated map is now in the list. You can exit mapsettoolkit and start mapsource. Best regards, Valentijn

gmaptool can readout the key which is used for your unit from the gmapsupp. Gert Münzel wrote:

Hi, Steve,

...

Can I install my genuine .IMG file as an "installed map"?

I think so but i guess this breacks copyrights law. I guess you have a CityNavigator on your streetpilot. If it's on a SD-Card you can take the gmapsupp.img and split it with the gmaptool(needs the free version of cgpsmapper) into a mapset. This can easily be installed in Mapsoure either via Mapsettoolkit or with the from gmaptool created install.bat. The biggest problem should be if you have a 25-sign-installationcode for this map or not. Probably not. There are keygens which can do the job but that breacks copyright law for shure. But may be you have a code.

For installing a simple single .img you can follow Valentins instructions. You could do it also with gmaptool (but its a little more tricky, you have to split this single map into a Mapset, sounds crazy but it works.)

A quick guide for the installtion with gmaptool you can find on opemmtbmap.org(needs the free version of cgpsmapper): Quickguide for Installtion with gmaptool <http://openmtbmap.org/tutorials/install/gmaptool-install-maps/>

Cheers

Gert

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Is it possible to implement global indexing incrementally? Looking at my mapsource, the "Find Places" has 4 tabs (city, feature, address, intersection). This is disabled when I am using mkgmap generated maps. Maybe we can focus first in creating the city search index first or address search and move on from there. [I honestly don't know how to do this, so disregard my message if it doesn't make sense]

You can rip an .MDR file out of an official Garmin .IMG (it isn't encrypted as such).

I have some mdr file from a cgpsmapper generated map. How can I "rip" them and send the output to. -- cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------

Hi On 19/09/09 12:43, Johann Gail wrote:

We have some basic knowledge of the format in the wiki. Has anyone tried to generate a mdr with the known sections and see what happens on mapsource or a gps unit.

Not as far as I know. But why don't we just start. I shall kick things off by creating the main files required. As there are a lot of different variations in the format, we should concentrate on one of the smallest known variations. It looks like you can create a file with only the sections: 1, 4, 5, 6, 7, 11, 15. I shall start off with a header size of 286. MDR 1: A reverse index from maps to the other sections. MDR 4: poi types, partially known MDR 5: Cities MDR 6: unknown, wiki claims it is a list of cities but I don't believe that MDR 7: Street names, optional as may be no streets MDR 11: POI's MDR 15: Strings MDR 1 points to a subsidiary header for each map, there are at least 8 subsections in this header, some of which I partially understand and I expect that all of them lead back into the other sections. It might also be useful to try for the on-device format as that is a smaller subset. ..Steve

On Mon, Sep 21, 2009 at 05:03:19PM +0100, Steve Hosgood wrote: [...]

NT maps (with MDR1 using a record-length of 4 bytes) seems to be listing the GMP subtile files. I hear that there are 8 bytes variants - don't know about them.

The 8byte are used in mapsource. Elrond

Steve Ratcliffe wrote:

On 21/09/09 17:03, Steve Hosgood wrote:

...

...

MDR 1: A reverse index from maps to the other sections. MDR 4: poi types, partially known MDR 5: Cities MDR 6: unknown, wiki claims it is a list of cities but I don't believe that

I wrote that bit of the wiki, and for me at least it's true. This is deduced from an NT map, with a record-length of 3 bytes. I wrote a

Yea, the big problem is that there is not just one format, I've no idea about NT, but there are multiple different header sizes and record sizes possible. Also there is a big difference between the full format as it comes with a map for mapsource, and the format when it transformed and put inside the gmapsupp.

Why do you try to go for NT format? Address search worked very well with non NT maps too. cpgsmapper also makes classic non NT maps. The only advantage of the NT format is that is smaller from what I notice on using it. On the other hand NT maps are much much slower. It is probably easier to start looking at non NT maps with address search. At least non NT maps are not needed to be transformed when sent to GPS.

In fact I've just looked at the on device version of MDR 6 and the records are three bytes and there does appear to be a map index as the first byte, which is entirely different from the mapsource version.

Its all useful, but you need to be clear on what exactly you are documenting. I'm planning to do the full version first, but if that doesn't pan out I may try for the on device version. Clearly we want both eventually.

...

NT maps (with MDR1 using a record-length of 4 bytes) seems to be listing the GMP subtile files. I hear that there are 8 bytes variants - don't know about them.

OK 4 byte records in MDR1 is the on device format, so I guess that confirms that you are working from a map on a GPS device?

...

It sounds like a good idea at least to try generating MDR files though - go for it!

So far I have a test mdr with a country, region and a city, but no luck getting it to work so far though.

..Steve

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

On Mon, Sep 21, 2009 at 07:26:39PM +0100, Steve Ratcliffe wrote: [...]

I've no idea what the mdr looks like in NT and of course if it is vastly different then it will not help the current effort.

MDR-for-NT and MDR-for-nonNT look quite close, it seems. MDR-for-NT has just some things more than the MDR-for-nonNT. But you're right: The wiki should document more closely, what parts were found on what kind of map. ( ) non-NT, for mapsource ( ) non-NT, for device ( ) NT, for mapsource ( ) NT, for device This should really help us to find out, what we need. Steve, why didn't you start on the for-device one first, it seems to have lots fewwer things needed, so we should arrive at "might work" or "might improve *something*" close? Elrond

it stands for "New Technology", a format that is foremost used by Garmin "City Navigator / CN " maps and I think some other "Road" maps. Not compatible with very old (say older 3-4 years) Garmin GPS. Nakor wrote:

Hello,

Excuse my ignorance here but what does NT stand for?

Thanks,

N. ------------------------------------------------------------------------

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Excellent progress! I mentioned before I don't get much of this hexa engineering. But I have several mdr files for anyone to look into. I anyone interested? Where do I dump them? On Tue, Sep 22, 2009 at 4:52 PM, Steve Hosgood <steve@tallyho.bc.nu> wrote:

Felix Hartmann wrote:

Steve Ratcliffe wrote:

On 21/09/09 17:03, Steve Hosgood wrote:

MDR 1: A reverse index from maps to the other sections. MDR 4: poi types, partially known MDR 5: Cities MDR 6: unknown, wiki claims it is a list of cities but I don't believe that

I wrote that bit of the wiki, and for me at least it's true. This is deduced from an NT map, with a record-length of 3 bytes. I wrote a

Yea, the big problem is that there is not just one format, I've no idea about NT, but there are multiple different header sizes and record sizes possible. Also there is a big difference between the full format as it comes with a map for mapsource, and the format when it transformed and put inside the gmapsupp.

I didn't know that, but it's all in need of documentation eventually. You're right though Steve - the wiki needs the sections talking about individual MDRs to specify more clearly what the record-size is, and maybe the other 4-byte value from the header too - the one tentatively marked "flags" or "?".

Why do you try to go for NT format?

Because that's all I've got. At the time I started, I didn't realise there was anything unusual about the map I have (it's the one that came with my Streetpilot).

As for NT maps being slower - well it seems that they're using compression on the postcode, street-name and city lists. I can imagine that there's a speed penalty.

Steve.

_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

-- cheers, maning ------------------------------------------------------ "Freedom is still the most radical idea of all" -N.Branden wiki: http://esambale.wikispaces.com/ blog: http://epsg4253.wordpress.com/ ------------------------------------------------------