Problem uploading data to JOSM wit 20 hour old OSM data, maybe a SPLIITTER issue?
I downloaded .a mapset (michigan-latest.osm.pbf) from http://download.geofabrik.de/north-america.html yesterday. The data is about 20 hours old. I had to use SPLITTER r428 with this command: java -Xmx1024m -Xms1024m -ea -jar \MKGMAP\splitter-r428\splitter.jar --max-nodes=1400000 --mapid="12390000" --geonames-file=\MKGMAP\cities15000.zip --description=MI-greg --max-areas="255" --no-trim --status-freq="600" michigan-latest.osm.pbf Since I could not load the entire Michigan 96MB file into JOSM r9329. The resulting file, 12390006.osm.pbf, is 10.8MB and I imported it into JOSM and copied a NODES tags to an underlying POLYGON and deleted the NODE and tried to upload to the OSM server and got this error message: "Uploading failed because the server has a newer version of one of your nodes, ways, or relations. Click Synchronize entire dataset to synchronize the entire local dataset with the server" It is possible that SPLITTER r428 is not preserving the dataset versions or is setting them all to 1 when they really have different versions (eg. 2,3,4) Greg
Hi Greg, yes, splitter removes the version info because it is meant as a tool for mkgmap. You should always load current data into JOSM when you plan to edit, else you may have to fix several conflicts. Gerd ________________________________ Von: mkgmap-dev-bounces@lists.mkgmap.org.uk <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von greg crago <gregcrago@gmail.com> Gesendet: Samstag, 27. Februar 2016 16:39 An: Development list for mkgmap Betreff: [mkgmap-dev] Problem uploading data to JOSM wit 20 hour old OSM data, maybe a SPLIITTER issue? I downloaded .a mapset (michigan-latest.osm.pbf) from http://download.geofabrik.de/north-america.html yesterday. The data is about 20 hours old. North America - Geofabrik<http://download.geofabrik.de/north-america.html> download.geofabrik.de Not what you were looking for? Geofabrik is a consulting and software development firm based in Karlsruhe, Germany specializing in OpenStreetMap services. I had to use SPLITTER r428 with this command: java -Xmx1024m -Xms1024m -ea -jar \MKGMAP\splitter-r428\splitter.jar --max-nodes=1400000 --mapid="12390000" --geonames-file=\MKGMAP\cities15000.zip --description=MI-greg --max-areas="255" --no-trim --status-freq="600" michigan-latest.osm.pbf Since I could not load the entire Michigan 96MB file into JOSM r9329. The resulting file, 12390006.osm.pbf, is 10.8MB and I imported it into JOSM and copied a NODES tags to an underlying POLYGON and deleted the NODE and tried to upload to the OSM server and got this error message: "Uploading failed because the server has a newer version of one of your nodes, ways, or relations. Click Synchronize entire dataset to synchronize the entire local dataset with the server" It is possible that SPLITTER r428 is not preserving the dataset versions or is setting them all to 1 when they really have different versions (eg. 2,3,4) Greg
I thought 20 hour old data was fairly recent data. I tried loading OSM data using JOSM, but it only allows a small window of data. Downloading OSM data from http://download.geofabrik.de/north-america.html allows me to download large areas of my country and allows me to make edits as I need to (offline) and does not force me to constantly be online in order to download OSM data where I want to work. Is there any other 'splitter' type program I can use to break up the larger osm.pbf maps into smaller 10MB 'chunks' I can use with JOSM? Can we change splitter to keep the version information so I can use it with JOSM? Greg On Sat, Feb 27, 2016 at 12:26 PM, Gerd Petermann < GPetermann_muenchen@hotmail.com> wrote:
Hi Greg,
yes, splitter removes the version info because it is meant as a tool for mkgmap.
You should always load current data into JOSM when you plan to edit, else you
may have to fix several conflicts.
Gerd
------------------------------ *Von:* mkgmap-dev-bounces@lists.mkgmap.org.uk < mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von greg crago < gregcrago@gmail.com> *Gesendet:* Samstag, 27. Februar 2016 16:39 *An:* Development list for mkgmap *Betreff:* [mkgmap-dev] Problem uploading data to JOSM wit 20 hour old OSM data, maybe a SPLIITTER issue?
I downloaded .a mapset (michigan-latest.osm.pbf) from http://download.geofabrik.de/north-america.html yesterday. The data is about 20 hours old. North America - Geofabrik <http://download.geofabrik.de/north-america.html> download.geofabrik.de Not what you were looking for? Geofabrik is a consulting and software development firm based in Karlsruhe, Germany specializing in OpenStreetMap services.
I had to use SPLITTER r428 with this command:
java -Xmx1024m -Xms1024m -ea -jar \MKGMAP\splitter-r428\splitter.jar --max-nodes=1400000 --mapid="12390000" --geonames-file=\MKGMAP\cities15000.zip --description=MI-greg --max-areas="255" --no-trim --status-freq="600" michigan-latest.osm.pbf
Since I could not load the entire Michigan 96MB file into JOSM r9329. The resulting file, 12390006.osm.pbf, is 10.8MB and I imported it into JOSM and copied a NODES tags to an underlying POLYGON and deleted the NODE and tried to upload to the OSM server and got this error message:
"Uploading failed because the server has a newer version of one of your nodes, ways, or relations. Click Synchronize entire dataset to synchronize the entire local dataset with the server"
It is possible that SPLITTER r428 is not preserving the dataset versions or is setting them all to 1 when they really have different versions (eg. 2,3,4)
Greg
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I want to download sections of the osm database (USA- Michigan) and make off line changes in JOSM and upload back to OSM when I am back online. I looked at Overpass API and it looks like HUGE sections of data (32GB?) How can I download smaller sections like http://download.geofabrik.de/north-america.html is laid out? Greg On Sat, Feb 27, 2016 at 2:20 PM, Nelson A. de Oliveira <naoliv@gmail.com> wrote:
On Sat, Feb 27, 2016 at 4:12 PM, greg crago <gregcrago@gmail.com> wrote:
I thought 20 hour old data was fairly recent data.
What exactly do you want to do? Can't you take your data from overpass? _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
On Sat, Feb 27, 2016 at 5:44 PM, greg crago <gregcrago@gmail.com> wrote:
I want to download sections of the osm database (USA- Michigan) and make off line changes in JOSM and upload back to OSM when I am back online.
Will you edit the whole state? Do you really need to have all of it? Editing large areas and a lot of objects isn't usually the best thing to do: it can cause duplicate data in OSM and/or you will hit a lot of conflicts. overpass is the easiest way to get fresh data, but you need to filter a smaller region or some specific data that you want.
I have thousands of map errors I have collected on my GPS from ground observations. I just have not had time to upload them to OSM. I do not have Internet access all the time. Downloading many states and then breaking them up using a 'splitter-type' program, gives me the flexibility to edit sections of the country I want to. I thought UPLOAD conflicts only occur when objects I HAVE MODIFIED, conflict with objects in the OSM database that have been modified since I download MY VERSION. If I have large areas of 'outdated' OSM data, that I do not modify, I should not have any problems when I upload, since I am not sending that 'untouched' data back to OSM, correct? I thought this was a viable 'offline' solution. I do not know how to filter overpass to download state-by-state regions. GEOFABRIK has already done this, if SPLITTER kept the version dataset information., I could use it. Greg On Sat, Feb 27, 2016 at 4:08 PM, Nelson A. de Oliveira <naoliv@gmail.com> wrote:
On Sat, Feb 27, 2016 at 5:44 PM, greg crago <gregcrago@gmail.com> wrote:
I want to download sections of the osm database (USA- Michigan) and make off line changes in JOSM and upload back to OSM when I am back online.
Will you edit the whole state? Do you really need to have all of it?
Editing large areas and a lot of objects isn't usually the best thing to do: it can cause duplicate data in OSM and/or you will hit a lot of conflicts.
overpass is the easiest way to get fresh data, but you need to filter a smaller region or some specific data that you want. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Greg, Using JOSM and working with data that other people are also working with is a recipe for disaster. Resolving the inevitable conflicts that will arise is tedious and complicated. I have spent hours resolving conflicts made by "the other user" only to find out later that it was _my own data_ I was working against. The method you're trying to use will only work well if you're the only person modifying the data in that particular part of the world. Another possibility is to download smaller areas to work in which will keep the number of edits down. That way you won't have thousands of conflicts when you do get Internet again. And if you do, you won't lose much work. Not very satisfying I'm afraid but working with a live Internet connection is by far the best option. On Sun, Feb 28, 2016 at 9:42 PM, greg crago <gregcrago@gmail.com> wrote:
I have thousands of map errors I have collected on my GPS from ground observations. I just have not had time to upload them to OSM. I do not have Internet access all the time.
Downloading many states and then breaking them up using a 'splitter-type' program, gives me the flexibility to edit sections of the country I want to.
I thought UPLOAD conflicts only occur when objects I HAVE MODIFIED, conflict with objects in the OSM database that have been modified since I download MY VERSION. If I have large areas of 'outdated' OSM data, that I do not modify, I should not have any problems when I upload, since I am not sending that 'untouched' data back to OSM, correct?
I thought this was a viable 'offline' solution. I do not know how to filter overpass to download state-by-state regions. GEOFABRIK has already done this, if SPLITTER kept the version dataset information., I could use it.
Greg
On Sat, Feb 27, 2016 at 4:08 PM, Nelson A. de Oliveira <naoliv@gmail.com> wrote:
On Sat, Feb 27, 2016 at 5:44 PM, greg crago <gregcrago@gmail.com> wrote:
I want to download sections of the osm database (USA- Michigan) and make off line changes in JOSM and upload back to OSM when I am back online.
Will you edit the whole state? Do you really need to have all of it?
Editing large areas and a lot of objects isn't usually the best thing to do: it can cause duplicate data in OSM and/or you will hit a lot of conflicts.
overpass is the easiest way to get fresh data, but you need to filter a smaller region or some specific data that you want. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-- Dave Swarthout Homer, Alaska Chiang Mai, Thailand Travel Blog at http://dswarthout.blogspot.com
Hi Greg, in addition to Daves commnets I'd like to mention that you can use the output of splitter as a separate layer in JOSM. I sometimes use this when I cut large multipolygon relations : one layer with the old data from splitter, one with the mapnik layer (tms[19]:https://{switch:a,b,c}.tile.openstreetmap.org/{zoom}/{x}/{y}.png) and one with the downloaded data. Switching between those layers allow to compare. Maybe that helps ? Gerd P.S. I still think about adding an --keep-version option to splitter, but for other reasons. ________________________________ Von: mkgmap-dev-bounces@lists.mkgmap.org.uk <mkgmap-dev-bounces@lists.mkgmap.org.uk> im Auftrag von Dave Swarthout <daveswarthout@gmail.com> Gesendet: Montag, 29. Februar 2016 05:31 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Problem uploading data to JOSM wit 20 hour old OSM data, maybe a SPLIITTER issue? Greg, Using JOSM and working with data that other people are also working with is a recipe for disaster. Resolving the inevitable conflicts that will arise is tedious and complicated. I have spent hours resolving conflicts made by "the other user" only to find out later that it was _my own data_ I was working against. The method you're trying to use will only work well if you're the only person modifying the data in that particular part of the world. Another possibility is to download smaller areas to work in which will keep the number of edits down. That way you won't have thousands of conflicts when you do get Internet again. And if you do, you won't lose much work. Not very satisfying I'm afraid but working with a live Internet connection is by far the best option. On Sun, Feb 28, 2016 at 9:42 PM, greg crago <gregcrago@gmail.com<mailto:gregcrago@gmail.com>> wrote: I have thousands of map errors I have collected on my GPS from ground observations. I just have not had time to upload them to OSM. I do not have Internet access all the time. Downloading many states and then breaking them up using a 'splitter-type' program, gives me the flexibility to edit sections of the country I want to. I thought UPLOAD conflicts only occur when objects I HAVE MODIFIED, conflict with objects in the OSM database that have been modified since I download MY VERSION. If I have large areas of 'outdated' OSM data, that I do not modify, I should not have any problems when I upload, since I am not sending that 'untouched' data back to OSM, correct? I thought this was a viable 'offline' solution. I do not know how to filter overpass to download state-by-state regions. GEOFABRIK has already done this, if SPLITTER kept the version dataset information., I could use it. Greg On Sat, Feb 27, 2016 at 4:08 PM, Nelson A. de Oliveira <naoliv@gmail.com<mailto:naoliv@gmail.com>> wrote: On Sat, Feb 27, 2016 at 5:44 PM, greg crago <gregcrago@gmail.com<mailto:gregcrago@gmail.com>> wrote:
I want to download sections of the osm database (USA- Michigan) and make off line changes in JOSM and upload back to OSM when I am back online.
Will you edit the whole state? Do you really need to have all of it? Editing large areas and a lot of objects isn't usually the best thing to do: it can cause duplicate data in OSM and/or you will hit a lot of conflicts. overpass is the easiest way to get fresh data, but you need to filter a smaller region or some specific data that you want. _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev _______________________________________________ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- Dave Swarthout Homer, Alaska Chiang Mai, Thailand Travel Blog at http://dswarthout.blogspot.com
On Sun, Feb 28, 2016 at 11:42 AM, greg crago <gregcrago@gmail.com> wrote:
Downloading many states and then breaking them up using a 'splitter-type' program, gives me the flexibility to edit sections of the country I want to.
For this you can use osmconvert http://wiki.openstreetmap.org/wiki/Osmconvert You can download a full country and split it into multiple parts, using poly files for this. Take a look at sections "Bounding-polygon task", "Premade polygon files" and "Converting to/from POLY format" here: http://wiki.openstreetmap.org/wiki/Osmosis/Polygon_Filter_File_Format
I thought UPLOAD conflicts only occur when objects I HAVE MODIFIED, conflict with objects in the OSM database that have been modified since I download MY VERSION. If I have large areas of 'outdated' OSM data, that I do not modify, I should not have any problems when I upload, since I am not sending that 'untouched' data back to OSM, correct?
It indeed can only conflict on modified objects, but what I was trying to say is that the chance to create duplicate objects or have conflicts increases a lot when using older data + editing in a large area + editing a lot of data.
participants (4)
-
Dave Swarthout -
Gerd Petermann -
greg crago -
Nelson A. de Oliveira