Generating Bahn Layouts

[BertTijhuis]

Hello everyone

Due to some progress at Openstreetmap (they create another way off accessing Openstreetmap data (in a compressed way) and some bug resolving I've created last 2 weeks version 0.19.
This version is available as from now.

So take a look at:

http://www.xs4all.nl/~wanl0049/ops2jbss/

again.


It solves also the issue where Chris is struggling with and it is much faster.

One small issue still appears (the 'cache' are is not cleaned after everything is well processed).

Please use the CCA possibility in the menu, to clean this.

Regards, Bert Tijhuis

[Chris]

I am getting an error: "XMLfile GB.OSM is invalid and will be deleted (Please restart the extractor again)" gb.osm in this case was the file from http://download.geofabrik.de/osm/europe/ which I am assured is valid. It turns out that the problem was converting the supplied filename to upper case. There's no need to do this: on a case-sensitive filesystem you will not find the file, and on a case-insensitive filesystem it doesn't matter what case you supply.

[BertTijhuis]

I am getting an error: "XMLfile GB.OSM is invalid and will be deleted (Please restart the extractor again)" gb.osm in this case was the file from http://download.geofabrik.de/osm/europe/ which I am assured is valid. It turns out that the problem was converting the supplied filename to upper case. There's no need to do this: on a case-sensitive filesystem you will not find the file, and on a case-insensitive filesystem it doesn't matter what case you supply.

I wasn't aware of this nice GEOFABRIK.
In the meantime I will download a file a try to find why it is failing
Regards,

[Chris]

My Perl is about as good as my Swahili, but there are two problems here. First the casing:

Code: Alles auswählen

# ops2jbss.pl:647
                $veld2 =~ tr/a-z/A-Z/;

I don't know why this line is here, since the few places I've see that handle user input are already rigged for both cases, e.g.

Code: Alles auswählen

                if ($choise =~ /^[gG]$/) { # both cases specified
                if ($choise =~ /^cca/i) { # case-insensitive match

The error message comes from here:

Code: Alles auswählen

# ops2jbss_e.pl:170
  print "Process file: $file\n" if ($debug);
  $dbh->do("begin transaction immediate");
  my $ref = eval { XMLin($file); };
  if ($@) {
  	 print "XMLfile $file is invalid and will be deleted (Please restart the extractor again)\n" if ($debug);
  	 unlink($file);
  	 return 0;
	}

That looks like it's producing that same message regardless of what the error actually was, which IMO makes it poor to assume it'll be an invalid file, and rather nasty to insist on unlinking the file afterwards.

Just my .02

[BertTijhuis]

Hello Chris,

Thanks for digging through the code:

The real problem is:\
Choise ?: > o great_britain.osm
Process file: GREAT_BRITAIN.OSM
Out of memory!
Out of memory!
Out of memory!

Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

Due to the fact that XMLin is part of the XML::Simple module (not maintained by myself) I can't solve this issue. Try to use smaller files


But please in you're case download the newest sources,
These won't delete the file afterwards, and specified the root course of the failure.

I've noticed that you are still working with an older release please use release 0.20 or upwards

Regards,
Bert Tijhuis

[Chris]

Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

That was not the cause of the issues I raised. The issue I raised was that you are incorrectly converting the case of the filename, and then throwing back an error message which is inaccurate. This is independent of the file being impossible to process this way on a 32-bit machine.

Code: Alles auswählen

chris@testbox:~/ops2jbss$ ls -l test.osm
-rw-r--r-- 1 chris chris 0 2010-05-28 19:22 test.osm
[...]
Choise ?:  > o test.osm
Process file: TEST.OSM
XMLfile TEST.OSM is invalid (Please restart the extractor when nescessarely)
Reason:
File does not exist: TEST.OSM at ops2jbss_e.pl line 187

I've noticed that you are still working with an older release please use release 0.20 or upwards

I was using 0.19 yesterday, and you released 0.20 today. I notice that 0.20 still has the upper-case tr - why is it there?

Also, what release should I have been using yesterday?

[micha88]

You can reduce file size with the tool osmosis: you can create a much smaller xml-file containing only railway lines.

[BertTijhuis]

Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

That was not the cause of the issues I raised. The issue I raised was that you are incorrectly converting the case of the filename, and then throwing back an error message which is inaccurate. This is independent of the file being impossible to process this way on a 32-bit machine.

Code: Alles auswählen

chris@testbox:~/ops2jbss$ ls -l test.osm
-rw-r--r-- 1 chris chris 0 2010-05-28 19:22 test.osm
[...]
Choise ?:  > o test.osm
Process file: TEST.OSM
XMLfile TEST.OSM is invalid (Please restart the extractor when nescessarely)
Reason:
File does not exist: TEST.OSM at ops2jbss_e.pl line 187

I've noticed that you are still working with an older release please use release 0.20 or upwards

I was using 0.19 yesterday, and you released 0.20 today. I notice that 0.20 still has the upper-case tr - why is it there?

Also, what release should I have been using yesterday?

-----
I've now 2 answers to you're questions

1) I will fix the bug that only occurs in a none Windows environment. (within 2 months)

2) I will start to work on a version that could use precompiled OSM files from the internet (e.g. http://download.geofabrik.de/osm/europe/ )
and use the splitter that is also used for the MKgmap utility. (within 2 months).



Of course in the mean time you can use OSMOSIS and rename you're filenames to lower case.


Regards,
Bert Tijhuis

[micha88]

I've cut of the German postings and moved them to "Probleme und Fehlermeldungen".

[Chris]

Something seems to be out of order somewhere. Did my first live extraction with 0.20 and got the following:

DBD::SQLite::db do failed: SQL logic error or missing database
cannot start a transaction within a transaction at ops2jbss_e.pl line 199.

It comes at the point where "$cache-file will be processed". Centred at 51.74N 4.065W, 50x50km, scale 64.

[Chris]

More bugs:

1. Start with no database. Attempt to import a file. You get a zero-length file where the database should be, and the SQL process bails with "table 'way' does not exist" (etc.). The database has not been initialised correctly. CDB does not rectify this (since all it does is unlink the file).
Workaround: start an online extraction, and abort shortly after it starts - the database is then initialised and the import can be performed.

2. Program deletes files it has attempted to import, even when the import has failed due to #1.

[ramops]

Anything yet on this?

[BertTijhuis]

Hello,

Just after one year, there is some news about this.

Due to a remark of Hamzaa I've found out that the current procedure isn't working anymore correctly.

So I've updated the version (021) and this is available as of 20 november 2011.

@Chris: With this current version it is possible to process large files from www.geofabrik.de but of course you've to unzip these files first (7zip)
The program doesn't consume that much memory anymore and you don't run out of memory.

Due to the fact that we don't run out of memory anymore I've introduced a disadvantage that its running slower than the previous versions.

Please try this new version, and give me some feedback

[BertTijhuis]

It seems that there is something wrong in Generating Bahn layouts.

Partly this has to do with the richness of the Openstreetmap data. So you cant find the needle in the haystack anymore.
And sometimes some large rectancular lines are generated.

So I still working on this issue and hope to solve this asap

Regards

[BertTijhuis]

Hello,

Just before going to Kuala Lumpur I've finalized this version 0.22. (This is a bugfix of 0.21)
Please DONt use version 0.21

Tested with several area's (from Geofabrik.de) with Luxembourg (Pretty small country and I like this), Ile de France (on request) and Kuala Lumpur and ofcourse the Netherlands

Everything seems to be working correctly