The Sorrows of Young OSM User

OpenStreetMap (OSM for short) is a web site that not only displays maps but also provides access to the database from which these maps are rendered. So far, so good.

Our company now came up with the idea that given a street name and a region where the street is located (around the globe there are tons of streets that go by the name "main street") we use the Overpass API to query the ways that make up these streets.

The plural streets reflects an important issue: Even within a single city there may be several streets that go by identical names.

In principle this can be achieved assuming that the user is young enough not to die of old age before the query yields a result.

We looked into alternative ways for directly fetching data from OSM but all either turned out to be too slow or simply not up to the task.

It seems as if the only feasible way is to have two server tasks:

1. One that fetches the data from OSM.
2. One that provides the full path via an API.

The point is that there are still three possible ways of implementing the first server task.

1. To have a job that is triggered at certain times and that imports the data.
2. To have let the server operate as some sort of caching proxy for the data.
3. To have a mix of the two previous solutions.

My idea is as follows:

1. Initially the server's database is populated using an appropriate import feature.
2. If a request occurs it is checked whether the requested data is in the database and is not outdated (with a reasonable definition of to be outdated that does not require a request to OSM).
3. If the server's data is outdated an updated version is requested from OSM. This can yield two results:
   1. The query can quickly return the current information. In this case the data is stored in the server's database and delivered to the user.
   2. The query does not return the result within a reasonable time. In this case the server returns the data that is stored locally. Chances are very good that the outdated data still is a reasonable approximation of the most recent data. This case has two subcases:
      1. The query simply takes too long but still returns a result. In this case the returned data is used to update the database.
      2. The query times out and does not return a result. In this case the outdated data is kept.
4. Every now and then the server checks which data is most outdated and tries to obtain these data.

Comments?

It's been a long time

I haven't been posting here for eons. Hopefully that will change for the better :-)

Polar Ice Page Update

So I updated the polar sea ice page once again. Note the ever growing area of sea ice on the Southern hemisphere.

PDF to animated GIF

In order to illustrate how fragmentation occurs, I created a PDF document (Fragmentierung.pdf) with one page for each step of the fragmentation.

 This is how I converted it to an animated GIF. The first step was to create a series of PPM files, one file for each page. The option -r 384 sets the resolution to 384 dpi. This somewhat odd resolution is four times 96 dpi which in turn is the resolution of the PDF being converted.

The pdftoppm command generates files that are numbered.

The files resulting from this conversation are huge in both display and storage size so the next step is to rescale them to a reasonable resolution. My choice was a width of 800 times 600. the for loop iterates over all PPM files, the output of the pamscale command sent to a file with a name based on the original file name using the shell's own capability to process strings. Finally, the --filter option controls how precisely the original high-resolution image is mapped to the low resolution one.

Let's assume the original file name is Fragmentierung-1.ppm. Then ${i%.ppm} yields Fragmentierung-1 and ${i%.ppm}_new.ppm results in Fragmentierung-1_new.ppm.

The result of this command - which on my old machine requires a considerable amount of time - is a new series of files that are considerably smaller than the original ones.

As the old files are no longer needed, I replace them with the corresponding new ones.

Now we have ppm files of the right size that are simply numbered.

The PPM files usually have more than 256 colors (which is the maximum for GIFs). For this reason I use pnmcolormap to generate colormaps for the individual files that have no more than 256 colors.

This command results in tons of diagnostics output created by pnmcolormap.

The next command performs two tasks at the same time: pnmremap uses the colormap to map the original colors to no more than 256 with the -fs option making sure that Floyd-Steinberg dithering is used. Then the output converted to GIF formatusing ppmtogif and is written to a file with the file name extension .gif.

In this case both commands are a bit chatty so you again get lots of output.

Now we are almost done as we have a set of GIF files.

Just discard all the intermediary files. The next command looks complicated like hell but it isn't. gifsicle generates an animated GIF from a number of input files that need to be in GIF format.

• --loopcount=forever makes the animation loop forever (i.e. after displaying the last frame of the animation it always restarts with the first one)
• --optimize=2 optimizes the generated file as much as possible (without this option the file gets even bigger)
• --colors=256 makes sure that the generated file has no more than 256 colors. This may seem superflous as each input file already has no more than 256 colors but the colors may happen to be different so that the overall number can be larger than 256.

The most important part of the command is the --delay options. If you provide this option, it applies to each image that follows up to the next command. Hence, in this case the first delay of 3 seconds applies to Fragmentierung-1.gif only, the delay of 1 seconds to Fragmentierung-2.gif through Fragmentierung-8.gif and the second delay of 3 seconds to Fragmentierung-9.gif.

Finally done. Here is the animated gif that results from this effort.

Katyusha (Катюша)

I am a huge fan of Igor Presnyakov and what he does to/with his acoustic guitar.

Hear him perform the Russian song Katyusha (Катюша)

Distances from Bonn

I came up with the strange idea of sorting European capitals by how far they are from the town I live in (Bonn, Germany) and found that half a dozen of foreign capitals are nearer than the capital of Germany.

City	Country	Distance in km
Luxembourg City	Luxembourg	142
City of Brussels	Belgium	195
Amsterdam	Netherlands	234
Paris	France	400
Bern	Switzerland	425
Vaduz	Liechtenstein	437
Berlin	Germany	478
London	United Kingdom	510
Prague	Czech Republic	527
Copenhagen	Denmark	659
Vienna	Austria	726
Ljubljana	Slovenia	755
Bratislava	Slovakia	779
Monaco	Monaco	779
City of San Marino	San Marino	857
Zagreb	Croatia	857
Budapest	Hungary	942
Dublin	Republic of Ireland	957
Warsaw	Poland	976
Oslo	Norway	1047
Rome	Italy	1065
Vatican City	Vatican City	1065
Sarajevo	Bosnia and Herzegovina	1143
Stockholm	Sweden	1181
Belgrade	Serbia	1195
Vilnius	Lithuania	1300
Riga	Latvia	1308
Podgorica	Montenegro	1347
Madrid	Spain	1421
Minsk	Belarus	1430
Tirana	Albania	1432
Ankara	Turkey	1447
Skopje	Macedonia	1463
Tallinn	Estonia	1474
Sofia	Bulgaria	1522
Helsinki	Finland	1530
Bucharest	Romania	1583
Chişinău	Moldova	1639
Kiev	Ukraine	1648
Valletta	Malta	1745
Lisbon	Portugal	1845
Athens	Greece	1931
Moscow	Russia	2086
Reykjavík	Iceland	2257
Tbilisi	Georgia	3034
Yerevan	Armenia	3106
Baku	Azerbaijan	3469

If you find this idea interesting what about composing a similar table for the location you live in?

Obtaining Tweeted Images in Original Size

An increasing number of tweets contains images uploded to twitter. Here is one example:

A particular example

I assume that you know how to find out URLs of images that are used on a web page.

The URL of the image you see in the tweet is https://pbs.twimg.com/media/A0BRGqGCcAEBdHW.jpg with the dimensions 600px × 428px (scaled to 435px × 310px).

If you click on the image to obtain a larger version you are presented with https://pbs.twimg.com/media/A0BRGqGCcAEBdHW.jpg:large with the dimensions 1,024px × 730px. This still isn't the original image.

I then made a well-educated guess and tried
https://pbs.twimg.com/media/A0BRGqGCcAEBdHW.jpg:orig and it actually worked; I got the original size image with dimensions 2,048px × 1,460px.

So all you need to do to obtain an original size image is to append »:orig« to the URL of the small version of the image.

Note that this only works for images uploaded to twitter not for images hosted on flickr.

Here is one example: https://twitter.com/VirtualAstro/status/241683461155475456 shows a nice photo of a blue moon; the 3 MP image can be found at https://pbs.twimg.com/media/A1qS3JvCQAE0QPv.jpg:orig