Script to update an OpenShift repository with changes made on server

If you haven’t tried OpenShift yet, you should!

Anyway I have a WordPress installation there. When upgrading WordPress to a new version, files are changed on the server but they would be overwritten if I push the repository. So I created a script to download a snapshot of the OpenShift app, extract the changes in the php directory where WordPress resides, and add them to repository.

It is fully automated except for the crucial git push.

The script is called update-repo-from-snapshot.sh and is meant to be commited to the root of each repository.

PHP Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes)

I’m a very happy user of New Relic for my websites, even if I’m only leeching on the free tier.

One of my WordPress sites, taurin.se has been suffering from this out-of-memory error for quite some time now and I finally tracked it down. New Relic shows the stack trace, with the failing line on top:

in wpdb::get_results called at /usr/share/wordpress/wp-includes/wp-db.php (1413)
in wpdb::get_results called at /usr/share/wordpress/wp-includes/meta.php (295)
in update_meta_cache called at /usr/share/wordpress/wp-includes/post.php (4133)
in update_postmeta_cache called at /usr/share/wordpress/wp-includes/post.php (4113)
in update_post_caches called at /usr/share/wordpress/wp-includes/query.php (2534)
in WP_Query::get_posts called at /usr/share/wordpress/wp-includes/query.php (2695)
in WP_Query::query called at /usr/share/wordpress/wp-includes/query.php (2791)
.
.
.

So, it fails when updating the postmeta cache. What’s that? Well, had a look in the database.

In the database I found the wp_postmeta table. A quick check revealed that it contained thousands of syndication_item_hash entries. These are added by the FeedWordPress plugin.

Unfortunately taurin.se is quite low priority for me, and the blogs I have been syndicating seem quite dead. But the upside is that I could just disable FeedWordPress and delete all syndication_item_hash entries from the wp_postmeta table.

Now my web server should work much better!

Disable some useless stuff in Raspbian

I’m currently running three Raspberry Pi projects, with Raspbian as my choice of distribution. One of the machines will be co-located at FS Data and run ownCloud. The other will be handle switching a central heater between oil and electricyty via SMS. The third is a surveillance camera heavily inspired by Dummy camera made smart with Raspberry Pi by my talented colleague Tomas Bjerre.

While examining the default Raspbian image I noticed there were some things I will never use, so I disabled them to free up some RAM:

  1. Disable the consoles by putting a # in front of all lines containing /sbin/getty in /etc/inittab:

    #1:2345:respawn:/sbin/getty --noclear 38400 tty1
    #2:23:respawn:/sbin/getty 38400 tty2
    #3:23:respawn:/sbin/getty 38400 tty3
    #4:23:respawn:/sbin/getty 38400 tty4
    #5:23:respawn:/sbin/getty 38400 tty5
    #6:23:respawn:/sbin/getty 38400 tty6
    .
    .
    .
    #T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

    If you get into network issues and need the console you can always mount the SD card on another machine and edit the file there!

  2. If you don’t have a keyboard connected or otherwise don’t want global hotkeys, disable the triggerhappy global hotkey daemon:

    sudo update-rc.d triggerhappy disable

  3. If you have Transmission installed but is not using UPnP, disable the minissdpd daemon for keeping track of UPnP devices:

    sudo update-rc.d minissdpd disable

  4. If you run Apache, disable modules you don’t use. Maybe:

    sudo a2dismod dir
    sudo a2dismod cgi
    sudo a2dismod negotiation
    sudo a2dismod autoindex
    sudo a2dismod setenvif

Using mod_spamhaus to block TOR in Apache

Some web spammers use the Tor Project to hide their wrongdoings. Because of this, I want to block Tor exit nodes from submitting forms on my web sites. However, there are many legitmate uses of the Tor Project, so I don’t want to block GET requests but primarily POST requests.
  1. Run sudo apt-get install libapache2-mod-spamhaus
  2. Open /etc/apache2/mods-enabled/mod-spamhaus.conf for editing (as root)
  3. Edit the MS_METHODS configuration setting. Make sure that POST is included and GET is not. Example:
    MS_METHODS  POST,PUT,OPTIONS,CONNECT
  4. Edit the MS_Dns configuration setting. If the IP address of your server is 198.51.100.222, and the port of your web server is 80 as usual, you start with the port number, reverse the numbers in the IP address and the setting becomes:
    MS_Dns 80.222.100.51.198.ip-port.exitlist.torproject.org
    So, for IP address A.B.C.D and port E, the value becomes E.D.C.B.A.ip-port.exitlist.torproject.org
  5. You may also want to edit the MS_CustomError setting. In my case it looks like this:
    MS_CustomError “Limited access for certain clients. Please contact abuse at 2good.net to get full access to our services.”
  6. Enable the apache module with sudo a2enmod mod-spamhaus
  7. Restart apache with sudo service apache2 restart
Your will get log entries like this:

[Fri Nov 16 04:28:05 2012] [crit] [client 37.59.162.218] mod_spamhaus: address 218.162.59.37.80.222.10.74.109.ip-port.exitlist.torproject.org is blacklisted. Deny connection to forum.taurin.se/index.php, referer: http://forum.taurin.se/index.php?topic=1731.0

Unable to use Mosh to connect to one of my servers

I’ve only used Mosh: the mobile shell for a  couple of weeks, but I am definitely a convert! However, I couldn’t get mosh to connect to one of my servers (an OpenVZ instance).

The error manifested itself as the client trying to connect to the server but not getting any response:

mosh: Connecting... [To quit: Ctrl-^ .]

Today I decided to debug the issue. As the client actually was trying to connect, I tried to simply start the server:

$ mosh-server new

MOSH CONNECT 60001 rSbp4ENDdzd/TKBOrrEeVg

mosh-server (mosh 1.2.2)
Copyright 2012 Keith Winstein
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[mosh-server detached, pid = 32582]

But when I checked the process list, it wasn’t running anymore:

$ pgrep mosh-server
$

Time to add some verbose output:

$ mosh-server new -v 

MOSH CONNECT 60001 hQ5ab1JjgUytXRZmy5ti6A

mosh-server (mosh 1.2.2)
Copyright 2012 Keith Winstein
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[mosh-server detached, pid = 14208]
forkpty: No such file or directory

This must be important: forkpty: No such file or directory, but what does it mean?

Some googling showed that it meant that /dev/pts was not mounted on my system, and it was indeed not mounted:

$ mount
 /dev/simfs on / type reiserfs (rw,usrquota,grpquota)

(Yes, I’ve also wondered why my host is running ReiserFS.)

So I checked my /etc/fsab:

$ cat /etc/fstab
proc  /proc       proc    defaults    0    0
none  /dev/pts    devpts  rw          0    0

It’s there, but still not mounted? Oh well, easy fix:

sudo mount /dev/pts

After that fix, I could successfully connect to my server with mosh!

Injecting context into controller?

I’m toying with my own PHP micro-framework. I’m trying to make it both simple and easy; explicit but not verbose. (Good luck to me!)

For testability I have abstracted the Superglobals so that I have full control of them in my tests. Currently they are passed as a parameter to the controller action. Would it make sense to have them injected into the controller instead? It might be simpler, but it is less explicit.

Incorrect result from WordPress plugin_dir_url

Consider this code in akismet.php from the ubiquitous Akismet plugin for WordPress:

define('AKISMET_PLUGIN_URL', plugin_dir_url( __FILE__ ));

My WordPress installation is at /usr/share/wordpress, but the Akismet plugin is installed to /var/lib/wordpress/plugins/akismet/ and there is a symlink (symbolic link) inbetween. This means that __FILE__ is /var/lib/wordpress/plugins/akismet/akismet.php but plugin_dir_url does not understand this, and will incorrectly return:

/wp-content/plugins/var/lib/wordpress/plugins/akismet/

This is not a proper URL for my WordPress installation!

I’m still on WordPress 3.0.something, but it seems like this issue is still a problem in newer versions. There are a number of bug reports for this and similar issues:

I used the WP_fix_for_plugin_basename_to_allow_symlinks_2011-07-28* pair of patches attached to bug report 16953 to solve the issue on my server. It will patch the plugin_basename implementation, which seems to contain the root cause of this issue.

The Inner-platform API anti-pattern

I learned the term Inner-platform effect the other day and it perfectly describes an API that I have touched upon.

First of all, what is the Inner-platform effect? It basically means that in attempt to make an application as flexible as possible it is implemented so that it creates a new platform that abstracts the original platform. I immediately associated to the API described below.

Let’s see if you can spot the Inner-platform effect in this URL, heavily anonymized but if you have been exposed to it you will probably recognize it immediately.

http://server/execute/clientSystemConnector
?service=createUser&key=id&value=97580
&key=name&value=David%20Eriksson&key=city&value=Ronneby

There are actually three examples of the Inner-platform effect in the above URL. The first is example is the “service” parameter. Instead of having separate URLs for separate services, the service name is a parameter. This means that the server platform first figures out what to about the /execute/clientSystemConnector path component of the URL, then the Inner Platform need to figure out how to handle the different possible values of the service parameter. (Anyone thinking about a giant “switch” statement?)  To avoid the Inner-platform effect, each service should have its own path in the URL.

The second example is blatantly obvious: having key and value parameters that specifies the names and values of the actual parameters. The impact of this “feature” is that the platform  provides a list of keys and a list of values to the inner platform, which must extract the actual parameters for use in the application. Without the Inner-platform effect the application could have received the parameters directly from the original platform.

It is probably not so easy to spot the third example, but it concerns authentication. Authentication (if it could be called that) for the services in this API is based on using different paths for different clients. So the above URL is used by the “clientSystem”. If “anotherClientSystem” need to execute the same service, the URL would be:

http://server/execute/anotherClientSystemConnector
?service=createUser&key=id&value=97580&...

So instead of using the built-in authentication mechanism (HTTP Basic Auth) in the platform, the Inner platform goes its own way.

How would I build an API to avoid the Inner platform? Except addressing the above issues I would also make it a POST (or PUT) request, as it handles the creation of a resource.

POST http://server/user/create HTTP/1.1
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
Content-Type: application/x-www-form-urlencoded

id=97580&name=David%20Eriksson&city=Ronneby

Goodbye, Inner Platform!

PS. The origins of the API is probably 8-10 years old by now, and maybe it was a good idea at the time.

PPS. See the Inner-platform effect article on wikipedia for more examples.