Increase disk size for OpenShift Origin Virtual Machine

Loving OpenShift, but having more than the three projects that already cost more than I earn from them I run an OpenShift Origin Virtual Machine on my home server.

So today I was trying to get Uptime running on OpenShift via uptime-openshift. A truly wonderful way of installing applications, except that it didn’t go so well:

$ rhc app create uptime nodejs-0.10 mongodb-2.4 --from-code=https://github.com/rkmallik/uptime-openshift
Application Options
-------------------
Domain: origin
Cartridges: nodejs-0.10, mongodb-2.4
Source Code: https://github.com/rkmallik/uptime-openshift
Gear Size: default
Scaling: no

Creating application 'uptime' ... 
The server did not respond correctly. This may be an issue with the server configuration or with your connection to the server (such as a Web proxy or firewall). Please
verify that you can access the OpenShift server https://172.16.3.3/broker/rest/domain/origin/applications

Or…

$ rhc app create uptime nodejs-0.10 mongodb-2.4 --from-code=https://github.com/rkmallik/uptime-openshift
Application Options
-------------------
Domain: origin
Cartridges: nodejs-0.10, mongodb-2.4
Source Code: https://github.com/rkmallik/uptime-openshift
Gear Size: default
Scaling: no

Creating application 'uptime' ... Unpacking premade journal
tar: journal/prealloc.1: Wrote only 2048 of 10240 bytes
tar: Exiting with failure status due to previous errors

What’s going on? It turns out I was running low on disk space. The virtual machine image was only 8 GB and there was too little disk space left when the Mongo cartridge wanted to install the pre-allocated journals!

So, time to enlarge the disk! In short, the steps I took were:

  1. Convert virtual machine disk image to .vdi format
  2. Enlarge .vdi disk image
  3. Add partition with new disk space
  4. Add new partition to LVM volume group
  5. Extend LVM logical volume
  6. Resize file system

Now, let’s dive into the details!

Convert virtual machine disk image to .vdi format

Shutdown the virtual machine first!

As the VirtualBox user on the hypervisor, in the directory for the virtual machine:

$ VBoxManage clonehd origin-vm.vmdk origin-vm.vdi --format VDI
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone hard disk created in format 'VDI'. UUID: 646d9ed4-c751-45a3-9f08-0946e11591c9

Enlarge .vdi disk image

As the VirtualBox user on the hypervisor, in the directory for the virtual machine:

$ VBoxManage modifyhd origin-vm.vdi --resize 81920
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Enlarge it with as much as you want to allow the disk image to grow on the hypervisor! The unit is megabytes.

Now detach the .vmdk file and attach the .vdi file from the virtual machine. I did this with phpVirtualBox, so the corresponding command line is left as an exercise for the reader.

Delete the the .vmdk file, or keep it as backup if you like.

Add partition with new disk space

Start the virtual machine and login. Default root password is changeme. Being a Linux user for about 15 years, I’m no stranger to fdisk:

# fdisk /dev/sda

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
 switch off the mode (command 'c') and change display units to
 sectors (command 'u').

Command (m for help): c
DOS Compatibility flag is not set

Command (m for help): u
Changing display/entry units to sector

Command (m for help): n
Command action
 e extended
 p primary partition (1-4)
p
Partition number (1-4): 3
First sector (16777216-167772159, default 16777216): 
Using default value 16777216
Last sector, +sectors or +size{K,M,G} (16777216-167772159, default 167772159): 
Using default value 167772159

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003c6cc

 Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 16777215 7875584 8e Linux LVM
/dev/sda3 16777216 167772159 75497472 8e Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

If told to reboot, do that. Otherwise Linux will not show the new partition.

Add new partition to LVM volume group

# vgextend vg_broker /dev/sda3
 No physical volume label read from /dev/sda3
 Physical volume /dev/sda3 not found
 Physical volume "/dev/sda3" successfully created
 Volume group "vg_broker" successfully extended

Extend LVM logical volume

Use all available space:

# lvextend -l +100%FREE /dev/mapper/vg_broker-lv_root
 Extending logical volume lv_root to 78,71 GiB
 Logical volume lv_root successfully resized

Resize file system

First find out the LSize in sectors:

# lvs --units s
 LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
 lv_root vg_broker -wi-ao---- 165060608S 
 lv_swap vg_broker -wi-ao---- 1671168S

Second, resize the file system. Change capital S to lower-case:

# resize2fs /dev/mapper/vg_broker-lv_root 165060608s
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/mapper/vg_broker-lv_root is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 5
Performing an on-line resize of /dev/mapper/vg_broker-lv_root to 20632576 (4k) blocks.
The filesystem on /dev/mapper/vg_broker-lv_root is now 20632576 blocks long.

 Done!

If only I remembered why I needed more disk space?!

Why use I²C to access 1-Wire from Raspberry Pi?

A friend of mine asked the excellent question on why I bought an I²C 1-Wire expansion module for 300 SEK (about €30) when there are kernel drivers for using 1-Wire over GPIO right away and exposing the 1-Wire network below /sys/bus/w1/. I honestly didn’t know of this option!

Now I’ve made some investigations and realized that for example the DS2406 1-Wire chips I already bought are not yet supported by the kernel. There is a patch, for those who like to compile their own Raspberry Pi kernel. I don’t.

Also, OWFS seems to do an OK job of hiding the way of interfacing the 1-Wire network, so as long as use OWFS to access my 1-Wire stuff I should be able to switch to the w1 kernel drivers in the future.

For now, I will continue to use the I²C 1-Wire expansion module, because it works fine, I already bought more than one of them (!), and all my stuff is supported.

Home automation with 1-Wire and Raspberry Pi

Me and my wife recently bought a cottage half an hour’s drive from where we live.

She is enjoying herself immensely with interior decorating, grinding floors and mowing the lawn. She is also looking forward to washing and painting the windows and exterior walls…

Naturally I will also do some work on and around the cottage. I use to say that I’m only handy when forced to, but that’s not entirely true… :-)

Now for the fun part. I have taken on to implement home automation for the cottage. At start it will be mostly monitoring temperature, measuring electricity consumption and checking door and motion sensors.

The heart of the home automation is a Raspberry Pi running Rasbian. On the Raspberry Pi I have mounted an I²C 1-Wire expansion module and installed owfs.

My main sources of 1-Wire hardware is the special interest Swedish web shop m.nu. For other stuff I use the Radio Shack of Sweden: Kjell & Company, and, of course, eBay!

This is a list of some stuff I bought on eBay for various parts of this project. Prices provided in case want to compare to ordinary web shops… :-)

  • 10 × PIR motion sensors, $12.59
  • RJ45/RJ11 cable tester, $2.91
  • 100 × RJ11 plugs, C $4.33
  • RJ45/RJ11 Crimp Tool, GBP 4.99
  • NoIR camera module for Raspberry Pi, $28.49
  • 2 × Infrared lights for NoIR camera module, $9.49
  • 9800mAh DC 12V rechargeable Lithium-ion Battery, $27.53
  • Micro USB splitter charge cable, GBP 1.49
  • 20 × magnetic door/window sensors, $17.12

So, what is all that stuff good for? My plan is to make a series of posts about the various parts of the project, but I make no guarantees… :-)

Now-proofing your development!

At work today we discussed the concept of future-proofing, which I consider a bad practice. I can elaborate another time but my talented colleague Anders Holmberg invented now-proofing on the spot. That’s what we should be aiming for!

Our bried discussion was inspired by this quote:

If you are not embarrassed by the first version of your product, you’ve launched too late.

– Reid Hoffman

Source: http://startupquote.com/post/855482768

David’s technology radar

Inspired by ThoughtWorks Technology Radar, here is an attempt to write down my current world view:

Adopt

Trial

  • Mozilla Persona – use the same e-mail address and password to login to many web sites (not so many yet though)
  • OpenShift – cloud application platform
  • “Private cloud” – run your own “cloud” server
  • Raspberry Pi – small and cheap computer
  • Server Sent Events
  • WebSockets

Assess

  • Chef or Puppet
  • Docker
  • Gearman
  • Vagrant

Script to update an OpenShift repository with changes made on server

If you haven’t tried OpenShift yet, you should!

Anyway I have a WordPress installation there. When upgrading WordPress to a new version, files are changed on the server but they would be overwritten if I push the repository. So I created a script to download a snapshot of the OpenShift app, extract the changes in the php directory where WordPress resides, and add them to repository.

It is fully automated except for the crucial git push.

The script is called update-repo-from-snapshot.sh and is meant to be commited to the root of each repository.

PHP Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes)

I’m a very happy user of New Relic for my websites, even if I’m only leeching on the free tier.

One of my WordPress sites, taurin.se has been suffering from this out-of-memory error for quite some time now and I finally tracked it down. New Relic shows the stack trace, with the failing line on top:

in wpdb::get_results called at /usr/share/wordpress/wp-includes/wp-db.php (1413)
in wpdb::get_results called at /usr/share/wordpress/wp-includes/meta.php (295)
in update_meta_cache called at /usr/share/wordpress/wp-includes/post.php (4133)
in update_postmeta_cache called at /usr/share/wordpress/wp-includes/post.php (4113)
in update_post_caches called at /usr/share/wordpress/wp-includes/query.php (2534)
in WP_Query::get_posts called at /usr/share/wordpress/wp-includes/query.php (2695)
in WP_Query::query called at /usr/share/wordpress/wp-includes/query.php (2791)
.
.
.

So, it fails when updating the postmeta cache. What’s that? Well, had a look in the database.

In the database I found the wp_postmeta table. A quick check revealed that it contained thousands of syndication_item_hash entries. These are added by the FeedWordPress plugin.

Unfortunately taurin.se is quite low priority for me, and the blogs I have been syndicating seem quite dead. But the upside is that I could just disable FeedWordPress and delete all syndication_item_hash entries from the wp_postmeta table.

Now my web server should work much better!

Disable some useless stuff in Raspbian

I’m currently running three Raspberry Pi projects, with Raspbian as my choice of distribution. One of the machines will be co-located at FS Data and run ownCloud. The other will be handle switching a central heater between oil and electricyty via SMS. The third is a surveillance camera heavily inspired by Dummy camera made smart with Raspberry Pi by my talented colleague Tomas Bjerre.

While examining the default Raspbian image I noticed there were some things I will never use, so I disabled them to free up some RAM:

  1. Disable the consoles by putting a # in front of all lines containing /sbin/getty in /etc/inittab:

    #1:2345:respawn:/sbin/getty --noclear 38400 tty1
    #2:23:respawn:/sbin/getty 38400 tty2
    #3:23:respawn:/sbin/getty 38400 tty3
    #4:23:respawn:/sbin/getty 38400 tty4
    #5:23:respawn:/sbin/getty 38400 tty5
    #6:23:respawn:/sbin/getty 38400 tty6
    .
    .
    .
    #T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

    If you get into network issues and need the console you can always mount the SD card on another machine and edit the file there!

  2. If you don’t have a keyboard connected or otherwise don’t want global hotkeys, disable the triggerhappy global hotkey daemon:

    sudo update-rc.d triggerhappy disable

  3. If you have Transmission installed but is not using UPnP, disable the minissdpd daemon for keeping track of UPnP devices:

    sudo update-rc.d minissdpd disable

  4. If you run Apache, disable modules you don’t use. Maybe:

    sudo a2dismod dir
    sudo a2dismod cgi
    sudo a2dismod negotiation
    sudo a2dismod autoindex
    sudo a2dismod setenvif

David’s Software Development Blog