The Node

Archive for July, 2012

Finally, two of my favourite things are combining. Zombies and Linux. I talked about the possibility of Steam coming to Linux two years ago and finally it’s happening. And even better news? One of my favourite multiplayer games will be the first to get the Linux treatment – Left 4 Dead 2. Steam will only be available for Ubuntu 12.04 to start with, but Valve are stating that other distributions will also get the treatment eventually. I suspect this will include Fedora and Linux Mint (being based on Ubuntu anyway).
Speaking of zombies, I hope you’re all playing TellTale’s awesome “The Walking Dead” episodic game because it’s awesome. Each chapter will take you about two hours to get through and all the decisions you make carry on over into the next episode. It’s available on Steam, Xbox Marketplace and the PSN Store. Go kill some zombies!

After a couple of hours downtime, The Node is back running the ridiculously named “Beefy Miracle” release 17 of Fedora Linux. This wasn’t a smooth upgrade and I’ve upgraded via “preupgrade-cli” since Fedora 7.

Anyway this looks like a script writing bug basically and the solution is simple if you’re physically sitting in front of your server. The whole reason to use “preupgrade-cli” rather than the graphical “preupgrade” tends to imply no X session or an SSH terminal. Which has always worked in the past.

This is basically a foot-note to this preupgrade article which details how to upgrade Fedora Linux remotely using only preupgrade’s command line. Except, before you reboot to complete the upgrade you’ll see this:-

sh: /sbin/grub: No such file or directory
/bin/echo: write error: Broken pipe
All finished. The upgrade will begin when you reboot.

Usually after this point (minus the error) you type “reboot” and then lose your connection, wait a couple of hours and then start attempting to login again. When you can, you’ll find you’re back on Fedora 16 again. Huh.

People on forums were trying all sorts of things like reinstalling the preupgrade package and GRUB2 and such…none of which is necessary. Red herring!

After the normal preupgrade has done it’s thing and before you reboot as requested, open up your grub configuration file:-

vi /boot/grub2/grub.cfg

You’ll see the list of “menuentry” stanzas for each kernel, the top one of which is the upgrade, which is clearly labelled. This is what needs to be booted to without a user necessarily being at the console.

Above this chunk you’ll see this line:-

set default="1"

This is the bit that tells Fedora which kernel to load from “/boot” if no physical keyboard changes happen. They’re loaded in order starting from the top down, one stanza at a time. You’ll notice that the upgrade stanza is top of the list.

In GRUB, the stanzas are counted from 0 not 1. So the default kernel to load will always be the next one down, which is usually your last Fedora 16 kernel.

Told ya it was a simple scripting error :-)

Change the set default line to read: -

set default="0"

Save the file and now reboot. Fixed and your new Fedora release should begin installing. Give it a couple of hours and then retry an SSH session to that machine. Once in, check with: -


uname -r
cat /etc/fedora-release


Phew! I was worried with all the technical chatter on Bugzilla regarding this release. I was also worried about the reported move from the mess of directories under /sbin/lib /lib /sbin, /usr/sbin and so on…thinking this upgrade rather than a fresh install might knacker my system. But I worried needlessly, the upgrade script creates sym-links for backwards compatibility. So everything works again. Sorry for the downtime.

The SSH protocol is a wonderful tool for *nix hosts. You can use it to securely connect to any remote server and also do neat tricks with it. One of the best techniques you can employ with SSH is the concept of SSH tunnelling. This essentially means using the SSH encrypted protocol to connect any service to another machine which will actually handle the requests. This is especially useful when corporate environments place restrictive rules on the use of external ports such as 110 and 25 (SMTP,POP3 e-mail) or 80 (HTTP for general web use).

If you find yourself in a restrictive network environment but port 22 (SSH) is open you can use it to get around restrictions in other network services imposed by the firewall of that network. For example, your current network might disallow port 80 so you can’t use the web or perhaps restrict your connections against connecting to external e-mail servers. SSH tunnelling gets around this and as an added bonus, your traffic is encrypted :-)

So let’s say you have a home Linux server which has all the access you want – SSH server, perhaps e-mail and web access. Maybe even FTP. But you’re stuck in a network environment where only internal traffic is allowed on anything but SSH. While you only have SSH, this can still be your window on the world.

So, how to connect your sole SSH port to your home server in order to serve up the web in a network environment which explicitly denies it? Linux sneakery, that’s how :-)

For the sake of argument, let’s assume your home server has a static hostname – if it doesn’t, grab yourself a a static hostname for your residential broadband server. Let’s also assume you’ve got OpenSSH server running on port 22 (the default) on your home server and that you can successfully SSH into a bash shell remotely.

Now, let’s also assume you’re on a corporate network where the only access to the outside Internet is via SSH (port 22) to allow developers and administrators to connect to remote servers. No web, no email, nothing. The horror. So let’s get our home server in on the action. I will assume that:-

1. Your home Linux server can be accessed via an SSH session remotely and that your home firewall allows SSH terminal sessions to your home machine.
2. Your home server has a hostname of “myhomeserver.net”
3. Your home server has all the ports open for the services you want to use.

To initially illustrate this concept, let’s take the commonly used open source MySQL database server which usually listens on a default port of 3306. On my home network only other hosts on the same internal network can connect to MySQL as my router’s firewall rejects any attempt to connect on 3306 remotely. Conversely, my theoretical corporate network doesn’t allow any outgoing traffic on port 3306 anyway…only port 22 for SSH.

So, on the corporate network, doing the following normally won’t work:-

mysql -H myhomeserver.net -u[DB_USER] -p[DB_PASSWORD]

The “-H” switch specifies the host to connect to (“myhomeserver.net”), and [DB_USER] and [DB_PASSWORD] are your normal login credentials for MySQL. You’ll be unsurprised to learn that you can’t connect. Firstly because my home router rejects requests on port 3306 from the Internet and secondly because the corporate machine I’m trying to connect from doesn’t allow outgoing MySQL requests on port 3306 anyway.

So let’s build a simple SSH tunnel to get around these restrictions. Remember, SSH is enabled for external access on both the corporate firewall and my own home firewall. Port 3306 is not enabled on either.

ssh [USER]@myhomeserver -L 3306:127.0.0.1:3306 -N

The “-L” switch basically says for SSH to listen on the local corporate host (127.0.0.1) port 3306. The second colon is where which port to forward the port to on my home server (which happens to be the same port 3306). As I’ll be essentially logged in to my home server on SSH at this point, port 3306 is open to me as a local user.
Entering this command, you’ll be greeted with a request for your normal SSH user password. Once this is done, you’re logged in and SSH is ready to forward requests. Press CTRL-Z to put the SSH process in the background. This will stop the process briefly and return you to the command prompt. Restart the process as a background process by simply typing:-

bg

…and you should get a response something like:-

[1]+ ssh yourusername@myhomeserver.net -L 3306:127.0.0.1:3306 -N &

The ampersand (&) states that the process is now running in the background and you’ll be returned to the command prompt. Now let’s try our “mysql” command again, but point our host to connect to the local machine we’re on this time instead of directly trying to connect to myhomeserver.net.

mysql -h 127.0.0.1 -u[DB_USER] -p[DB_PASSWORD]

where [DB_USER] and [DB_PASSWORD] are the login credentials for your MySQL database. You’ll now be able to access MySQL. If you run “SHOW DATABASES”, you should get a list of databases on your home Linux server even though both firewalls explicitly disallow traffic on port 3306 externally. The local corporate MySQL client thought it was connecting to the localhost on port 3306 and so allowed it, whilst the home router thought that the MySQL traffic was coming in through SSH and so also allowed it.
What you’re essentially doing here is using the SSH protocol to “tunnel” other protocols through it…securely. This can basically be done for any protocol, very useful even only for the fact that you can use it to tunnel insecure protocols through a secure link provided by SSH.

Exit out of MySQL back to the command prompt on the corporate machine. Type:-

fg

…to bring the SSH process back into the foreground and then press CTRL-C to quit it and close the tunnel. That’s it for this time. In the second part of this series, I’ll look into using SOCKS for enabling web access through an SSH tunnel and how to use arbitrary ports for SSH when 22 isn’t available. Have fun :-)

Users typically don’t have access to each other’s directories, so sharing files between users who, for example, might be working on the same project ordinarily can’t be achieved. Here, I’ll show you how to set up a shared directory between two users.

For this, I’m going to create a directory that I’ll use for sharing called “/home/public”. So I’ll create this directory first as root.

mkdir /home/public

Next, I’ll add a user group that all users who want to access the “/home/public” directory have to belong to called “share”.

/usr/sbin/groupadd share

Next, I’ll change the user owner to root, but set the group owner of the directory to the above group “share”.

chown -R root.share /home/public

Next, I’ll add the first user I want to the “share” group (you can do this as many times as you like for different users).

/usr/bin/gpasswd -a user1 share

I’ll then switch the “/home/public” to full permissions for the group and owner with: -

chmod ug+rwx -R /home/public

Lastly, I’ll set the SGID bit on the shared directory. Normally whenever you create files in a directory, by default it belongs to the default group or user. When a file is created in a directory with the SGID bit set it belongs to the same group as the directory. The result being that all users of the “share” group can create/alter files in “/home/public” directory.

chmod g+s /home/public

If you now do a detailed list command on the “/home” directory using:-

ls -la

…you’ll get output similar to:-


drwxrwsr-x 2 user1 shared 4096 Jul 11 12:19 public


That ‘s’ listed under the file permissions means the SGID bit is set successfully.

You can verify that the users you want to be able to write to the public directory are added by listing the “/etc/group” file with: -

cat /etc/group

…which will show the users assigned to the group. For example: -

shared:x:1002:user1,user2,user3

That’s it :-)

Back in the good old days of Linux/UNIX, there was a file (usually) located under “/etc/rc.local” that would, after all run-level specific processes have been started, run whatever scripts were in the rc.local file. With Fedora Linux moving away from the more traditional “sysvinit” service manager, this rc.local file no longer exists for running whatever of your own bash scripts/commands you might want to run after the run-level specific processes have been started.

Luckily, you can re-enable this functionality :-) Obviously, you’ll need to be root for this so if you’re not already, elevate yourself to the root user:-

su - root

Then you need to re-create the rc.local file with:-


touch /etc/rc.local
chmod +x /etc/rc.local


Next, open the file for editing:-

vi /etc/rc.local

As this is basically a bash script itself, you need to include the bash interpreter as the first line of the file.

#!/bin/bash

You can now add whatever scripts or commands you like here – they will be run after everything else at your specific run-level has been started. In order for systemd to recognise and use this file, the systemd rc-local.service must be enabled.

systemctl enable rc-local.service

You can check the status of this service with:-

systemctl status rc-local.service

That’s all that’s required. Anything contained in this rc.local file will now be executed last-thing on reboot. Enjoy!

VirtualBox is a great piece of open source software that allows you to run a wide variety of operating systems as virtual machines on the host operating system. I’ve only ever run VirtualBox under Windows but developed a hankering to run it on my headless Fedora Linux home server. And apparently, this can be done! Here’s how.

First, you need to make sure you have the right package dependencies satisfied.

yum install binutils qt gcc make patch libgomp glibc-headers glibc-devel kernel-headers kernel-devel dkms

Okay.

This is geared towards Fedora Linux, but aside of the package manager specific stuff on installing VirtualBox, you should be able to install and run VirtualBox on any flavour of Linux if you know it’s package manager syntax.

yum install VirtualBox

If you’re using CentOS or some other RedHat-based distribution and the VirtualBox package cannot be found, you can install the repo file manually straight from VirtualBox itself.


cd /etc/yum.repos.d/
wget http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo
yum update
yum install VirtualBox-4.2


Right – that was easy. You will probably find that you also need the VirtualBox kernel module installed for this magic to happen. If you pre-loaded the module beforehand, no worries but like me, I’m guessing you haven’t :-)

/etc/sysconfig/modules/VirtualBox.modules

If this doesn’t work or you’re not using Fedora, you can try: -

/etc/init.d/vboxdrv setup

or..
service vboxdrv setup

The above will automatically create vboxusers group and user and also search and automatically rebuild required kernel modules. If the below build process fails, you will get a warning messages. Please have a look at “/var/log/vbox-install.log” to trace why the build process failed.

For this example, I’m going to install the 32-bit version of Ubuntu 10.10 Server with 1GB of RAM and a 10GB hard disk. The first step will be to download the ISO image from Ubuntu.com. So make a directory for your VirtualBox stuff and download the ISO file there.


mkdir ~/vbox
cd ~/vbox
wget http://old-releases.ubuntu.com/10.10/ubuntu-10.10-server-i386.iso


Now that you have the Ubuntu 10.10 Server installation CD file image, you can create your virtual machine and register it with VirtualBox. I’m using the VM name of “Ubuntu-10-10-Server-32″ but you can use whatever you like for the name.

VBoxManage createvm --name "Ubuntu-10-10-Server-32" --register

Next, we set up some basic parameters for the virtual machine. As stated previously, this VM is going to be assigned 1GB of RAM. We’re also going to set the virtual DVD drive as the first boot device so that we can link it to the ISO file and begin the installation when the virtual machine starts up. We also set the first network card to bridged networking so that it gets it’s own IP address separate from the host OS and link it to the “eth0″ network adapter. Bridged networking means that any virtual machine running will try to obtain an IP address from the same source your currently active default network address got its IP address. Hence the term bridged, as the two are connected. If you want to, you can assign it to “eth1″ or whatever is appropriate, but the default for Linux is “eth0″ normally.

VBoxManage modifyvm "Ubuntu-10-10-Server-32" --memory 1024 --acpi on --boot1 dvd --nic1 bridged --bridgeadapter1 eth0

The next step is to create a virtual hard disk and assign it a size. We want our VDI to be 10GB, so modify the registered VM accordingly. I’ve named the VDI file as the same as the name of the virtual machine we’ve created:-

VBoxManage createhd --filename Ubuntu-10-10-Server-32.vdi --size 10000

You also need to create a virtual IDE controller. You can obviously create a SATA controller instead if you prefer but for the purposes of this demonstration, we’ll use IDE.

VBoxManage storagectl "Ubuntu-10-10-Server-32" --name "IDE Controller" --add ide

he last step here is to attach the newly created IDE controller to the hard disk. We’ll use IDE port 0 for this.

VBoxManage storageattach "Ubuntu-10-10-Server-32" --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium Ubuntu-10-10-Server-32.vdi

We’ll also use the IDE controller to link the DVD drive to our ISO image file so that the operating system installation can initialise.

VBoxManage storageattach "Ubuntu-10-10-Server-32" --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium ubuntu-10.10-desktop-i386.iso

We’re also going to set some miscellaneous virtual machine settings like the the amount of video RAM available to the graphics card – in this case, 256MB – and setting 3D acceleration and audio controller drivers.

VBoxManage modifyvm "Ubuntu-10-10-Server-32" --vram 256 --accelerate3d on --audio alsa --audiocontroller ac97

One final thing to do is to install the VirtualBox Extension pack – it’s on the download page. Make sure you download the version appropriate for your version of VirtualBox. Oracle has modularised some functionality of VirtualBox since version 4.0 and the extension pack is required for remote desktop connection support (VRDP) which we’ll need in order to complete the installation of the virtual operating system.

You’ll need to be root for this, so:-


su - root
wget http://download.virtualbox.org/virtualbox/4.2.12/Oracle_VM_VirtualBox_Extension_Pack-4.2.12-84980.vbox-extpack


Once you’ve downloaded the extension pack, install it with the following: -


VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.2.12-84980.vbox-extpack


Now you can drop back down to a normal user by typing “exit”. Aside from installing the extension pack, everything else can be achieved as a normal Linux user. Now that the extension pack is installed, you can set up the virtual remote desktop connection. Here, I’m using a non-standard port of 45005, but if you’re happy to use the default port (3389) you can omit the “vrdeport” parameter. Just make sure you have set your Fedora host OS’s firewall to accept the connection.

VBoxManage modifyvm "Ubuntu-10-10-Server-32" --vrde on --vrdeport 45005

Now you can fire up your virtual machine to continue the OS installation. You can either use the Screen utility to manage your terminal sessions or (as I have done below) load the VirtualBox process into background with “nohup” and the “&” on the end.

nohup VBoxHeadless --startvm "Ubuntu-10-10-Server-32" &

Now you can either use rdesktop to connection via RDP or use Windows’ Remote Desktop Connection client. Either way, if you’re on a non-standard port you’d use: -

rdesktop [YOUR_HOST_IP_ADDRESS]:[PORT]

This will connect you to your VM via the host IP address on which you’re running VirtualBox to continue the installation. If, upon boot, you get the following error (as I did):-

piix4_smbus 0000.00.07.0: SMBus base address uninitialized - upgrade bios or use force_addr=0xaddr

Don’t worry about this for now. It’s an annoying warning but doesn’t affect anything at this point. Once the virtual machine OS is installed, we can fix this easily. Go through your usual Ubuntu installation procedure and then once the login prompt appears, switch windows back to your host Fedora bash prompt and power down the VM with: -

VBoxManage controlvm "Ubuntu-10-10-Server-32" poweroff

We’re powering off the VM because we need to detach the ISO installation file for this Ubuntu VM, else every time we fire up the virtual machine, it’ll load the ISO installation media again and now Ubuntu is installed we want to boot straight into the VM. You can find out the information on which IDE (in this case – it could be SATA) controllers are currently attached to the VM by running: -

VBoxManage showvminfo "Ubuntu-10-10-Server-32"

This will display a bunch of information about the current state of the VM. The line you’re looking for looks something like: -


Storage Controller Name (0): IDE Controller
Storage Controller Type (0): PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0): 2
Storage Controller Port Count (0): 2
Storage Controller Bootable (0): on
IDE Controller (0, 0): /home/vboxuser/vbox/Ubuntu-10-10-Server-32.vdi (UUID: 3e097b88-5feb-45c7-9e63-2b7a6610dd4f)
IDE Controller (1, 0): /home/vboxuser/vbox/ubuntu-10.10-server-i386.iso (UUID: 51c9864e-0427-492e-871e-a93109fd0d86)


You can now detach this device with: -

VBoxManage storageattach "Ubuntu-10-10-Server-32" --storagectl "IDE Controller" --port 1 --device 0 --medium emptydrive

If you run “vminfo” again, you’ll see the virtual DVD drive is now empty.


Storage Controller Name (0): IDE Controller
Storage Controller Type (0): PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0): 2
Storage Controller Port Count (0): 2
Storage Controller Bootable (0): on
IDE Controller (0, 0): /home/vboxuser/vbox/Ubuntu-10-10-Server-32.vdi (UUID: 3e097b88-5feb-45c7-9e63-2b7a6610dd4f)
IDE Controller (1, 0): Empty


Okay, now that’s taken care of, fire up the virtual machine again and use rdesktop or Microsoft’s RDP utility to log back into your VM. If you want to make things easier for yourself at this point you could set up the SSH server and log straight into your VM via your Fedora command line SSH client. Either way, you should now have shell access to your Ubuntu 10.10 virtual server. Once you’ve logged in, you can find out your VM’s IP address in the usual fashion with:-

ifconfig

This is far from ideal, but I’ve not found any way to get the bridged network IP address from the host OS command line as of yet. A call to “vminfo” fails to show the IP address of the running guest VM. If anybody knows how to do this, please leave a comment!

If you’ve not got that smbus base address error, you’re done – congratulations! Enjoy your Ubuntu virtual machine running in Fedora :-)

Now to fix that annoying “piix4_smbus 0000.00.07.0: SMBus base address uninitialized – upgrade bios or use force_addr=0xaddr” error (assuming this is a problem for you). It’s only a warning and doesn’t actually affect anything, but it would be nice not to have it. This error is caused by the Ubuntu VM having no smbus but Ubuntu always trying to load the kernel module anyway. So let’s disable that. First, check that the module is being loaded with:-

lsmod | grep i2c_piix4

If so, you need to disable it. Open the following file in “vi” or whatever your favourite text editor is:-

vi /etc/modprobe.d/blacklist.conf

and add the line: -

blacklist i2c_piix4

Now reinitialise the RAMDisk with:-

update-initramfs -u -k all

Done. No more error.

On your Internet travels, you may have come across CD image files with a .bin or .cue extension. You might even hypothetically want to mount these files under Linux and read their contents. It’s very easy once you know how and here is how you do it.

The utility that makes this possible is called bchunk. Your friendly Linux software repository will no doubt have it. For Red Hat-based systems this means running: -

yum install bchunk

and for Debian-based systems, this means running: -

sudo apt-get install bchunk

BIN and CUE files are extensively used as CD file image formats in burning software under Windows but is less supported on other platforms where the now more common ISO file format is used. The difference between the two is largely irrelevant to this topic but suffice to say the entirety consists of the BIN file which holds the actual data and a text CUE file which acts as an index of the data. To convert the files to ISO, simply run: -

bchunk yourfile.bin yourfile.cue yourfile.iso

…where the first parameter is the BIN file, the second is it’s corresponding CUE file and the last is the output ISO file.

For mounting this resulting ISO file on to your Linux file-system, create a mount point directory for it and run: -

mount -o loop -t iso9660 yourfile.iso /mnt/yourmountpoint

That’s it! Simple but useful.