Speeding Up Performance on a RetroPie Setup

I was desperately trying to get hold of a NES Mini Classic console before Christmas but because Nintendo are hopeless they were out of stock except from third-party sellers on Amazon and eBay selling with with 1000% markup on price. I kinda wanted one because it would look neat next to my TV and also I could introduce the “good old days” to my little boy. But no, they were sold out everywhere, naturally.
Then I thought, “what the hell am I doing?” and bought a Raspberry Pi 3 from Amazon and downloaded RetroPie to an SD card. RetroPie is the regular Raspbian/Pixel Linux OS for Raspberry Pi with a huge number of retro emulators built in, all wrapped in a nice GUI. All you have to do is setup your USB controller mappings and point it at some game ROM files and presto…your very own retro games console.
The benefits were obvious to me – you can play nearly every game from for any system prior to 1995, including NES, arcades, Sega Megadrive (Genesis in the colonies), Master-System, PC Engine, Neo Geo, SNES etc, you can choose your controller (I went for PS3-alike controllers) myself) and it’s very customisable with menus and themes and such. You can even (with a little programming magic) add things like background music to the menus and things like that.
There are *plenty* of guides on how to setup RetroPie on a Raspberry Pi, like here, here and here so I won’t repeat the information here. Suffice to say, it’s pretty easy once you’ve imaged the SD card properly.
The problem I was having is that even on a Raspberry Pi 3, some of the emulators (like the SNES one for some reason) would be very slow with lagging sound and sinking frame-rates. Some emulators, of course, like the Sega Dreamcast and N64 emulators won’t run well on even a Pi 3 – they simply require too much power to run well on any Pi – but the SNES one should run fine.
I investigated overclocking as a potential solution, but the Pi 3 doesn’t officially support overclocking yet although it can be done (use a heatsink!). The menu option which used to be there to do so for the Pi 1 & 2 is missing on the Pi 3. The most likely cause is a configuration issue. So I tried various things in the /opt/retropie/configs/all/retroarch.cfg config file like trying various fixes for sound issues and a few other tips and tricks to try to speed up emulation with varying degrees of success depending on the emulator.
The problem ended up being the “scaling governor” for the CPU on the Pi which is somehow stuck to powersave mode meaning that it was taking too long to process. The fix is to set each CPU core scaling governor to “ondemand” rather than “powersave”. You can do this by adding the following lines to your “/etc/rc.local” file which (if you know your Linux) is run at startup.

echo ondemand > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo ondemand > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo ondemand > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo ondemand > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

This fixed all my Raspberry Pi 3 emulation performance issues, at least with emulators that technically should run well on the Raspberry Pi 3. I hope you found this useful. If you are still experiencing performance issues, especially on a Pi3, try some of the other fixes above. Overclocking is an option but should probably be a last resort.

, , ,

The Node is now HTTPS only

, ,

How I Automated My ExpressVPN Connection Under Ubuntu

I’ve just signed up for a VPN service. I decided to go with ExpressVPN as they have clients for pretty much every device/OS out there and also support OpenVPN. They’re not the cheapest my any means but their Twitter feed shows they’re passionate for digital privacy so up I signed.

Unlike the Windows client, the Linux client is a command line tool, which is actually great for Linux folks because you can then script it. Once you’ve activated your client on the machine you want to use it on, the usage is very simple: –

expressvpn connect smart         <-- Connects to the "smart" location which offers best speed for your country location. 
expressvpn disconnect            <-- Disconnects the VPN, obviously :-)
expressvpn refresh               <-- Refreshes the list of VPN hosts in various countries
expressvpn list                  <-- List current hosts in various countries
expressvpn connect {alias}       <-- Connection to a specific country from the list
expressvpn status                <-- Shows if VPN connected and if so, which country.

If you run “expressvpn list” you’ll get a list similar to this: –

-----	---------------			------------------------------	-----------
smart	Smart Location			UK - London			Y
smi					USA - Miami			
usta1					USA - Tampa - 1			
uslv					USA - Las Vegas			
usla3					USA - Los Angeles - 3		
frpa1	France (FR)			France - Paris - 1		Y
frpa2					France - Paris - 2		
defr1	Germany (DE)			Germany - Frankfurt - 1		Y
defr2					Germany - Frankfurt - 2		
dedu					Germany - Dusseldorf		
deda					Germany - Darmstadt		
ch1	Switzerland (CH)		Switzerland			Y
itmi	Italy (IT)			Italy - Milan			Y
itco					Italy - Cosenza			
es1	Spain (ES)			Spain				Y
camo2	Canada (CA)			Canada - Montreal - 2		
cato					Canada - Toronto		
cato2					Canada - Toronto - 2		
cava					Canada - Vancouver		
se1	Sweden (SE)			Sweden				
ie1	Ireland (IE)			Ireland				
is1	Iceland (IS)			Iceland				
no1	Norway (NO)			Norway				
dk1	Denmark (DK)			Denmark				
be1	Belgium (BE)			Belgium				
fi1	Finland (FI)			Finland			

When you list the countries that you can connect to, there is a “Y” next to some of the items. This means these countries/locations are “recommended” for best speed. They differ from the “smart” connection option in that the smart connection will always be the country you reside in, although a different IP address.
I thought it would be good so that when I login to my Ubuntu machine, it connects to a randomly selected VPN country which has the “recommended” “Y” next to it. Since I don’t use Unity under Ubuntu but Cinnamon (much better, IMHO) I’d also like to display the country I’m connected to as a Desklet in my top desktop panel.

Here is the bash script which does this on boot.

expressvpn refresh
expressvpn list | tail -n+3 | column -ts $'t' | sed 's/   */:/g' > data.tmp
cat data.tmp | while read LINE
        if [ `echo $LINE | tail -c 2` == 'Y' ]; then
                TEMP=`echo $LINE | grep : | awk -F: '{print $1}'`
                echo "$TEMP" >> array.tmp
ARRAY_SIZE=`wc -l array.tmp | awk {'print $1'}`
RND_VPN=`shuf -i 1-$ARRAY_SIZE -n 1`
VPN=`sed "${RND_VPN}q;d" array.tmp`
rm -f data.tmp
rm -f array.tmp
expressvpn connect $VPN

It’s pretty easy to understand. First, I refresh the ExpressVPN country/location list. Then I list each location into a colon separated file called data.tmp. Next, I iterate through each line in that file and add each country which has a “recommended” “Y” next to it and put those in a file called array.tmp. Next, I randomly choose a country location from that list using the shuf command and extract the alias code into a variable called RND_VPN. This is what the ExpressVPN client uses in conjunction with the connect parameter.

The last bit is showing this information in Cinnamon as an applet. I don’t know much about writing JavaScript applets apart from they’re written in JavaScript as the Cinnamon documentation for this is super-weak, but I found a simple network status applet which is basically a refreshing label and that will do. All this does is put the output from the “expressvpn status” command into a label and displays it. Simple applets are composed from two files, the JavaScript JS file and a JSON descriptor file. Here are my horrible hacked together versions.


const Applet = imports.ui.applet;
const Lang = imports.lang;
const Mainloop = imports.mainloop;
const GLib = imports.gi.GLib;
const UUID = "expressvpn@jon";

function log(message) {
    global.log(UUID + '#' + log.caller.name + ': ' + message);

function logError(error) {
    global.logError(UUID + '#' + logError.caller.name + ': ' + error);

function MyApplet(orientation) {

MyApplet.prototype = {
    __proto__: Applet.TextApplet.prototype,
    _init: function (orientation) {
        Applet.TextIconApplet.prototype._init.call(this, orientation);
        try {
            this.set_applet_tooltip(_("ExpressVPN Node location."));
        catch (error) {

    refreshLocation: function refreshLocation() {
        let [res, out] = GLib.spawn_command_line_sync(" expressvpn status ")
        let out = String(out).replace("n", "");
        let out = String(out).trim();
        //let out = String(out).replace(/ /g, "-");
        //let out = String(out).replace(/   /g, " ");
        this.set_applet_label('VPN: ' + out + '');
        Mainloop.timeout_add_seconds(REFRESH_INTERVAL, Lang.bind(this, function refreshTimeout() {

function main(metadata, orientation) {
    let myapplet = new MyApplet(orientation);
    return myapplet;


    "dangerous": true,
    "description": "Shows ExpressVPN Location",
    "name": "ExpressVPNStatus",
    "uuid": "expressvpn@jon"

Both these files go in the directory “$HOME/.local/share/cinnamon/applets/expressvpn@jon” as the directory is “applet_name@author” format. You can then add the applet as normal in Cinnamon to a panel.

I suspect something similar could be done with other VPN clients if you wanted :-)

, , , , ,

Create an Encrypted Disk Partition or FIle Container With Ubuntu

It can sometimes be useful to have an encrypted partition to keep your stuff secret from prying eyes. Most tutorials will tell you to encrypt your home partition and have it open at boot time. But this means that the encrypted partition is un-encrypted as long as your machine is switched on and you are logged in.
Here, I’ll show you how to set up a free partition or disk as an “on demand” encrypted partition. Meaning that we’ll write a couple of shell scripts to open and close the protected partition so that the partition is only open when you need it. It also has the added benefit on being mostly invisible to a cursory glance at the system. Unless somebody thinks to check the contents of /dev, your encrypted partition will not be visible because it’s not always mounted.

I will explain how to encrypt your partition using Linux Unified Key Setup (LUKS) on Ubuntu Linux. Ubuntu uses the AES-256 cipher to encrypt the disk volume, so it’s pretty secure. LUKS is the basic standard for disk encryption for Linux and is available for all Linux systems running a 2.6+ kernel…which these days is pretty much every system.

For this, you will either need a second hard disk you want to use as an encrypted drive – meaning the partition takes up the whole disk or an empty partition on your primary disk, assuming you didn’t use the entire disk when you set up your Ubuntu system. If you don’t have a spare disk or partition available, don’t worry – LUKS is flexible enough that you can use a file container to act as an encrypted storage area. If you’re using a partition, I’ll assume it’s called /dev/sdb1 (first primary partition on the second hard disk). Yours can be anything, obviously.

First you will need cryptsetup, a utility for setting up encrypted file-systems using Device Mapper and the dm-crypt kernel module. You can install it with: –

sudo apt-get install cryptsetup

Once this is installed, you can use it to set up the initial partition. Remember, we’re using /dev/sdb1 (first partition on the second disk) but you can use anything.

sudo cryptsetup -y -v luksFormat /dev/sdb1

This will warn you that this will wipe the data on the partition, so make sure you’ve passed it the right one :-) Here is also where you’ll set the password/passphrase for the partition, so don’t forget it!

This will overwrite data on /dev/sdb1 irrevocably.
Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:
Command successful.

Okay, now the partition needs to be opened by the encryption manager. Do this with: –

sudo cryptsetup luksOpen /dev/sdb1 encrypted

You will be prompted for your password you created when you created the encrypted partition. The “encrypted” string here is arbitrary – it’ll just used as a label for the device mapper. You can call this anything. I’ve called mine “encrypted” for clarity. You’ll see this if you issue a “ls /dev/mapper” command. The new encrypted partiton is listed as “/dev/mapper/encrypted”.

Next, we’re going to make sure the partition is securely clean by using the dd command to zero out all the bits.

sudo dd if=/dev/zero of=/dev/mapper/encrypted bs=128M

The partition is now available but it still needs to be formatted with the normal file-system tools in order to be usable. You can format it in the usual manner.

sudo mkfs.ext4 /dev/mapper/encrypted

Now you can start using it! Here is the shell script to open the partition. I’m mounting my encrypted partition on $HOME/encrypted but you can change that part of the script to anything you like.


sudo cryptsetup luksOpen /dev/sdb1 encrypted
# Comment out the above and uncomment the below if
# you want to use a file container instead of a partition
# See further down for file containers example.
# sudo cryptsetup luksOpen $HOME/encrypted-container encrypted
sudo mount /dev/mapper/encrypted $HOME/encrypted
df -h | grep encrypted
mount | grep encrypted

and the shell script to close the partition again…


sudo umount $HOME/encrypted
sudo cryptsetup luksClose encrypted
df -h

That’s it. Now here’s how to create a file container instead of a partition instead if you don’t have any easy way to create or use another partition. To store our encrypted data this way, we’ll need to create a file which will act as our storage device instead of the partition.

We want to create an empty file, but we can’t have that be a sparse “empty” file, because it doesn’t actually allocate the full file size when it is created which is no good for us. There are two ways to do this.

The easiest and the quickest way of going about this operation is with the “fallocate” command. This instantly allocates the amount of disk you would like for a file and assigns it the filename you give it. For this example we’ll create a 2GB file for our encrypted storage container.

fallocate -l 2G $HOME/encrypted-container

So this will create a 2GB file under your $HOME directory called “encrypted-container”. This disadvantage of this method is that it will not overwrite whatever old, deleted data that used to be used by those blocks with zeros or random data. This probably will not be desirable for your purposes because we don’t want people to be able to tell which portion of the file has encrypted data written to it.

A better but much slower way is to use the “dd” command in a similar manner as we used on the partition above. It will overwrite the blocks with zeroes at the file container location, giving us a zero’ed file of the proper size we want – in this case 2GB (count=2000).

sudo dd if=/dev/zero of=$HOME/encrypted-container bs=1M count=2000

Now we can use this file container and set up LUKS for it.

sudo cryptsetup -y luksFormat $HOME/encrypted-container

As with setting up a partition in the same way, you’ll be warned that it will wipe the data in the file and ask you to set the password/passphrase for the encrypted container.

Once this is done, you can check out the file and see that it’s now a LUKS encrypted file.

file $HOME/encrypted-container
encrypted-container: LUKS encrypted file, ver 1 [aes, cbc-essiv:sha256, sha1] UUID: 0451db36-3423-4ee1-8e2e-cc34c49e05f3

Now you can open the encrypted container using your password.

sudo cryptsetup luksOpen $HOME/encrypted-container encrypted

This is much the same as the process for opening a partition above. We’re using the same mapping name of “encrypted”.

As before the container still needs to be formatted as a regular EXT4 “partition” (even though it’s a file). Do so with: –

sudo mkfs.ext4 -j /dev/mapper/encrypted

Now you can use the same shell scripts as listed above as long as you change the parameters passed to the luksOpen command from the partition to the path to the file container. the rest is the same because really the only difference is how the /dev/mapper/encrypted device is created via luksOpen. Once you have that, be it from a file container or a partition, the rest of the process is identical.

, , , ,

Repair a broken Ext4 Partition Superblock

In Linux, the entire disk space of a partition is subdivided into multiple file system blocks. The blocks are used for two different purposes. Most blocks stores user data or normal files. Some blocks in every file system store the file-system’s metadata. Metadata describes the structure of the file system. The most common metadata structures are superblocks, inodes and directories. Each file-system has a superblock, which contains information about the file-system such as file-system type (ext2, ext4, etc), size of the partition and it’s mount status amongst other things. If this information is lost, you are in trouble (data loss!) so Linux maintains multiple redundant copies of the superblock in every file system. This is very important in many emergency situations, for example you can use backup copies to restore damaged primary superblocks.

For this example, let’s assume your secondary drive’s first partition is corrupt (/dev/sdb1). If your primary root file-system is corrupt, you’ll need to boot your system from a live DVD/CD and repair it from the live OS using the root user account or “sudo [command]” on Ubuntu.

So if you see an error like the below when attempting to mount a file-system: –

/dev/sdb1: Input/output error
mount: /dev/sdb1: can’t read superblock

…your superblock is corrupt and the partition file-system is not accessible. You can restore the superblock from a backup but unless you’ve checked obvious things like SATA cables, your hard disk is probably on the way out and should be replaced as soon as possible, even if you restore the superblock from a backup on the partition.

Anyway, first make sure your partition is UNMOUNTED (umount /mountpoint). I cannot stress this enough. If you attempt to fix the partition whilst it is mounted, you will corrupt the drive even further.

You can try to run an initial file-system check using the “fsck” command.

fsck.ext4 -v /dev/sdb1

This will probably return something like: –

fsck /dev/sdb1
fsck 1.41.4 (27-Jan-2009)
e2fsck 1.41.4 (27-Jan-2009)
fsck.ext4: Group descriptors look bad... trying backup blocks...
fsck.ext4: Bad magic number in super-block while trying to open /dev/sdb1
The superblock could not be read or does not describe a correct ext4
filesystem.  If the device is valid and it really contains an ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>

Next, recover the list of backup superblocks from the partition like so: –

dumpe2fs /dev/sdb1 | grep superblock

This will produce a list of alternate superblocks you can use.

Primary superblock at 0, Group descriptors at 1-6
Backup superblock at 32768, Group descriptors at 32769-32774
Backup superblock at 98304, Group descriptors at 98305-98310
Backup superblock at 163840, Group descriptors at 163841-163846
Backup superblock at 229376, Group descriptors at 229377-229382
Backup superblock at 294912, Group descriptors at 294913-294918
Backup superblock at 819200, Group descriptors at 819201-819206
Backup superblock at 884736, Group descriptors at 884737-884742
Backup superblock at 1605632, Group descriptors at 1605633-1605638
Backup superblock at 2654208, Group descriptors at 2654209-2654214
Backup superblock at 4096000, Group descriptors at 4096001-4096006
Backup superblock at 7962624, Group descriptors at 7962625-7962630
Backup superblock at 11239424, Group descriptors at 11239425-11239430
Backup superblock at 20480000, Group descriptors at 20480001-20480006
Backup superblock at 23887872, Group descriptors at 23887873-23887878

Now you can use a alternate superblock and attempt to repair the file-system.

fsck -y -b 32768 /dev/sdb1

This will produce output similar to the below: –

fsck 1.40.2 (12-Jul-2007)
e2fsck 1.40.2 (12-Jul-2007)
/dev/sdb1 was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #241 (32254, counted=32253).
Fix? yes
Free blocks count wrong for group #362 (32254, counted=32248).
Fix? yes
Free blocks count wrong for group #368 (32254, counted=27774).
Fix? yes
/dev/sdb1: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb1: 59586/30539776 files (0.6% non-contiguous), 3604682/61059048 blocks

You should now be able to mount the file-system as normal (or reboot if it’s the primary root file-system): –

mount /dev/sdb1 $HOME/mount

Here, I’m mounting the file-system on the mount subdirectory of my user’s (/root in this case) home directory. If this doesn’t work, run through the fsck command above trying each backup superblock number in turn until you find one that works. Once you can successfully mount the file-system at a directory mount point, you can access your files.

Now would be the time to backup those files before the disk fails completely. Sometimes superblocks get corrupted and the disk will be fine for a while longer, but I take no chances :-)

, , , ,

How To Fix Ubuntu Ethernet Connectivity After Update

Sometime last weekend after doing a regular Ubuntu update, I lost my network connectivity. After doing an “ifconfig”, I noticed I had no assigned IP address from my router. Wonderful. Here’s how I fixed it – it’s actually very simple.
I’m on Ubuntu 14.04 LTS at present (at least until the 16.04 LTS point release becomes available).

First, make sure that your ethernet card is actually still recognised by the system, as in it’s still “seen” by the operating system. I list my device settings for eth0 (my NIC card).

ethtool eth0

This will give you something back like the following. Basically, if you get output back, your card is recognised by the system. If you get no output back, probably your card is broken. I’ve never seen a working ethernet device not return any data with this command, regardless if it’s been configured or not.

Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Link detected: yes

So, your card is still recognised by Ubuntu, just not configured. In my case, an update had somehow deleted the network settings for this device. This is no problem, you can recreate them. Edit the following file : –

sudo vi /etc/network/interfaces

You’ll probably see something like the following: –

auto lo
iface lo inet loopback

This means that only the loopback ( network interface is configured. This was the problem in my case too. While I assign a static IP address to my Ubuntu machine, it’s static on the router, not the device itself. Meaning that Ubuntu is using DHCP on the network but my router always hands it the same IP address. If you’re using DHCP to get your router to assign the IP address, the settings you need to add are: –

auto eth0 
iface eth0 inet dhcp 

…for device eth0. If you’re using a “proper” static IP address, then the settings will be something like: –

iface eth0 inet static

Change your “address”, netmask and gateway settings as appropriate. Save this file and restart networking with: –

sudo /etc/init.d/network restart

If you now type the “ifconfig” command, you should have networking again and be assigned an IP address.

, , , ,

Making a RAID-1 Mirrored Disk Array Under CentOS

RAID (Redundant Array of Inexpensive Disks) is a good way to ensure data integrity. In my experience, RAID-1 is most often used. This is two or more disks which are treated as one device. As data is written to one drive, it is also mirrored to the other(s). If one of the disks fails, you can simply replace the failed disk in the array and rebuild the data from the surviving drives – a good way to ensure your data remains intact after hardware failure. If you want to see what other types of RAID configuration are available, you can look here.
In this article, I’m going to create a software-based RAID-1 solution. As discussed above, this is two or more drives or partitions which together form a mirrored array which appears to the operating system as one device.

For this, I’m actually using CentOS 6.7 under a VirtualBox VM, although if you have the correct hardware available you can use a physical system :-). This is useful as you can create as many drives as you like of any size and can create or delete them at will from the VM settings/storage section. I’ve created two extra “disks” for the CentOS system (actually VDI files which act as drives under the VM) each of 2GB in size as this is only a demonstration. Both of these disks will form a RAID-1 device of 2GB in size with one drive mirrored to the other. The drives will be /dev/sdb1 and /dev/sdc1. Using fdisk, I have partitioned both drives to type 82 which is the Linux default partition type – ext4 in this case. Both drives have one primary partition each that takes up the entire 2GB space on the drive.

/dev/sdb1 - Disk 1 of the array. 2GB
/dev/sdc1 - Disk 2 of the array. 2GB

First, make sure you have the RAID management tool installed. This is usually installed by default, but it’s good to make sure.

yum install mdadm

mdadm has 7 major modes of operation. Normal operation just uses the ‘Create’, ‘Assemble’ and ‘Monitor’ commands – the rest come in handy when you’re messing with your array; typically fixing it or changing it.

1. Create
Create a new array with per-device superblocks (normal creation).

2. Assemble
Assemble the parts of a previously created array into an active array. Components can be explicitly given or can be searched for. mdadm checks that the components do form a bona fide array, and can, on request, fiddle superblock information so as to assemble a faulty array. Typically you do this in the init scripts after rebooting.

3. Follow or Monitor
Monitor one or more md devices and act on any state changes. This is only meaningful for RAID-1, 4, 5, 6, 10 or multi-path arrays as only these have interesting state. RAID-0 or linear never have missing, spare, or failed drives, so there is nothing to monitor. Typically you do this after rebooting too.

4. Build
Build an array that doesn’t have per-device superblocks. For these sorts of arrays, mdadm cannot differentiate between initial creation and subsequent assembly of an array. It also cannot perform any checks that appropriate devices have been requested. Because of this, the Build mode should only be used together with a complete understanding of what you are doing.

5. Grow
Grow, shrink or otherwise reshape an array in some way. Currently supported growth options including changing the active size of component devices in RAID level 1/4/5/6 and changing the number of active devices in RAID-1.

6. Manage
This is for doing things to specific components of an array such as adding new spares and removing faulty devices.

7. Misc
This is an ‘everything else’ mode that supports operations on active arrays, operations on component devices such as erasing old superblocks, and information gathering operations.

So, first we need to create the array from the partitions we’ve created.

mdadm --create --verbose /dev/md0 --level=mirror --raid-devices=2 /dev/sdb1 /dev/sdc1

This creates the RAID device /dev/md0 from the /dev/sdb1 and /dev/sdc1 partitions in mirrored RAID-1 mode. You can check the RAID status by listing the output from /proc/mdstat like so: –

cat /proc/mdstat

This should give you something like: –

Personalities : [raid1] 
md0 : active raid1 sdc1[1] sdb1[0]
      2095424 blocks super 1.2 [2/2] [UU]
unused devices: < none >

After you’ve created your array, it’s important to save the configuration in the proper mdadm configuration file. Under CentOS/RHEL this is usually stored directly under /etc. Save the configuration like: –

mdadm --detail --scan >> /etc/mdadm.conf

Now you can format the array like you would for any normal block device as you would directly after partitioning.

mkfs.ext4 /dev/md0

Once this is done, you can mount it as you would a normal drive. I’ll mount it under /mnt/RAID.

mkdir /mnt/RAID
mount /dev/md0 /mnt/RAID

If you now do a “df -h” command, you’ll see the RAID available as below.

Filesystem      Size  Used Avail Use% Mounted on
/dev/md0        2.0G  3.0M  1.9G   1% /mnt/RAID

Now you can write to it like any other partition. Under the hood, any data you write is being automatically mirrored to the other partition.

If you wish to stop the array for any reason, you can issue the following command: –

mdadm --stop /dev/md0

To start it again, you can use: –

mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1

Even easier if you have a simple setup is simply: –

mdadm --assemble --scan

Now we come to the meat of the subject…what to do when one of the disks in your array fails. Common sense dictates that if you have a failing drive, you’ll see lots of error messages in your systems logs “/var/log/messages” or “/var/log/syslog”. You can also check the kernel process file directly with: –

cat /proc/mdstat

If the RAID is fine, the line will read: –

2095424 blocks super 1.2 [2/2] [UU]

Here you can see that both drives in the array are “UP” ([UU]). If one of them was failing, you’d see that only one of the drives would be in the “UP” state so the string would read “[U_]” instead. The U’s are sequential, so the first “U” is /dev/sdb1 and the second is /dev/sdc1. If the string reads [U_], this means that /dev/sdc1 is failing and needs to be replaced.

So, let’s pretend that /dev/sdc1 has failed. First, you need to inform the array that the disk has failed. Do so with: –

mdadm --manage /dev/md0 --fail /dev/sdc1

This will produce the following result to tell us that the RAID manager utility has successfully marked the drive as failed.

mdadm: set /dev/sdc1 faulty in /dev/md0

You can double check this by running the “cat /proc/mdstat” command again. It will return the following to confirm the drive has been marked as failed.

Personalities : [raid1] 
md0 : active raid1 sdc1[1](F) sdb1[0]
      2095424 blocks super 1.2 [2/1] [U_]
unused devices: < none >

Note the “[U_]” string as mentioned above. Next, you need to remove the failed drive from the array with: –

mdadm --manage /dev/md0 --remove /dev/sdc1

The RAID manager will come back with: –

mdadm: hot removed /dev/sdc1 from /dev/md0

Now, you need to power down the system using “shutown -h now” or “poweroff” or similar. You’ll now need to physically replace the faulty drive. Since I’m using a VM, this is just a case of creating a new VDI virtual drive :-)

Once you’ve put in a new drive which is our new “/dev/sdc” you need to make sure the partition layout is identical to the first drive. You can easily do this with: –

sfdisk -d /dev/sdb | sfdisk /dev/sdc

This copies the partition layout from /dev/sdb to /dev/sdc. This will produce a bunch of output, but essentially saying the copy was a success. You can check this with “fdisk -l” if you like.

Now you can re-add the /dev/sdc1 partition to the RAID array again.

mdadm --manage /dev/md0 --add /dev/sdc1

Depending on the amount of data in the array, the synchronisation of the drives in the array might take a while. You can check it’s progress by running: –

cat /proc/mdstat

…to check it’s progress. The output will be something like…

md0 : active raid1 sdb1[0] sdc1[1]
      24418688 blocks [2/1] [U_]
      [=>...................]  recovery =  6.4% (1572096/24418688) finish=1.9min speed=196512K/sec

unused devices: < none >

Once the array rebuild has completed, the output from “cat /proc/mdstat” should be back the way it was before the drive failed. something like: –

md0 : active raid1 sdb1[0] sdc1[1]
      24418688 blocks [2/2] [UU]

unused devices: < none >

That’s it – you’ve successfully replaced a failed drive in a RAID-1 array :-)

, , , , ,