Archive for August, 2011
Mostly because I’ve rediscovered gaming. I downloaded Mass Effect from Games on Demand and thoroughly enjoyed it from start to finish. I also revisited Borderlands after I got stuck around level 5. I’m now a level 29 Rifleman :-) Borderlands has no real overall plot to speak of but it’s one of the purest videogames on the market. First person shooter looter with RPG elements would be a good description. The downloadable content such as The Zombie Island of Doctor Ned is also of a consistently good quality. Recommended for both single player and co-op. If anyone wants to help me out on quests, my gamertag is Jon256. See you in the badlands…
Here we’ll be focusing a little bit on kernel updates and compiling your own kernel. I’ll not go into this very deeply as it’s obviously a huge subject, but I’ll touch on the basics of how it’s done. Besides, pretty much all Linux users should be at least aware of the process of compiling your own kernel even if you never have the need or desire. In this part, I’ll be concentrating on compiling a generic mainline kernel from kernel.org. I’ll cover distribution-specific kernel hacking in parts two and three. While I’ll be using RedHat Enterprise Linux 6 for this article, it shouldn’t be specific to RedHat-based systems. I’ll cover specific distribution kernels later.
In all likelihood you’re using a pre-compiled binary of the Linux kernel in your server OS. That’s fine, but if you need hardware that isn’t supported “out of the box” or want to wring a little more speed from your server, compiling your own kernel may be the way to go. It’s good practice to know in general how to compile your own kernel, even if you’re just doing it for fun. Like anything, there are some advantages and disadvantages to compiling and running your own Linux kernel.
1. Pre-compiled binary kernels (such as you get by updating your system via yum or rpm) are made to run on a wide variety of hardware. Generic binaries have to work on a wide variety of processors, making it less specific to your machine. When programs are compiled for the specific architecture of your machine, they generally run faster and are more stable.
2. You can configure the kernel build process by excluding components and modules you do not need. This reduces the size of the kernel, making it boot faster and run in a more optimized manner. This is probably the main reason for people to want to compile their own Linux kernel.
3. With your own custom kernel, you can add whatever features or optimizations you want, even modifying the source code if the desire takes you :)
4. You’ll learn a little bit more about the process of compiling your own kernel and how the Linux boot process works.
With anything however, there are some distinct disadvantages to having your own custom kernel setup…
1. By compiling your own custom kernel, you now have to become the kernel maintainer. That means finding and applying all updates and security patches yourself and any future versions will depend on you and you alone. No more using yum to update the kernel on your system. This is probably the biggest downside for the casual kernel hacker.
2. Advocates of compiling your own kernel assume you know how to do it correctly. If you do not or if you make a mistake, things could go terribly wrong, and your server could even stop working altogether. While this is unlikely to be fatal in the long run, prepare for things to go wrong and take backups and also ensure you know how to rescue a failed system.
3. It takes a long time to compile a kernel – sometimes hours – and uses up a lot of CPU power during the process. Knowing how to configure everything properly requires a great deal of technological understanding and isn’t for those who just want to use a computer.
I know all that – let’s get our geek on!
First off, you’ll need some packages installed to compile a kernel. Please make sure you have the following installed using your package manager:-
make gcc ncurses-devel bzip2 bison
There are two major ways to get the main kernel source. One way is to get the kernel source from the maintainers of your distribution. The other way is getting the “mainline” kernel source from the Linux maintainers at kernel.org. The kernel source as maintained by your distribution will also get their source from mainline, but will add their own tweaks and optimisations that make it specific to their distribution. As distro-specific kernel source code is a few iterations behind mainline, I’ll use this as an excuse to compile the new (at the time of writing) 3.0.1 kernel from mainline to my RHEL box. While you can do all of this as a normal user, you can obviously only install a new kernel as root, so I’ll assume that you at least have access to the root account where necessary.
First thing to do is to grab the kernel source from kernel.org – for the lazy, it’s here.
You can extract the tar archive from the bzip2 file with: -
This will create a linux-3.0.1 sub-directory from wherever you extracted it. Enter that directory and run: -
This will give you the version of the current kernel that is running. Make a note of this version string and then run: -
cp /boot/config-[the version number] /path/to/your/linux-3.0.1/default.config
This will copy your current config file from boot to your kernel compilation directory, saved as default.config.
Next, fire up the kernel configuration TUI with: -
This is where you configure your new kernel. Since the configuration is pretty complicated, I’ll go into that another time. For now, we want to compile a version 3.0.1 kernel using our previously saved kernel configuration. Tab down to Load Configuration File and load the config file you’ve previously saved. Next, tab down to General Settings and change the local name to something memorable. This string will be appended to the kernel name after the version number. I’ll just call mine “-custom”. Tab to “Exit” you’ll be prompted to save the current configuration.
Now it’s time to compile. Type the following from the extracted kernel directory:-
This will compile the kernel and save it as “bzImage”. The location of the new kernel depends on the architecture used. I’m using a 64-bit system, so the location of the new kernel will be: -
[your extracted kernel directory]/arch/x86/boot/bzImage
Compilation will take a long time, so go and make a cup of tea…
Once it’s finished compiling, you should have a bzImage file in the location that matches your kernel version and architecture above. However, we’re not finished just yet – you next need to build the kernel modules. This are on-demand kernel modules that are loaded on an as-needed basis. Run the following next: -
Again, wait a while. Once the modules have finished compiling, they need to be installed. Don’t worry about overwriting anything, installed kernel modules get their own sub-directory named after the kernel version string :-) Run the following to install the modules: -
This installs the modules to /lib/modules/KERNEL_VERSION_STRING. You can see this directory by typing: -
ls -l /lib/modules
If you have built your main boot drivers as modules (e.g., SCSI host adapter, filesystem, RAID drivers) then you will need to create an initial RAMdisk image. The initrd is a way of sidestepping the chicken and egg problem of booting — drivers are needed to load the root filesystem but the filesystem cannot be loaded because the drivers are on the filesystem. An initial RAMdisk is simply a temporary filesystem that includes this information and makes it available to the kernel at boot time. Why might you have something so fundamental as a kernel module rather than building it in as part of the kernel? Well, usually generic kernels do this because they have no way of knowing how much memory is available on the system so on-demand kernel modules mean that not everything has to be loaded into the kernel and risk having a kernel that is so big, it cannot be held in RAM.
Based on the location of our new kernel modules directory (/lib/modules/3.0.1-custom) you can create an initial RAMdisk with: -
mkinitrd /boot/3.0.1.img 3.0.1-custom
Okay, so you have your kernel and you have compiled and installed modules for that kernel. Depending on how you’ve configured your kernel, you may now have an initial RAMdisk image also. This leaves actually telling the system about the new kernel. For this, I’ll assume you’re using GRUB as your bootloader. For anything else like LILO, consult Google on how to install a new kernel with a different bootloader.
First you need to copy the relevant files to the correct location under /boot and create a new symbolic link to the new System.map file.
cp arch/x86/boot/bzImage /boot/vmLinuz-3.0.1
cp System.map /boot/System.map-3.0.1
ln -s /boot/System.map-3.0.1 /boot/System.map
Finally, you need to add a new stanza to the /boot/grub/menu.lst. Copy the first entry and put it above that entry. Modify the entry so that it looks like: -
title My Custom Kernel (3.0.1)
kernel /boot/vmLinuz-3.0.1 ro root=LABEL=/
There is an entry called “default” at the top which will no doubt have “default=0″. This is the default kernel to attempt to load without any user intervention (i.e. you didn’t press a key during the boot process to get to the GRUB menu and choose a kernel entry to load manually). “0″ will load the first kernel stanza, “1″ will load the second and so on. Since we’ve put our custom kernel entry above the previous one, it will be loaded by default. Keep in mind that GRUB counts starting from 0, so (hd0,1) references the first disk, second partition – mostly your kernels will probably all be under (hd0,0) unless you have a custom disk set up. If you have created an initial RAMdisk be sure to include it here too.
If you run into boot issues Google search the errors and problems. You may need to compile in certain drivers to get stuff working. Double check your kernel configuration and make sure you have covered everything for your hardware. More importantly, don’t get frustrated if your boot fails. Creating custom kernels isn’t the easiest thing to do and should generally only be done if you need an optimized system or if you are working with embedded systems. Google for kernel configuration options and start pruning that kernel configuration tree for a lightning fast custom Linux system :-)