Lab on hacking the Linux kernel [GRADED]

This lab session is dedicated to the development of C code at Linux kernel level i.e., today, you will modify the Linux kernel. "Modify" means changing the source code of Linux, then compiling a new kernel, and then booting that new kernel.

You have to write a report - in pdf format - for this lab session, and you have to send me your report before Dec. 17th, 18:00, Paris time. I need only the pdf of your report: the pdf shall contain screen captures of your code, along with explanations and proofs that your code compiles and executes as expected. If you need to send me your code (syscal code, bonus work, ...), then send me a link (Eurecom ftp, weshare, etc.) to your VM.
More precisely, explain:

• I. What you have made for the history to work. (no screen capture needed). [2 points]


• II. Give the option you have removed and make a screen capture of the failed boot. [2 points]


• III. Provide a screen capture proving that your system call works. Nothing else is required. [2 points]


• IV. Give your code or screen shots of your code, and provide screen shots of your tests. [14 points]


• Bonus work. Give a link to your VM. [2 points]

Last, you have to work alone on this lab, therefore you have to produce your own code and results.

Information about the computer system

How to start / install the lab

The lab uses a vmware virtual machine running a TinyCore GNU/Linux distribution. This light distribution makes it possible to recompile a kernel in a few minutes, thus making it possible to test modifications on the Linux kernel quite quickly. If you were to recompile the Linux kernel of a regular distribution, it would take up to 30 minutes.

You will have to work in text mode. You may use several different terminals using ALT-F2, ALT-F2, ..., ALT-F6 combinations of keys. Then log in as root, with the password: eurecom. Also, to have the focus on the virtualized Linux, just click in the vmware window. To come back to your main OS (Windows, Linux), you may use "CTRL-ALT": the focus is then lost in the virtual machine. You can scroll in the terminal of the virtual machine by using the Shift-Page Up and Shift-Page Down shortcuts.

If you are at Eurecom

Today, you will log onto a Eurecom PC, and then you will start another operating system (Linux) using Vmware. Vmware makes it possible to start another Operating System in a virtual machine. Under this instance of Linux running inside this virtual machine, you are authorized to log in as root, without having any additional privilege under the top level Linux (and so, on the Eurecom network ...). This virtual machine could also be started under Windows, MacOSX, or more generally in all OS in which vmplayer can be started.

Starting the virtual machine

At Eurecom, the virtual machine is stored in a set of files located in the local disk of your PC in /home/Admin_Data/OS_Lab/. Uncompress and start it as follows:

$ mkdir -p /home/Local_Data/OS_Lab/
$ cd /home/Local_Data/OS_Lab/
$ tar xzf /home/Admin_Data/TPData/OS_Lab/TinyCoreFall2021.tgz
$ vmplayer

Then, click on "open a virtual machine", and select TinyCore in the right directory. "Power on" the machine.
You may be prompted a question about the origin of the VM. Answer "I copied it". Then Linux should boot in the virtual machine.

If you are at home

Use a pre-prepared Linux virtual machine

Follow the following steps.

1. Install vmplayer on your computer. If you are on macOS, see below "Use your own virtual machine"


2. Download the VM by ftp

$ ftp ftp.eurecom.fr

(There, you can use "anonymous" as login and enter your email as the paswword)

$ bin
$ cd incoming
$ get TinyCoreFall2021.tgz

3. Uncompress this file, and start vmplayer. Then, click on "open a virtual machine", and select TinyCore in the right directory. "Power on" the machine. You may be prompted a question about the origin of the VM. Answer "I copied it". Then Linux should boot in the virtual machine.

Use your own Linux virtual machine

If you don't want to install vmplayer or if this is not possible on your computer (e.g. on mac), the alternative is to install your own TinyCore distribution in another virtual machine, e.g. in VirtualBox.

My virtual machine is corrupted, what to do?

If the Linux virtual machine is damaged by your work, restart from a fresh version. That is redo the commands to install the virtual machine.

Basic commands

To edit a C file, you may do:

[~] # nano myfile.c

or

 
[~] # vi myfile.c

(i.e., emacs is not installed). You can switch between the US (qwerty) and other keyboard layouts by using the loadkmap command. To simplify switching to US or FR keyboards, the us and fr aliases were added to the .profile. For instance, for a "fr" keyboard, simply enter:

$ fr

If you want to use an italian keyboard, you can first type:

echo 'alias it=`loadkmap < /usr/share/kmap/qwerty/it.kmap`' >>/usr/src/initrd/root/.profile

Now, when you type "it", you switch to the Italian keyboard.

During this lab session, you may need to use Linux manual pages, section 9 (kernel). These manual pages are available inside the guest OS. e.g.:

[~] # man kmalloc

Saving your work

Exchanging files between host/VM

For the source code, and results of code execution, you can use the /root/shared folder which is shared between the guest virtual machine and its host at least at Eurecom, check your own vmplayer configuration). When you write something in this folder, it appears in the host machine in the /home/Local_Data/OS_Shared folder. If not yet done, you need to create this directory on your (Eurecom) local PC:

$ mkdir -p /home/Local_Data/OS_Shared/

You can change the path of the shared folder on the host in the settings of the virtual machine or create a symbolic link from this location to your preferred location.

Saving your work

If you are at Eurecom, the virtual machine is on a local disk. Thus, you cannot find the same virtual machine if you log on another computer.

To save your work, e.g. to work at home or to continue it on another PC, the best way is to copy the entire content of the virtual machine to a USB key. First remake a compressed archive of files you have uncompressed,

$ cd /home/LocalData/OS_Lab/
$ tar czf TinyCore.tgz TinyCore/

and then make the copy of that archive to your USB key.

Last, but not least, at Eurecom avoid copying the file to your home directory because it could saturate your quota.

I. Building the rootfs

Now, all following commands and exercises have to be done on the virtualized Linux, logged in as root, password: "eurecom".

Make the following command:

$ mount

You should obtain three main partitions for storing files:

• Two partitions using the "ext3" file system. These partitions are kept upon reboot i.e. they are stored on a disk.
• One partition based on RAMFS.

Thus, the files you create or modify in the guest VM that are stored in the RAMFS partition will be lost upon reboot. To test this, create a file in the home directory of the root account and reboot. The file should not be there anymore. Now, enter the following command:

[~] # df -T /root

You should obtain "rootfs" which means "root filesystem". rootfs is a kind of filesystem derived from the "ramfs" filesystem, which is a RAM-based filesystem. Any operation targeting this filesystem only affects the content inside the RAM, and is thus lost upon reboot. When the kernel boots, the RAM filesystem is read from a compressed file (/boot/core.gz) and loaded into memory.

When you type commands in the interactive ash - a shell derived from bash - instance in your terminal, the commands are written in a file ~/.ash_history so that you can see the history of the commands later. Why is your history empty upon reboot?

[~] # history
   0 history
[~] #

You are going to modify the rootfs to specify another file to store the history so that your ash history is kept across sessions. First, find a location to store your history that is not reset upon reboot. You can re-run the "more /etc/fstab" command to find a location which is not lost when rebooting.

We now need to update the .profile file not in the RAM filesystem, but in the /boot/core.gz file which is used to create the filesystem. This /boot/core.gz file is created from the /usr/src/initrd directory. Change to this directory and modify the /usr/src/initrd/root/.profile file to specify the new location of the history file:

export HISTFILE="path-to-your-file"

We need to include the modified .profile file to the compressed rootfs that is loaded in memory at boot time. To do this, go to /usr/src/initrd and create a new rootfs using the following alias:

[initrd] # makeinitrd > /boot/core.gz

Reboot now. Verify that the /root/.profile contains the line you have added. If it does, reboot again, and you should see the history of the commands you have run before rebooting.

II. Compiling the Linux kernel

Unlike Windows, Linux makes it possible to customize the kernel, i.e., to change the sources and then to recompile the sources to generate a new kernel. Compiling the kernel is composed of four stages:

1. Configuration. Functionalities included into the kernel are chosen at this stage.
2. Building. Kernel sources are compiled to a new kernel file (called a kernel image).
3. Installing. The system must be configured to be able to boot the newly generated kernel.
4. Rebooting. Your computer is restarted, and you can select the newly generated kernel when the boot menu appears.

In this exercise, you will learn how to perform these four stages.

1. Version of your Linux kernel

The uname UNIX command allows us to get various information about the kernel. Find an option of this command (use manual pages) to get your Linux kernel's current release number. Is the kernel adapted to multiprocessor systems?

Kernel sources are located into /usr/src/linux-4.2.9. This directory contains a set of subdirectories. Each of them contains a specific part of the kernel sources:

Directory	Explanations
arch	Hardware-dependent Components. Arch contains a subdirectory per supported platform (i386, alpha, mips, sparc, etc.)
drivers	Explicit!
fs	File System
include	Header files. They contain definitions of data structures and constants.
ipc	Interprocess communication (shared memory, semaphore, etc.).
kernel	Core functions of the kernel (process management, scheduler, etc.).
mm	Memory management.
net	Protocol stacks such as TCP/IP, etc.

Note that it might be a good idea to explore some sub-directories and visualize the content of a few files, and deduce more closely the content of each subdirectory and of each file (don't spend too much time on this, no more than a few minutes).

2. The objective is now to change an option of the kernel

Go into the kernel sources' directory (i.e., in /usr/src/linux-4.2.9) and run the configuration tool:

[linux-4.2.9] # make menuconfig

This tool makes it possible to explore various kernel functionalities and to set whether they are included in the kernel, not included, or set as loadable module. For some of these functionalities, figure out what they do.

Now, the objective is to remove a functionality which will cause the kernel not to boot correctly. I suggest removing a functionality regarding the filesystem management. Indeed, if you remove the support for the filesystem used for the root partition, the kernel will not be able to access system files, and therefore, the boot process will stop because the kernel cannot load its required files.
Remove support for initial RAM filesystem and RAM disk, and save this configuration. Note that during menuconfig, you can press the / key to search for a specific entry.

3. Compile the kernel

Go into the kernel sources' directory. Since you have just modified configuration option, clean older object files:

[linux-4.2.9] # make clean

Note that you only need to do this after modifying the configuration file (through make menuconfig).

You can now generate the new kernel:

[linux-4.2.9] # make -j8 bzImage

Depending of your hardware configuration, this command may take a few minutes. This command compiles all kernel sources and generates a file named "bzImage". This file contains the executable code of the new kernel. You can also run the two Makefile targets once at a time by typing:

[linux-4.2.9] # make -j8 clean bzImage

Note that if you included a new feature as a module, you would also need to compile the new modules:

[linux-4.2.9] # make -j8 modules

4. Add the possibility to select this new kernel at boot up

Once compiled, the new kernel should be made available at startup choice menu. This menu is offered by the grub bootloader. Explain what a bootloader does and at which step of the boot process it is called

Grub can be configured to be able to select your new kernel at boot up:

1. Copy the newly generated kernel to the boot directory. To do so, type the following command from the kernel sources main directory:

[linux-4.2.9] # cp arch/x86/boot/bzImage /boot/myNewKernel

with myNewKernel being the name of your new kernel.

Note that if you had compiled new modules, you would have had to install them into the initial RAM file system.


2. Open the Grub configuration file located in /boot/grub/menu.lst, and add the following lines:

title New Kernel
kernel /myNewKernel quiet superuser nozswap multivt syslog vga=832 tce=sda1
initrd /core.gz

5. Reboot the system

[~] # reboot

At startup, select your new kernel and check whether it works, or not. If it doesn't boot, check where it stops during the boot process, and explain why. Don't spend too much time on this exercise, the most interesting part of this laboratory course is still to come! When you have finished this first exercise, reboot on the default kernel i.e., on the one initially provided, and rerun make menuconfig: your objective is to select initially selected options (configuration you had when you first run make menuconfig).

Now, the next objectives of this lab is to add several system calls to Linux.

III. Adding a basic system call to Linux: "kernelprint" system call

This lab has been built from kernel.org / adding a new system call

1. Explain what are system calls, and basically explain how they work (not in Linux, but more generally)

In Linux, when you write a C program that uses library functions, you don't really realize how those libraries work. But these libraries often make system calls, that is, they ask the Linux kernel to do subtasks for them.

In Linux, each system call is given an identifier under the form of a natural value. When a system call is performed, the corresponding value is put into the EAX register of the CPU. Then, a trap is generated. Usually, this trap is accomplished by the INT 0x80 assembly instruction. Arguments of the system call are passed to the Operating System via other registers. Note that this is different from a typical function call where the stack is used to pass arguments (on x86 architectures).

Now, you are going to program your first system call. This system call simply prints a kernel message of your choice in the console. For this very first system call, you will be guided for each programming step, so, don't panic!

2. Add the entry point for your new system call.

.c Kernel files containing C code for architecture-independent features are usually placed into a subdirectory of the main directory. For example, kernel files regarding drivers are put under the drivers subdirectory. Select the subdirectory adapted to your case and create your .c file. I suggest you use a file named myservices.c.

myservices.c should look like this:

The components of this file are:

• #include directives to make the definition of printk and SYSCALL_DEFINE0 available.
• The declaration of your new system call: SYSCALL_DEFINE0(kernelprint)
  • The SYSCALL_DEFINE0 macro is defined in include/linux/syscall.h and is used by other tools to get metadata about the system call. It means that we are defining a new system call.
  • The 0 means that this system call takes zero arguments.
  • The first (and, in this case, only) parameter passed to the macro is the system call name. Here, we named it kernelprint. Feel free to modify this name, but be careful to use the same name for the rest of the modifications.
• The actual code of the system call. printk is an internal kernel function that prints out a string to the /var/log/messages file. Basically, it works like printf(), but it is reserved for the kernel.

Since you added a new file, you need to add a directive to specify to the compilation process that this new file should be processed during compilation. These compilation directives are given within a Makefile. You need to update the Makefile located in the directory of the newly added file because it needs to reference the files into which your new system call is implemented.

Open this Makefile and add myservices.o to the list of objects (obj-y).

3. Add a prototype for the system call.

You should now define a prototype for your system call so that user-space programs can call it. Architecture-independent system call prototypes should be added to the include/linux/syscalls.h header. Edit this file and add a new line for your system call: While editing that file, which syscalls can you recognize?

4. Add an table entry for your system call.

As mentioned before, system calls cannot be called directly from user processes. Instead, they are called indirectly via interrupts and lookup in the system call table. Thus, when you define a new system call, you need to insert a new entry in this table. You thus need to define an index in this table (for example, 223).

To do this, edit the file arch/x86/entry/syscalls/syscall_32.tbl. You will find a list of system call entries. Find an unused number and add an entry for your system call:

5. Generate a new kernel that includes your system call

Your system call is now almost ready to be used ... but only almost! Indeed, the kernel has not been recompiled yet, so, the kernel you're currently using doesn't have any knowledge about the system call you've just defined. So, you need to recompile the kernel (you now know the procedure).

The first time you compile the kernel, you should perform a make clean before compiling the kernel (make bzImage). But next time, do avoid making dependencies and cleaning, you will save lot of time at compilation step, just do make bzImage. Of course, each time, you must copy the newly generated kernel to /boot.

Reboot on the new kernel (don't forget to select it when system starts!).

6. Using your system call

Here is an example of a small program that uses your system call. Place it in the shared directory to ease the modifications on this file.

example.c:

7. Testing your system call

First, compile example.c.

[shared] # gcc -I /usr/src/linux-4.2.9/include -o example example.c

Then, execute your program as follows:

[shared] # ./example

To visualize the result, you may use the file /var/log/messages into which console messages are appended.

[shared] # tail -f /var/log/messages

IV. Your system call ("kernelTrace")

Instead of printing something to the console, the objective is now to implement a new system call (called kernelTrace) that takes as argument a string "s".

• If the string "s" is of the form "trace id" then the syscall returns all strings that have been registered with this id, and well at the time at which they have been registered. Error -1 is returned in case no elements are registered with the provided id.


• If the string is of the form "register id s" then the syscall registers string "s" with the provided id. The syscall returns -2 in case no more space is available to store elements (see below).


• If the string is of the form "rm id s" then the syscall removes all strings with the provided id. -3 is returned in case id does not exist.

Also, to avoid saturating kernel's memory, we will assume that at most the 1024 strings can be registered, corresponding to at most 1024 ids. Last, you are free to use a list of input and output arguments of your choice for your syscall.

Let's see an execution scenario for kernelTrace, assuming that this is the first time this syscall is called:

kernelTrace("register 1 hello")

returns 0

Then,

kernelTrace("register 1 hello world!")

returns 0

Then,

kernelTrace("trace 1")

returns "date/time: hello\ndate/time: hello world!"

Then,

kernelTrace("rm 2")

returns -3

Then,

kernelTrace("rm 1")

returns 0

Finally,

kernelTrace("trace 1")

returns -1
...


So, myservices.c could look like this:
And example.c could look like this (but you will surely have to modify this...):

I suggest to proceed as follows:

• Modify syscalls.h and syscall_32.tbl.


• Test the new system call (recompile the kernel and so on). You will probably obtain an error message when executing your new program. Provide this error message and explain it.


• Modify the system call (and the test program) to resolve the previous error: to resolve this error, you will probably have to change the definition of the syscall and the example I provide. You may also need to use some functions / hints provided in the section untitled "Linux system calls: some useful functions", just below.


• Once your syscall works, provide different execution traces including the execution scenario trace given before, but also other traces obtained by using several sequences of calls. For instance, show what happens when the id facility is full (i.e., after more than 1024 calls).

Bonus work: Optimizing the size of the Linux kernel

Now, you are less guided. The objective is to remove options in the Linux kernel until its size is very small. The Linux kernel must still boot and your last system call (log) must still be operational. Give the size you have obtained, and provide me with the options you have selected / deselected. Keep your VM somewhere so that I can evaluate your small kernel.

Annex

Passing arguments to a system call

Example of the definition of a system call:


Example: definition of a system call named "foo" that takes as input an int named 'i':

Init function

For each system call, it is possible to set an initialize function that is called at system bootup. This function must be added in your source code. It should start with "__init" (two underscores). Also, you must include < linux/init.h > for the "__init" macro to be recognized at compilation step.
Example:


At last, you must inform the kernel to call your initialization function at startup. To achieve that, you need to add two lines to init/main.c:


and within "asmlinkage __visible void __init start_kernel(void)" function:

Various kernel libraries functions (use manual pages for more information, section 9)

Functions	Explanations
printk()	Print messages to console
kmalloc(), kfree()	Allocate / free chunks of memory
cti(), sti()	Disable / Enable interrupts
get_user(), put_user(), copy_from_user(), copy_to_user()	Copy data between kernel space and user space