Commit | Line | Data |
---|---|---|
7e07e7ae IL |
1 | =================== |
2 | Testing BPF on s390 | |
3 | =================== | |
4 | ||
5 | 1. Introduction | |
6 | *************** | |
7 | ||
8 | IBM Z are mainframe computers, which are descendants of IBM System/360 from | |
9 | year 1964. They are supported by the Linux kernel under the name "s390". This | |
10 | document describes how to test BPF in an s390 QEMU guest. | |
11 | ||
12 | 2. One-time setup | |
13 | ***************** | |
14 | ||
15 | The following is required to build and run the test suite: | |
16 | ||
17 | * s390 GCC | |
18 | * s390 development headers and libraries | |
19 | * Clang with BPF support | |
20 | * QEMU with s390 support | |
21 | * Disk image with s390 rootfs | |
22 | ||
23 | Debian supports installing compiler and libraries for s390 out of the box. | |
24 | Users of other distros may use debootstrap in order to set up a Debian chroot:: | |
25 | ||
26 | sudo debootstrap \ | |
27 | --variant=minbase \ | |
28 | --include=sudo \ | |
29 | testing \ | |
30 | ./s390-toolchain | |
31 | sudo mount --rbind /dev ./s390-toolchain/dev | |
32 | sudo mount --rbind /proc ./s390-toolchain/proc | |
33 | sudo mount --rbind /sys ./s390-toolchain/sys | |
34 | sudo chroot ./s390-toolchain | |
35 | ||
36 | Once on Debian, the build prerequisites can be installed as follows:: | |
37 | ||
38 | sudo dpkg --add-architecture s390x | |
39 | sudo apt-get update | |
40 | sudo apt-get install \ | |
41 | bc \ | |
42 | bison \ | |
43 | cmake \ | |
44 | debootstrap \ | |
45 | dwarves \ | |
46 | flex \ | |
47 | g++ \ | |
48 | gcc \ | |
49 | g++-s390x-linux-gnu \ | |
50 | gcc-s390x-linux-gnu \ | |
51 | gdb-multiarch \ | |
52 | git \ | |
53 | make \ | |
54 | python3 \ | |
55 | qemu-system-misc \ | |
56 | qemu-utils \ | |
57 | rsync \ | |
58 | libcap-dev:s390x \ | |
59 | libelf-dev:s390x \ | |
60 | libncurses-dev | |
61 | ||
62 | Latest Clang targeting BPF can be installed as follows:: | |
63 | ||
64 | git clone https://github.com/llvm/llvm-project.git | |
65 | ln -s ../../clang llvm-project/llvm/tools/ | |
66 | mkdir llvm-project-build | |
67 | cd llvm-project-build | |
68 | cmake \ | |
69 | -DLLVM_TARGETS_TO_BUILD=BPF \ | |
70 | -DCMAKE_BUILD_TYPE=Release \ | |
71 | -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \ | |
72 | ../llvm-project/llvm | |
73 | make | |
74 | sudo make install | |
75 | export PATH=/opt/clang-bpf/bin:$PATH | |
76 | ||
77 | The disk image can be prepared using a loopback mount and debootstrap:: | |
78 | ||
79 | qemu-img create -f raw ./s390.img 1G | |
80 | sudo losetup -f ./s390.img | |
81 | sudo mkfs.ext4 /dev/loopX | |
82 | mkdir ./s390.rootfs | |
83 | sudo mount /dev/loopX ./s390.rootfs | |
84 | sudo debootstrap \ | |
85 | --foreign \ | |
86 | --arch=s390x \ | |
87 | --variant=minbase \ | |
88 | --include=" \ | |
89 | iproute2, \ | |
90 | iputils-ping, \ | |
91 | isc-dhcp-client, \ | |
92 | kmod, \ | |
93 | libcap2, \ | |
94 | libelf1, \ | |
95 | netcat, \ | |
96 | procps" \ | |
97 | testing \ | |
98 | ./s390.rootfs | |
99 | sudo umount ./s390.rootfs | |
100 | sudo losetup -d /dev/loopX | |
101 | ||
102 | 3. Compilation | |
103 | ************** | |
104 | ||
105 | In addition to the usual Kconfig options required to run the BPF test suite, it | |
106 | is also helpful to select:: | |
107 | ||
108 | CONFIG_NET_9P=y | |
109 | CONFIG_9P_FS=y | |
110 | CONFIG_NET_9P_VIRTIO=y | |
111 | CONFIG_VIRTIO_PCI=y | |
112 | ||
113 | as that would enable a very easy way to share files with the s390 virtual | |
114 | machine. | |
115 | ||
116 | Compiling kernel, modules and testsuite, as well as preparing gdb scripts to | |
117 | simplify debugging, can be done using the following commands:: | |
118 | ||
119 | make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig | |
120 | make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb | |
121 | make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \ | |
122 | -C tools/testing/selftests \ | |
123 | TARGETS=bpf \ | |
124 | INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \ | |
125 | install | |
126 | ||
127 | 4. Running the test suite | |
128 | ************************* | |
129 | ||
130 | The virtual machine can be started as follows:: | |
131 | ||
132 | qemu-system-s390x \ | |
133 | -cpu max,zpci=on \ | |
134 | -smp 2 \ | |
135 | -m 4G \ | |
136 | -kernel linux/arch/s390/boot/compressed/vmlinux \ | |
137 | -drive file=./s390.img,if=virtio,format=raw \ | |
138 | -nographic \ | |
139 | -append 'root=/dev/vda rw console=ttyS1' \ | |
140 | -virtfs local,path=./linux,security_model=none,mount_tag=linux \ | |
141 | -object rng-random,filename=/dev/urandom,id=rng0 \ | |
142 | -device virtio-rng-ccw,rng=rng0 \ | |
143 | -netdev user,id=net0 \ | |
144 | -device virtio-net-ccw,netdev=net0 | |
145 | ||
146 | When using this on a real IBM Z, ``-enable-kvm`` may be added for better | |
147 | performance. When starting the virtual machine for the first time, disk image | |
148 | setup must be finalized using the following command:: | |
149 | ||
150 | /debootstrap/debootstrap --second-stage | |
151 | ||
152 | Directory with the code built on the host as well as ``/proc`` and ``/sys`` | |
153 | need to be mounted as follows:: | |
154 | ||
155 | mkdir -p /linux | |
156 | mount -t 9p linux /linux | |
157 | mount -t proc proc /proc | |
158 | mount -t sysfs sys /sys | |
159 | ||
160 | After that, the test suite can be run using the following commands:: | |
161 | ||
162 | cd /linux/tools/testing/selftests/kselftest_install | |
163 | ./run_kselftest.sh | |
164 | ||
165 | As usual, tests can be also run individually:: | |
166 | ||
167 | cd /linux/tools/testing/selftests/bpf | |
168 | ./test_verifier | |
169 | ||
170 | 5. Debugging | |
171 | ************ | |
172 | ||
173 | It is possible to debug the s390 kernel using QEMU GDB stub, which is activated | |
174 | by passing ``-s`` to QEMU. | |
175 | ||
176 | It is preferable to turn KASLR off, so that gdb would know where to find the | |
177 | kernel image in memory, by building the kernel with:: | |
178 | ||
179 | RANDOMIZE_BASE=n | |
180 | ||
181 | GDB can then be attached using the following command:: | |
182 | ||
183 | gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux | |
184 | ||
185 | 6. Network | |
186 | ********** | |
187 | ||
188 | In case one needs to use the network in the virtual machine in order to e.g. | |
189 | install additional packages, it can be configured using:: | |
190 | ||
191 | dhclient eth0 | |
192 | ||
193 | 7. Links | |
194 | ******** | |
195 | ||
196 | This document is a compilation of techniques, whose more comprehensive | |
197 | descriptions can be found by following these links: | |
198 | ||
199 | - `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_ | |
200 | - `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_ | |
201 | - `Building LLVM <https://llvm.org/docs/CMake.html>`_ | |
202 | - `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_ | |
203 | - `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_ | |
204 | - `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_ | |
205 | - `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_ |