Commit | Line | Data |
---|---|---|
d7d3c2ea AG |
1 | The PPC KVM paravirtual interface |
2 | ================================= | |
3 | ||
4 | The basic execution principle by which KVM on PowerPC works is to run all kernel | |
5 | space code in PR=1 which is user space. This way we trap all privileged | |
6 | instructions and can emulate them accordingly. | |
7 | ||
8 | Unfortunately that is also the downfall. There are quite some privileged | |
9 | instructions that needlessly return us to the hypervisor even though they | |
10 | could be handled differently. | |
11 | ||
12 | This is what the PPC PV interface helps with. It takes privileged instructions | |
13 | and transforms them into unprivileged ones with some help from the hypervisor. | |
14 | This cuts down virtualization costs by about 50% on some of my benchmarks. | |
15 | ||
16 | The code for that interface can be found in arch/powerpc/kernel/kvm* | |
17 | ||
18 | Querying for existence | |
19 | ====================== | |
20 | ||
21 | To find out if we're running on KVM or not, we leverage the device tree. When | |
22 | Linux is running on KVM, a node /hypervisor exists. That node contains a | |
23 | compatible property with the value "linux,kvm". | |
24 | ||
25 | Once you determined you're running under a PV capable KVM, you can now use | |
26 | hypercalls as described below. | |
27 | ||
28 | KVM hypercalls | |
29 | ============== | |
30 | ||
31 | Inside the device tree's /hypervisor node there's a property called | |
32 | 'hypercall-instructions'. This property contains at most 4 opcodes that make | |
33 | up the hypercall. To call a hypercall, just call these instructions. | |
34 | ||
35 | The parameters are as follows: | |
36 | ||
37 | Register IN OUT | |
38 | ||
39 | r0 - volatile | |
40 | r3 1st parameter Return code | |
41 | r4 2nd parameter 1st output value | |
42 | r5 3rd parameter 2nd output value | |
43 | r6 4th parameter 3rd output value | |
44 | r7 5th parameter 4th output value | |
45 | r8 6th parameter 5th output value | |
46 | r9 7th parameter 6th output value | |
47 | r10 8th parameter 7th output value | |
48 | r11 hypercall number 8th output value | |
49 | r12 - volatile | |
50 | ||
51 | Hypercall definitions are shared in generic code, so the same hypercall numbers | |
52 | apply for x86 and powerpc alike with the exception that each KVM hypercall | |
53 | also needs to be ORed with the KVM vendor code which is (42 << 16). | |
54 | ||
55 | Return codes can be as follows: | |
56 | ||
57 | Code Meaning | |
58 | ||
59 | 0 Success | |
60 | 12 Hypercall not implemented | |
61 | <0 Error | |
62 | ||
63 | The magic page | |
64 | ============== | |
65 | ||
66 | To enable communication between the hypervisor and guest there is a new shared | |
67 | page that contains parts of supervisor visible register state. The guest can | |
68 | map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE. | |
69 | ||
70 | With this hypercall issued the guest always gets the magic page mapped at the | |
71 | desired location in effective and physical address space. For now, we always | |
72 | map the page to -4096. This way we can access it using absolute load and store | |
73 | functions. The following instruction reads the first field of the magic page: | |
74 | ||
75 | ld rX, -4096(0) | |
76 | ||
77 | The interface is designed to be extensible should there be need later to add | |
78 | additional registers to the magic page. If you add fields to the magic page, | |
79 | also define a new hypercall feature to indicate that the host can give you more | |
80 | registers. Only if the host supports the additional features, make use of them. | |
81 | ||
82 | The magic page has the following layout as described in | |
83 | arch/powerpc/include/asm/kvm_para.h: | |
84 | ||
85 | struct kvm_vcpu_arch_shared { | |
86 | __u64 scratch1; | |
87 | __u64 scratch2; | |
88 | __u64 scratch3; | |
89 | __u64 critical; /* Guest may not get interrupts if == r1 */ | |
90 | __u64 sprg0; | |
91 | __u64 sprg1; | |
92 | __u64 sprg2; | |
93 | __u64 sprg3; | |
94 | __u64 srr0; | |
95 | __u64 srr1; | |
96 | __u64 dar; | |
97 | __u64 msr; | |
98 | __u32 dsisr; | |
99 | __u32 int_pending; /* Tells the guest if we have an interrupt */ | |
100 | }; | |
101 | ||
102 | Additions to the page must only occur at the end. Struct fields are always 32 | |
103 | or 64 bit aligned, depending on them being 32 or 64 bit wide respectively. | |
104 | ||
d1e87c7e AG |
105 | Magic page features |
106 | =================== | |
107 | ||
108 | When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE, | |
109 | a second return value is passed to the guest. This second return value contains | |
110 | a bitmap of available features inside the magic page. | |
111 | ||
112 | The following enhancements to the magic page are currently available: | |
113 | ||
114 | KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page | |
115 | ||
116 | For enhanced features in the magic page, please check for the existence of the | |
117 | feature before using them! | |
118 | ||
d7d3c2ea AG |
119 | MSR bits |
120 | ======== | |
121 | ||
122 | The MSR contains bits that require hypervisor intervention and bits that do | |
123 | not require direct hypervisor intervention because they only get interpreted | |
124 | when entering the guest or don't have any impact on the hypervisor's behavior. | |
125 | ||
126 | The following bits are safe to be set inside the guest: | |
127 | ||
128 | MSR_EE | |
129 | MSR_RI | |
130 | MSR_CR | |
131 | MSR_ME | |
132 | ||
133 | If any other bit changes in the MSR, please still use mtmsr(d). | |
134 | ||
135 | Patched instructions | |
136 | ==================== | |
137 | ||
138 | The "ld" and "std" instructions are transormed to "lwz" and "stw" instructions | |
25985edc | 139 | respectively on 32 bit systems with an added offset of 4 to accommodate for big |
d7d3c2ea AG |
140 | endianness. |
141 | ||
142 | The following is a list of mapping the Linux kernel performs when running as | |
143 | guest. Implementing any of those mappings is optional, as the instruction traps | |
144 | also act on the shared page. So calling privileged instructions still works as | |
145 | before. | |
146 | ||
147 | From To | |
148 | ==== == | |
149 | ||
150 | mfmsr rX ld rX, magic_page->msr | |
151 | mfsprg rX, 0 ld rX, magic_page->sprg0 | |
152 | mfsprg rX, 1 ld rX, magic_page->sprg1 | |
153 | mfsprg rX, 2 ld rX, magic_page->sprg2 | |
154 | mfsprg rX, 3 ld rX, magic_page->sprg3 | |
155 | mfsrr0 rX ld rX, magic_page->srr0 | |
156 | mfsrr1 rX ld rX, magic_page->srr1 | |
157 | mfdar rX ld rX, magic_page->dar | |
158 | mfdsisr rX lwz rX, magic_page->dsisr | |
159 | ||
160 | mtmsr rX std rX, magic_page->msr | |
161 | mtsprg 0, rX std rX, magic_page->sprg0 | |
162 | mtsprg 1, rX std rX, magic_page->sprg1 | |
163 | mtsprg 2, rX std rX, magic_page->sprg2 | |
164 | mtsprg 3, rX std rX, magic_page->sprg3 | |
165 | mtsrr0 rX std rX, magic_page->srr0 | |
166 | mtsrr1 rX std rX, magic_page->srr1 | |
167 | mtdar rX std rX, magic_page->dar | |
168 | mtdsisr rX stw rX, magic_page->dsisr | |
169 | ||
170 | tlbsync nop | |
171 | ||
172 | mtmsrd rX, 0 b <special mtmsr section> | |
173 | mtmsr rX b <special mtmsr section> | |
174 | ||
175 | mtmsrd rX, 1 b <special mtmsrd section> | |
176 | ||
cbe487fa AG |
177 | [Book3S only] |
178 | mtsrin rX, rY b <special mtsrin section> | |
179 | ||
d7d3c2ea AG |
180 | [BookE only] |
181 | wrteei [0|1] b <special wrteei section> | |
182 | ||
183 | ||
184 | Some instructions require more logic to determine what's going on than a load | |
185 | or store instruction can deliver. To enable patching of those, we keep some | |
186 | RAM around where we can live translate instructions to. What happens is the | |
187 | following: | |
188 | ||
189 | 1) copy emulation code to memory | |
190 | 2) patch that code to fit the emulated instruction | |
191 | 3) patch that code to return to the original pc + 4 | |
192 | 4) patch the original instruction to branch to the new code | |
193 | ||
194 | That way we can inject an arbitrary amount of code as replacement for a single | |
195 | instruction. This allows us to check for pending interrupts when setting EE=1 | |
196 | for example. |