Commit | Line | Data |
---|---|---|
d7d3c2ea AG |
1 | The PPC KVM paravirtual interface |
2 | ================================= | |
3 | ||
4 | The basic execution principle by which KVM on PowerPC works is to run all kernel | |
5 | space code in PR=1 which is user space. This way we trap all privileged | |
6 | instructions and can emulate them accordingly. | |
7 | ||
8 | Unfortunately that is also the downfall. There are quite some privileged | |
9 | instructions that needlessly return us to the hypervisor even though they | |
10 | could be handled differently. | |
11 | ||
12 | This is what the PPC PV interface helps with. It takes privileged instructions | |
13 | and transforms them into unprivileged ones with some help from the hypervisor. | |
14 | This cuts down virtualization costs by about 50% on some of my benchmarks. | |
15 | ||
16 | The code for that interface can be found in arch/powerpc/kernel/kvm* | |
17 | ||
18 | Querying for existence | |
19 | ====================== | |
20 | ||
21 | To find out if we're running on KVM or not, we leverage the device tree. When | |
22 | Linux is running on KVM, a node /hypervisor exists. That node contains a | |
23 | compatible property with the value "linux,kvm". | |
24 | ||
25 | Once you determined you're running under a PV capable KVM, you can now use | |
26 | hypercalls as described below. | |
27 | ||
28 | KVM hypercalls | |
29 | ============== | |
30 | ||
31 | Inside the device tree's /hypervisor node there's a property called | |
32 | 'hypercall-instructions'. This property contains at most 4 opcodes that make | |
33 | up the hypercall. To call a hypercall, just call these instructions. | |
34 | ||
35 | The parameters are as follows: | |
36 | ||
37 | Register IN OUT | |
38 | ||
39 | r0 - volatile | |
40 | r3 1st parameter Return code | |
41 | r4 2nd parameter 1st output value | |
42 | r5 3rd parameter 2nd output value | |
43 | r6 4th parameter 3rd output value | |
44 | r7 5th parameter 4th output value | |
45 | r8 6th parameter 5th output value | |
46 | r9 7th parameter 6th output value | |
47 | r10 8th parameter 7th output value | |
48 | r11 hypercall number 8th output value | |
49 | r12 - volatile | |
50 | ||
51 | Hypercall definitions are shared in generic code, so the same hypercall numbers | |
52 | apply for x86 and powerpc alike with the exception that each KVM hypercall | |
53 | also needs to be ORed with the KVM vendor code which is (42 << 16). | |
54 | ||
55 | Return codes can be as follows: | |
56 | ||
57 | Code Meaning | |
58 | ||
59 | 0 Success | |
60 | 12 Hypercall not implemented | |
61 | <0 Error | |
62 | ||
63 | The magic page | |
64 | ============== | |
65 | ||
66 | To enable communication between the hypervisor and guest there is a new shared | |
67 | page that contains parts of supervisor visible register state. The guest can | |
68 | map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE. | |
69 | ||
70 | With this hypercall issued the guest always gets the magic page mapped at the | |
a4cd8b23 SW |
71 | desired location. The first parameter indicates the effective address when the |
72 | MMU is enabled. The second parameter indicates the address in real mode, if | |
73 | applicable to the target. For now, we always map the page to -4096. This way we | |
74 | can access it using absolute load and store functions. The following | |
75 | instruction reads the first field of the magic page: | |
d7d3c2ea AG |
76 | |
77 | ld rX, -4096(0) | |
78 | ||
79 | The interface is designed to be extensible should there be need later to add | |
80 | additional registers to the magic page. If you add fields to the magic page, | |
81 | also define a new hypercall feature to indicate that the host can give you more | |
82 | registers. Only if the host supports the additional features, make use of them. | |
83 | ||
54f65795 SW |
84 | The magic page layout is described by struct kvm_vcpu_arch_shared |
85 | in arch/powerpc/include/asm/kvm_para.h. | |
d7d3c2ea | 86 | |
d1e87c7e AG |
87 | Magic page features |
88 | =================== | |
89 | ||
90 | When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE, | |
91 | a second return value is passed to the guest. This second return value contains | |
92 | a bitmap of available features inside the magic page. | |
93 | ||
94 | The following enhancements to the magic page are currently available: | |
95 | ||
96 | KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page | |
f3383cf8 | 97 | KVM_MAGIC_FEAT_MAS0_TO_SPRG7 Maps MASn, ESR, PIR and high SPRGs |
d1e87c7e AG |
98 | |
99 | For enhanced features in the magic page, please check for the existence of the | |
100 | feature before using them! | |
101 | ||
f3383cf8 AG |
102 | Magic page flags |
103 | ================ | |
104 | ||
105 | In addition to features that indicate whether a host is capable of a particular | |
106 | feature we also have a channel for a guest to tell the guest whether it's capable | |
107 | of something. This is what we call "flags". | |
108 | ||
109 | Flags are passed to the host in the low 12 bits of the Effective Address. | |
110 | ||
111 | The following flags are currently available for a guest to expose: | |
112 | ||
5d4f6f3d | 113 | MAGIC_PAGE_FLAG_NOT_MAPPED_NX Guest handles NX bits correctly wrt magic page |
f3383cf8 | 114 | |
d7d3c2ea AG |
115 | MSR bits |
116 | ======== | |
117 | ||
118 | The MSR contains bits that require hypervisor intervention and bits that do | |
119 | not require direct hypervisor intervention because they only get interpreted | |
120 | when entering the guest or don't have any impact on the hypervisor's behavior. | |
121 | ||
122 | The following bits are safe to be set inside the guest: | |
123 | ||
124 | MSR_EE | |
125 | MSR_RI | |
d7d3c2ea AG |
126 | |
127 | If any other bit changes in the MSR, please still use mtmsr(d). | |
128 | ||
129 | Patched instructions | |
130 | ==================== | |
131 | ||
17180032 | 132 | The "ld" and "std" instructions are transformed to "lwz" and "stw" instructions |
25985edc | 133 | respectively on 32 bit systems with an added offset of 4 to accommodate for big |
d7d3c2ea AG |
134 | endianness. |
135 | ||
136 | The following is a list of mapping the Linux kernel performs when running as | |
137 | guest. Implementing any of those mappings is optional, as the instruction traps | |
138 | also act on the shared page. So calling privileged instructions still works as | |
139 | before. | |
140 | ||
141 | From To | |
142 | ==== == | |
143 | ||
144 | mfmsr rX ld rX, magic_page->msr | |
145 | mfsprg rX, 0 ld rX, magic_page->sprg0 | |
146 | mfsprg rX, 1 ld rX, magic_page->sprg1 | |
147 | mfsprg rX, 2 ld rX, magic_page->sprg2 | |
148 | mfsprg rX, 3 ld rX, magic_page->sprg3 | |
149 | mfsrr0 rX ld rX, magic_page->srr0 | |
150 | mfsrr1 rX ld rX, magic_page->srr1 | |
151 | mfdar rX ld rX, magic_page->dar | |
152 | mfdsisr rX lwz rX, magic_page->dsisr | |
153 | ||
154 | mtmsr rX std rX, magic_page->msr | |
155 | mtsprg 0, rX std rX, magic_page->sprg0 | |
156 | mtsprg 1, rX std rX, magic_page->sprg1 | |
157 | mtsprg 2, rX std rX, magic_page->sprg2 | |
158 | mtsprg 3, rX std rX, magic_page->sprg3 | |
159 | mtsrr0 rX std rX, magic_page->srr0 | |
160 | mtsrr1 rX std rX, magic_page->srr1 | |
161 | mtdar rX std rX, magic_page->dar | |
162 | mtdsisr rX stw rX, magic_page->dsisr | |
163 | ||
164 | tlbsync nop | |
165 | ||
166 | mtmsrd rX, 0 b <special mtmsr section> | |
167 | mtmsr rX b <special mtmsr section> | |
168 | ||
169 | mtmsrd rX, 1 b <special mtmsrd section> | |
170 | ||
cbe487fa AG |
171 | [Book3S only] |
172 | mtsrin rX, rY b <special mtsrin section> | |
173 | ||
d7d3c2ea AG |
174 | [BookE only] |
175 | wrteei [0|1] b <special wrteei section> | |
176 | ||
177 | ||
178 | Some instructions require more logic to determine what's going on than a load | |
179 | or store instruction can deliver. To enable patching of those, we keep some | |
180 | RAM around where we can live translate instructions to. What happens is the | |
181 | following: | |
182 | ||
183 | 1) copy emulation code to memory | |
184 | 2) patch that code to fit the emulated instruction | |
185 | 3) patch that code to return to the original pc + 4 | |
186 | 4) patch the original instruction to branch to the new code | |
187 | ||
188 | That way we can inject an arbitrary amount of code as replacement for a single | |
189 | instruction. This allows us to check for pending interrupts when setting EE=1 | |
190 | for example. | |
6024f1a4 AG |
191 | |
192 | Hypercall ABIs in KVM on PowerPC | |
193 | ================================= | |
194 | 1) KVM hypercalls (ePAPR) | |
195 | ||
196 | These are ePAPR compliant hypercall implementation (mentioned above). Even | |
197 | generic hypercalls are implemented here, like the ePAPR idle hcall. These are | |
198 | available on all targets. | |
199 | ||
200 | 2) PAPR hypercalls | |
201 | ||
202 | PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU). | |
203 | These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of | |
204 | them are handled in the kernel, some are handled in user space. This is only | |
205 | available on book3s_64. | |
206 | ||
207 | 3) OSI hypercalls | |
208 | ||
209 | Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long | |
210 | before KVM). This is supported to maintain compatibility. All these hypercalls get | |
211 | forwarded to user space. This is only useful on book3s_32, but can be used with | |
212 | book3s_64 as well. |