Commit | Line | Data |
---|---|---|
8d497515 CY |
1 | An Intel MIC X100 device is a PCIe form factor add-in coprocessor |
2 | card based on the Intel Many Integrated Core (MIC) architecture | |
3 | that runs a Linux OS. It is a PCIe endpoint in a platform and therefore | |
4 | implements the three required standard address spaces i.e. configuration, | |
5 | memory and I/O. The host OS loads a device driver as is typical for | |
6 | PCIe devices. The card itself runs a bootstrap after reset that | |
af190494 DC |
7 | transfers control to the card OS downloaded from the host driver. The |
8 | host driver supports OSPM suspend and resume operations. It shuts down | |
9 | the card during suspend and reboots the card OS during resume. | |
8d497515 CY |
10 | The card OS as shipped by Intel is a Linux kernel with modifications |
11 | for the X100 devices. | |
12 | ||
13 | Since it is a PCIe card, it does not have the ability to host hardware | |
14 | devices for networking, storage and console. We provide these devices | |
15 | on X100 coprocessors thus enabling a self-bootable equivalent environment | |
16 | for applications. A key benefit of our solution is that it leverages | |
17 | the standard virtio framework for network, disk and console devices, | |
18 | though in our case the virtio framework is used across a PCIe bus. | |
19 | ||
a8035843 SY |
20 | MIC PCIe card has a dma controller with 8 channels. These channels are |
21 | shared between the host s/w and the card s/w. 0 to 3 are used by host | |
22 | and 4 to 7 by card. As the dma device doesn't show up as PCIe device, | |
23 | a virtual bus called mic bus is created and virtual dma devices are | |
24 | created on it by the host/card drivers. On host the channels are private | |
25 | and used only by the host driver to transfer data for the virtio devices. | |
26 | ||
8d497515 CY |
27 | Here is a block diagram of the various components described above. The |
28 | virtio backends are situated on the host rather than the card given better | |
29 | single threaded performance for the host compared to MIC, the ability of | |
30 | the host to initiate DMA's to/from the card using the MIC DMA engine and | |
31 | the fact that the virtio block storage backend can only be on the host. | |
32 | ||
a8035843 SY |
33 | | |
34 | +----------+ | +----------+ | |
35 | | Card OS | | | Host OS | | |
36 | +----------+ | +----------+ | |
37 | | | |
38 | +-------+ +--------+ +------+ | +---------+ +--------+ +--------+ | |
39 | | Virtio| |Virtio | |Virtio| | |Virtio | |Virtio | |Virtio | | |
40 | | Net | |Console | |Block | | |Net | |Console | |Block | | |
41 | | Driver| |Driver | |Driver| | |backend | |backend | |backend | | |
42 | +-------+ +--------+ +------+ | +---------+ +--------+ +--------+ | |
43 | | | | | | | | | |
44 | | | | |User | | | | |
45 | | | | |------|------------|---------|------- | |
46 | +-------------------+ |Kernel +--------------------------+ | |
47 | | | | Virtio over PCIe IOCTLs | | |
48 | | | +--------------------------+ | |
49 | +-----------+ | | | +-----------+ | |
50 | | MIC DMA | | | | | MIC DMA | | |
51 | | Driver | | | | | Driver | | |
52 | +-----------+ | | | +-----------+ | |
53 | | | | | | | |
54 | +---------------+ | | | +----------------+ | |
55 | |MIC virtual Bus| | | | |MIC virtual Bus | | |
56 | +---------------+ | | | +----------------+ | |
57 | | | | | | | |
58 | | +--------------+ | +---------------+ | | |
59 | | |Intel MIC | | |Intel MIC | | | |
60 | +---|Card Driver | | |Host Driver | | | |
61 | +--------------+ | +---------------+-----+ | |
62 | | | | | |
63 | +-------------------------------------------------------------+ | |
64 | | | | |
65 | | PCIe Bus | | |
66 | +-------------------------------------------------------------+ |