Commit | Line | Data |
---|---|---|
aa4d16e4 JW |
1 | ******************************** |
2 | RDMA Network Block Device (RNBD) | |
3 | ******************************** | |
4 | ||
5 | Introduction | |
6 | ------------ | |
7 | ||
8 | RNBD (RDMA Network Block Device) is a pair of kernel modules | |
9 | (client and server) that allow for remote access of a block device on | |
10 | the server over RTRS protocol using the RDMA (InfiniBand, RoCE, iWARP) | |
11 | transport. After being mapped, the remote block devices can be accessed | |
12 | on the client side as local block devices. | |
13 | ||
14 | I/O is transferred between client and server by the RTRS transport | |
15 | modules. The administration of RNBD and RTRS modules is done via | |
16 | sysfs entries. | |
17 | ||
18 | Requirements | |
19 | ------------ | |
20 | ||
21 | RTRS kernel modules | |
22 | ||
23 | Quick Start | |
24 | ----------- | |
25 | ||
26 | Server side: | |
27 | # modprobe rnbd_server | |
28 | ||
29 | Client side: | |
30 | # modprobe rnbd_client | |
31 | # echo "sessname=blya path=ip:10.50.100.66 device_path=/dev/ram0" > \ | |
32 | /sys/devices/virtual/rnbd-client/ctl/map_device | |
33 | ||
34 | Where "sessname=" is a session name, a string to identify the session | |
35 | on client and on server sides; "path=" is a destination IP address or | |
36 | a pair of a source and a destination IPs, separated by comma. Multiple | |
37 | "path=" options can be specified in order to use multipath (see RTRS | |
38 | description for details); "device_path=" is the block device to be | |
39 | mapped from the server side. After the session to the server machine is | |
40 | established, the mapped device will appear on the client side under | |
41 | /dev/rnbd<N>. | |
42 | ||
43 | ||
44 | RNBD-Server Module Parameters | |
45 | ============================= | |
46 | ||
47 | dev_search_path | |
48 | --------------- | |
49 | ||
50 | When a device is mapped from the client, the server generates the path | |
51 | to the block device on the server side by concatenating dev_search_path | |
52 | and the "device_path" that was specified in the map_device operation. | |
53 | ||
54 | The default dev_search_path is: "/". | |
55 | ||
56 | dev_search_path option can also contain %SESSNAME% in order to provide | |
57 | different device namespaces for different sessions. See "device_path" | |
58 | option for details. | |
59 | ||
60 | ============================ | |
61 | Protocol (rnbd/rnbd-proto.h) | |
62 | ============================ | |
63 | ||
64 | 1. Before mapping first device from a given server, client sends an | |
65 | RNBD_MSG_SESS_INFO to the server. Server responds with | |
66 | RNBD_MSG_SESS_INFO_RSP. Currently the messages only contain the protocol | |
67 | version for backward compatibility. | |
68 | ||
69 | 2. Client requests to open a device by sending RNBD_MSG_OPEN message. This | |
70 | contains the path to the device and access mode (read-only or writable). | |
71 | Server responds to the message with RNBD_MSG_OPEN_RSP. This contains | |
72 | a 32 bit device id to be used for IOs and device "geometry" related | |
73 | information: side, max_hw_sectors, etc. | |
74 | ||
75 | 3. Client attaches RNBD_MSG_IO to each IO message send to a device. This | |
76 | message contains device id, provided by server in his rnbd_msg_open_rsp, | |
77 | sector to be accessed, read-write flags and bi_size. | |
78 | ||
79 | 4. Client closes a device by sending RNBD_MSG_CLOSE which contains only the | |
80 | device id provided by the server. | |
81 | ||
82 | ========================================= | |
83 | Contributors List(in alphabetical order) | |
84 | ========================================= | |
85 | Danil Kipnis <danil.kipnis@profitbricks.com> | |
86 | Fabian Holler <mail@fholler.de> | |
87 | Guoqing Jiang <guoqing.jiang@cloud.ionos.com> | |
88 | Jack Wang <jinpu.wang@profitbricks.com> | |
89 | Kleber Souza <kleber.souza@profitbricks.com> | |
90 | Lutz Pogrell <lutz.pogrell@cloud.ionos.com> | |
91 | Milind Dumbare <Milind.dumbare@gmail.com> | |
92 | Roman Penyaev <roman.penyaev@profitbricks.com> | |
ef8048dd | 93 | Swapnil Ingle <ingleswapnil@gmail.com> |