Commit | Line | Data |
---|---|---|
18a5bf27 SB |
1 | Introduction |
2 | ============ | |
3 | ||
4 | The device-mapper "unstriped" target provides a transparent mechanism to | |
5 | unstripe a device-mapper "striped" target to access the underlying disks | |
6 | without having to touch the true backing block-device. It can also be | |
7 | used to unstripe a hardware RAID-0 to access backing disks. | |
8 | ||
9 | Parameters: | |
10 | <number of stripes> <chunk size> <stripe #> <dev_path> <offset> | |
11 | ||
12 | <number of stripes> | |
13 | The number of stripes in the RAID 0. | |
14 | ||
15 | <chunk size> | |
16 | The amount of 512B sectors in the chunk striping. | |
17 | ||
18 | <dev_path> | |
19 | The block device you wish to unstripe. | |
20 | ||
21 | <stripe #> | |
22 | The stripe number within the device that corresponds to physical | |
23 | drive you wish to unstripe. This must be 0 indexed. | |
24 | ||
25 | ||
26 | Why use this module? | |
27 | ==================== | |
28 | ||
29 | An example of undoing an existing dm-stripe | |
30 | ------------------------------------------- | |
31 | ||
32 | This small bash script will setup 4 loop devices and use the existing | |
33 | striped target to combine the 4 devices into one. It then will use | |
34 | the unstriped target ontop of the striped device to access the | |
35 | individual backing loop devices. We write data to the newly exposed | |
36 | unstriped devices and verify the data written matches the correct | |
37 | underlying device on the striped array. | |
38 | ||
39 | #!/bin/bash | |
40 | ||
41 | MEMBER_SIZE=$((128 * 1024 * 1024)) | |
42 | NUM=4 | |
43 | SEQ_END=$((${NUM}-1)) | |
44 | CHUNK=256 | |
45 | BS=4096 | |
46 | ||
47 | RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512)) | |
48 | DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}" | |
49 | COUNT=$((${MEMBER_SIZE} / ${BS})) | |
50 | ||
51 | for i in $(seq 0 ${SEQ_END}); do | |
52 | dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct | |
53 | losetup /dev/loop${i} member-${i} | |
54 | DM_PARMS+=" /dev/loop${i} 0" | |
55 | done | |
56 | ||
57 | echo $DM_PARMS | dmsetup create raid0 | |
58 | for i in $(seq 0 ${SEQ_END}); do | |
59 | echo "0 1 unstriped ${NUM} ${CHUNK} ${i} /dev/mapper/raid0 0" | dmsetup create set-${i} | |
60 | done; | |
61 | ||
62 | for i in $(seq 0 ${SEQ_END}); do | |
63 | dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct | |
64 | diff /dev/mapper/set-${i} member-${i} | |
65 | done; | |
66 | ||
67 | for i in $(seq 0 ${SEQ_END}); do | |
68 | dmsetup remove set-${i} | |
69 | done | |
70 | ||
71 | dmsetup remove raid0 | |
72 | ||
73 | for i in $(seq 0 ${SEQ_END}); do | |
74 | losetup -d /dev/loop${i} | |
75 | rm -f member-${i} | |
76 | done | |
77 | ||
78 | Another example | |
79 | --------------- | |
80 | ||
81 | Intel NVMe drives contain two cores on the physical device. | |
82 | Each core of the drive has segregated access to its LBA range. | |
83 | The current LBA model has a RAID 0 128k chunk on each core, resulting | |
84 | in a 256k stripe across the two cores: | |
85 | ||
86 | Core 0: Core 1: | |
87 | __________ __________ | |
88 | | LBA 512| | LBA 768| | |
89 | | LBA 0 | | LBA 256| | |
90 | ---------- ---------- | |
91 | ||
92 | The purpose of this unstriping is to provide better QoS in noisy | |
93 | neighbor environments. When two partitions are created on the | |
94 | aggregate drive without this unstriping, reads on one partition | |
95 | can affect writes on another partition. This is because the partitions | |
96 | are striped across the two cores. When we unstripe this hardware RAID 0 | |
97 | and make partitions on each new exposed device the two partitions are now | |
98 | physically separated. | |
99 | ||
100 | With the dm-unstriped target we're able to segregate an fio script that | |
101 | has read and write jobs that are independent of each other. Compared to | |
102 | when we run the test on a combined drive with partitions, we were able | |
103 | to get a 92% reduction in read latency using this device mapper target. | |
104 | ||
105 | ||
106 | Example dmsetup usage | |
107 | ===================== | |
108 | ||
109 | unstriped ontop of Intel NVMe device that has 2 cores | |
110 | ----------------------------------------------------- | |
111 | dmsetup create nvmset0 --table '0 512 unstriped 2 256 0 /dev/nvme0n1 0' | |
112 | dmsetup create nvmset1 --table '0 512 unstriped 2 256 1 /dev/nvme0n1 0' | |
113 | ||
114 | There will now be two devices that expose Intel NVMe core 0 and 1 | |
115 | respectively: | |
116 | /dev/mapper/nvmset0 | |
117 | /dev/mapper/nvmset1 | |
118 | ||
119 | unstriped ontop of striped with 4 drives using 128K chunk size | |
120 | -------------------------------------------------------------- | |
121 | dmsetup create raid_disk0 --table '0 512 unstriped 4 256 0 /dev/mapper/striped 0' | |
122 | dmsetup create raid_disk1 --table '0 512 unstriped 4 256 1 /dev/mapper/striped 0' | |
123 | dmsetup create raid_disk2 --table '0 512 unstriped 4 256 2 /dev/mapper/striped 0' | |
124 | dmsetup create raid_disk3 --table '0 512 unstriped 4 256 3 /dev/mapper/striped 0' |