Commit | Line | Data |
---|---|---|
aeb04e52 MCC |
1 | RAID arrays |
2 | =========== | |
1da177e4 LT |
3 | |
4 | Boot time assembly of RAID arrays | |
5 | --------------------------------- | |
6 | ||
aeb04e52 | 7 | Tools that manage md devices can be found at |
93431e06 | 8 | https://www.kernel.org/pub/linux/utils/raid/ |
aeb04e52 MCC |
9 | |
10 | ||
1da177e4 LT |
11 | You can boot with your md device with the following kernel command |
12 | lines: | |
13 | ||
aeb04e52 MCC |
14 | for old raid arrays without persistent superblocks:: |
15 | ||
1da177e4 LT |
16 | md=<md device no.>,<raid level>,<chunk size factor>,<fault level>,dev0,dev1,...,devn |
17 | ||
aeb04e52 MCC |
18 | for raid arrays with persistent superblocks:: |
19 | ||
1da177e4 | 20 | md=<md device no.>,dev0,dev1,...,devn |
aeb04e52 MCC |
21 | |
22 | or, to assemble a partitionable array:: | |
23 | ||
1da177e4 | 24 | md=d<md device no.>,dev0,dev1,...,devn |
aeb04e52 MCC |
25 | |
26 | ``md device no.`` | |
27 | +++++++++++++++++ | |
28 | ||
29 | The number of the md device | |
30 | ||
31 | ================= ========= | |
32 | ``md device no.`` device | |
33 | ================= ========= | |
34 | 0 md0 | |
35 | 1 md1 | |
36 | 2 md2 | |
37 | 3 md3 | |
38 | 4 md4 | |
39 | ================= ========= | |
40 | ||
41 | ``raid level`` | |
42 | ++++++++++++++ | |
43 | ||
44 | level of the RAID array | |
45 | ||
46 | =============== ============= | |
47 | ``raid level`` level | |
48 | =============== ============= | |
49 | -1 linear mode | |
50 | 0 striped mode | |
51 | =============== ============= | |
52 | ||
53 | other modes are only supported with persistent super blocks | |
54 | ||
55 | ``chunk size factor`` | |
56 | +++++++++++++++++++++ | |
57 | ||
58 | (raid-0 and raid-1 only) | |
59 | ||
60 | Set the chunk size as 4k << n. | |
61 | ||
62 | ``fault level`` | |
63 | +++++++++++++++ | |
64 | ||
65 | Totally ignored | |
66 | ||
67 | ``dev0`` to ``devn`` | |
68 | ++++++++++++++++++++ | |
69 | ||
70 | e.g. ``/dev/hda1``, ``/dev/hdc1``, ``/dev/sda1``, ``/dev/sdb1`` | |
71 | ||
72 | A possible loadlin line (Harald Hoyer <HarryH@Royal.Net>) looks like this:: | |
73 | ||
74 | e:\loadlin\loadlin e:\zimage root=/dev/md0 md=0,0,4,0,/dev/hdb2,/dev/hdc3 ro | |
1da177e4 LT |
75 | |
76 | ||
77 | Boot time autodetection of RAID arrays | |
78 | -------------------------------------- | |
79 | ||
80 | When md is compiled into the kernel (not as module), partitions of | |
81 | type 0xfd are scanned and automatically assembled into RAID arrays. | |
82 | This autodetection may be suppressed with the kernel parameter | |
aeb04e52 | 83 | ``raid=noautodetect``. As of kernel 2.6.9, only drives with a type 0 |
1da177e4 LT |
84 | superblock can be autodetected and run at boot time. |
85 | ||
aeb04e52 | 86 | The kernel parameter ``raid=partitionable`` (or ``raid=part``) means |
1da177e4 LT |
87 | that all auto-detected arrays are assembled as partitionable. |
88 | ||
6ff8d8ec N |
89 | Boot time assembly of degraded/dirty arrays |
90 | ------------------------------------------- | |
91 | ||
92 | If a raid5 or raid6 array is both dirty and degraded, it could have | |
93 | undetectable data corruption. This is because the fact that it is | |
aeb04e52 | 94 | ``dirty`` means that the parity cannot be trusted, and the fact that it |
6ff8d8ec N |
95 | is degraded means that some datablocks are missing and cannot reliably |
96 | be reconstructed (due to no parity). | |
97 | ||
98 | For this reason, md will normally refuse to start such an array. This | |
99 | requires the sysadmin to take action to explicitly start the array | |
aeb04e52 MCC |
100 | despite possible corruption. This is normally done with:: |
101 | ||
6ff8d8ec N |
102 | mdadm --assemble --force .... |
103 | ||
104 | This option is not really available if the array has the root | |
105 | filesystem on it. In order to support this booting from such an | |
aeb04e52 | 106 | array, md supports a module parameter ``start_dirty_degraded`` which, |
6ff8d8ec N |
107 | when set to 1, bypassed the checks and will allows dirty degraded |
108 | arrays to be started. | |
109 | ||
aeb04e52 | 110 | So, to boot with a root filesystem of a dirty degraded raid 5 or 6, use:: |
6ff8d8ec N |
111 | |
112 | md-mod.start_dirty_degraded=1 | |
113 | ||
1da177e4 LT |
114 | |
115 | Superblock formats | |
116 | ------------------ | |
117 | ||
118 | The md driver can support a variety of different superblock formats. | |
aeb04e52 | 119 | Currently, it supports superblock formats ``0.90.0`` and the ``md-1`` format |
1da177e4 LT |
120 | introduced in the 2.5 development series. |
121 | ||
122 | The kernel will autodetect which format superblock is being used. | |
123 | ||
aeb04e52 | 124 | Superblock format ``0`` is treated differently to others for legacy |
1da177e4 LT |
125 | reasons - it is the original superblock format. |
126 | ||
127 | ||
128 | General Rules - apply for all superblock formats | |
129 | ------------------------------------------------ | |
130 | ||
aeb04e52 | 131 | An array is ``created`` by writing appropriate superblocks to all |
1da177e4 LT |
132 | devices. |
133 | ||
aeb04e52 | 134 | It is ``assembled`` by associating each of these devices with an |
1da177e4 LT |
135 | particular md virtual device. Once it is completely assembled, it can |
136 | be accessed. | |
137 | ||
138 | An array should be created by a user-space tool. This will write | |
139 | superblocks to all devices. It will usually mark the array as | |
aeb04e52 MCC |
140 | ``unclean``, or with some devices missing so that the kernel md driver |
141 | can create appropriate redundancy (copying in raid 1, parity | |
142 | calculation in raid 4/5). | |
1da177e4 LT |
143 | |
144 | When an array is assembled, it is first initialized with the | |
145 | SET_ARRAY_INFO ioctl. This contains, in particular, a major and minor | |
146 | version number. The major version number selects which superblock | |
147 | format is to be used. The minor number might be used to tune handling | |
148 | of the format, such as suggesting where on each device to look for the | |
149 | superblock. | |
150 | ||
151 | Then each device is added using the ADD_NEW_DISK ioctl. This | |
152 | provides, in particular, a major and minor number identifying the | |
153 | device to add. | |
154 | ||
155 | The array is started with the RUN_ARRAY ioctl. | |
156 | ||
157 | Once started, new devices can be added. They should have an | |
b6fec069 | 158 | appropriate superblock written to them, and then be passed in with |
1da177e4 LT |
159 | ADD_NEW_DISK. |
160 | ||
161 | Devices that have failed or are not yet active can be detached from an | |
162 | array using HOT_REMOVE_DISK. | |
163 | ||
164 | ||
aeb04e52 MCC |
165 | Specific Rules that apply to format-0 super block arrays, and arrays with no superblock (non-persistent) |
166 | -------------------------------------------------------------------------------------------------------- | |
1da177e4 | 167 | |
aeb04e52 MCC |
168 | An array can be ``created`` by describing the array (level, chunksize |
169 | etc) in a SET_ARRAY_INFO ioctl. This must have ``major_version==0`` and | |
170 | ``raid_disks != 0``. | |
1da177e4 LT |
171 | |
172 | Then uninitialized devices can be added with ADD_NEW_DISK. The | |
173 | structure passed to ADD_NEW_DISK must specify the state of the device | |
a33f3224 | 174 | and its role in the array. |
1da177e4 LT |
175 | |
176 | Once started with RUN_ARRAY, uninitialized spares can be added with | |
177 | HOT_ADD_DISK. | |
bb636547 N |
178 | |
179 | ||
bb636547 N |
180 | MD devices in sysfs |
181 | ------------------- | |
aeb04e52 MCC |
182 | |
183 | md devices appear in sysfs (``/sys``) as regular block devices, | |
184 | e.g.:: | |
185 | ||
bb636547 N |
186 | /sys/block/md0 |
187 | ||
aeb04e52 | 188 | Each ``md`` device will contain a subdirectory called ``md`` which |
bb636547 N |
189 | contains further md-specific information about the device. |
190 | ||
191 | All md devices contain: | |
aeb04e52 | 192 | |
bb636547 | 193 | level |
aeb04e52 | 194 | a text file indicating the ``raid level``. e.g. raid0, raid1, |
d33a56d3 | 195 | raid5, linear, multipath, faulty. |
bb636547 | 196 | If no raid level has been set yet (array is still being |
d33a56d3 N |
197 | assembled), the value will reflect whatever has been written |
198 | to it, which may be a name like the above, or may be a number | |
aeb04e52 | 199 | such as ``0``, ``5``, etc. |
bb636547 N |
200 | |
201 | raid_disks | |
202 | a text file with a simple number indicating the number of devices | |
203 | in a fully functional array. If this is not yet known, the file | |
11373542 N |
204 | will be empty. If an array is being resized this will contain |
205 | the new number of devices. | |
206 | Some raid levels allow this value to be set while the array is | |
207 | active. This will reconfigure the array. Otherwise it can only | |
208 | be set while assembling an array. | |
209 | A change to this attribute will not be permitted if it would | |
210 | reduce the size of the array. To reduce the number of drives | |
211 | in an e.g. raid5, the array size must first be reduced by | |
aeb04e52 | 212 | setting the ``array_size`` attribute. |
bb636547 | 213 | |
3b34380a | 214 | chunk_size |
aeb04e52 | 215 | This is the size in bytes for ``chunks`` and is only relevant to |
11373542 | 216 | raid levels that involve striping (0,4,5,6,10). The address space |
3b34380a N |
217 | of the array is conceptually divided into chunks and consecutive |
218 | chunks are striped onto neighbouring devices. | |
3f6dee9b | 219 | The size should be at least PAGE_SIZE (4k) and should be a power |
3b34380a N |
220 | of 2. This can only be set while assembling an array |
221 | ||
08a02ecd | 222 | layout |
aeb04e52 | 223 | The ``layout`` for the array for the particular level. This is |
751d5b27 | 224 | simply a number that is interpreted differently by different |
08a02ecd N |
225 | levels. It can be written while assembling an array. |
226 | ||
11373542 N |
227 | array_size |
228 | This can be used to artificially constrain the available space in | |
229 | the array to be less than is actually available on the combined | |
230 | devices. Writing a number (in Kilobytes) which is less than | |
231 | the available size will set the size. Any reconfiguration of the | |
232 | array (e.g. adding devices) will not cause the size to change. | |
aeb04e52 | 233 | Writing the word ``default`` will cause the effective size of the |
11373542 | 234 | array to be whatever size is actually available based on |
aeb04e52 | 235 | ``level``, ``chunk_size`` and ``component_size``. |
11373542 N |
236 | |
237 | This can be used to reduce the size of the array before reducing | |
238 | the number of devices in a raid4/5/6, or to support external | |
239 | metadata formats which mandate such clipping. | |
240 | ||
08a02ecd | 241 | reshape_position |
aeb04e52 MCC |
242 | This is either ``none`` or a sector number within the devices of |
243 | the array where ``reshape`` is up to. If this is set, the three | |
08a02ecd N |
244 | attributes mentioned above (raid_disks, chunk_size, layout) can |
245 | potentially have 2 values, an old and a new value. If these | |
aeb04e52 MCC |
246 | values differ, reading the attribute returns:: |
247 | ||
08a02ecd | 248 | new (old) |
aeb04e52 MCC |
249 | |
250 | and writing will effect the ``new`` value, leaving the ``old`` | |
08a02ecd N |
251 | unchanged. |
252 | ||
a35b0d69 N |
253 | component_size |
254 | For arrays with data redundancy (i.e. not raid0, linear, faulty, | |
255 | multipath), all components must be the same size - or at least | |
256 | there must a size that they all provide space for. This is a key | |
257 | part or the geometry of the array. It is measured in sectors | |
258 | and can be read from here. Writing to this value may resize | |
259 | the array if the personality supports it (raid1, raid5, raid6), | |
260 | and if the component drives are large enough. | |
261 | ||
8bb93aac N |
262 | metadata_version |
263 | This indicates the format that is being used to record metadata | |
264 | about the array. It can be 0.90 (traditional format), 1.0, 1.1, | |
aeb04e52 | 265 | 1.2 (newer format in varying locations) or ``none`` indicating that |
8bb93aac | 266 | the kernel isn't managing metadata at all. |
aeb04e52 | 267 | Alternately it can be ``external:`` followed by a string which |
11373542 N |
268 | is set by user-space. This indicates that metadata is managed |
269 | by a user-space program. Any device failure or other event that | |
270 | requires a metadata update will cause array activity to be | |
271 | suspended until the event is acknowledged. | |
8bb93aac | 272 | |
a94213b1 N |
273 | resync_start |
274 | The point at which resync should start. If no resync is needed, | |
aeb04e52 | 275 | this will be a very large number (or ``none`` since 2.6.30-rc1). At |
06e3c817 | 276 | array creation it will default to 0, though starting the array as |
aeb04e52 | 277 | ``clean`` will set it much larger. |
a94213b1 | 278 | |
664aed04 | 279 | new_dev |
6d7ff738 N |
280 | This file can be written but not read. The value written should |
281 | be a block device number as major:minor. e.g. 8:0 | |
282 | This will cause that device to be attached to the array, if it is | |
283 | available. It will then appear at md/dev-XXX (depending on the | |
284 | name of the device) and further configuration is then possible. | |
285 | ||
664aed04 | 286 | safe_mode_delay |
16f17b39 | 287 | When an md array has seen no write requests for a certain period |
aeb04e52 MCC |
288 | of time, it will be marked as ``clean``. When another write |
289 | request arrives, the array is marked as ``dirty`` before the write | |
290 | commences. This is known as ``safe_mode``. | |
291 | The ``certain period`` is controlled by this file which stores the | |
16f17b39 N |
292 | period as a number of seconds. The default is 200msec (0.200). |
293 | Writing a value of 0 disables safemode. | |
294 | ||
664aed04 | 295 | array_state |
9e653b63 N |
296 | This file contains a single word which describes the current |
297 | state of the array. In many cases, the state can be set by | |
298 | writing the word for the desired state, however some states | |
299 | cannot be explicitly set, and some transitions are not allowed. | |
300 | ||
0fd62b86 | 301 | Select/poll works on this file. All changes except between |
aeb04e52 MCC |
302 | Active_idle and active (which can be frequent and are not |
303 | very interesting) are notified. active->active_idle is | |
304 | reported if the metadata is externally managed. | |
0fd62b86 | 305 | |
9e653b63 N |
306 | clear |
307 | No devices, no size, no level | |
aeb04e52 | 308 | |
9e653b63 | 309 | Writing is equivalent to STOP_ARRAY ioctl |
aeb04e52 | 310 | |
9e653b63 N |
311 | inactive |
312 | May have some settings, but array is not active | |
aeb04e52 MCC |
313 | all IO results in error |
314 | ||
9e653b63 | 315 | When written, doesn't tear down array, but just stops it |
aeb04e52 | 316 | |
9e653b63 N |
317 | suspended (not supported yet) |
318 | All IO requests will block. The array can be reconfigured. | |
aeb04e52 | 319 | |
9e653b63 | 320 | Writing this, if accepted, will block until array is quiessent |
aeb04e52 | 321 | |
9e653b63 N |
322 | readonly |
323 | no resync can happen. no superblocks get written. | |
aeb04e52 MCC |
324 | |
325 | Write requests fail | |
326 | ||
9e653b63 | 327 | read-auto |
aeb04e52 MCC |
328 | like readonly, but behaves like ``clean`` on a write request. |
329 | ||
330 | clean | |
331 | no pending writes, but otherwise active. | |
9e653b63 | 332 | |
9e653b63 | 333 | When written to inactive array, starts without resync |
aeb04e52 | 334 | |
9e653b63 | 335 | If a write request arrives then |
aeb04e52 MCC |
336 | if metadata is known, mark ``dirty`` and switch to ``active``. |
337 | if not known, block and switch to write-pending | |
338 | ||
9e653b63 N |
339 | If written to an active array that has pending writes, then fails. |
340 | active | |
341 | fully active: IO and resync can be happening. | |
342 | When written to inactive array, starts with resync | |
343 | ||
344 | write-pending | |
aeb04e52 | 345 | clean, but writes are blocked waiting for ``active`` to be written. |
9e653b63 N |
346 | |
347 | active-idle | |
348 | like active, but no writes have been seen for a while (safe_mode_delay). | |
349 | ||
43a70507 N |
350 | bitmap/location |
351 | This indicates where the write-intent bitmap for the array is | |
352 | stored. | |
aeb04e52 MCC |
353 | |
354 | It can be one of ``none``, ``file`` or ``[+-]N``. | |
355 | ``file`` may later be extended to ``file:/file/name`` | |
356 | ``[+-]N`` means that many sectors from the start of the metadata. | |
357 | ||
358 | This is replicated on all devices. For arrays with externally | |
359 | managed metadata, the offset is from the beginning of the | |
360 | device. | |
361 | ||
43a70507 N |
362 | bitmap/chunksize |
363 | The size, in bytes, of the chunk which will be represented by a | |
364 | single bit. For RAID456, it is a portion of an individual | |
365 | device. For RAID10, it is a portion of the array. For RAID1, it | |
366 | is both (they come to the same thing). | |
aeb04e52 | 367 | |
43a70507 N |
368 | bitmap/time_base |
369 | The time, in seconds, between looking for bits in the bitmap to | |
370 | be cleared. In the current implementation, a bit will be cleared | |
aeb04e52 | 371 | between 2 and 3 times ``time_base`` after all the covered blocks |
43a70507 | 372 | are known to be in-sync. |
aeb04e52 | 373 | |
43a70507 N |
374 | bitmap/backlog |
375 | When write-mostly devices are active in a RAID1, write requests | |
376 | to those devices proceed in the background - the filesystem (or | |
377 | other user of the device) does not have to wait for them. | |
aeb04e52 | 378 | ``backlog`` sets a limit on the number of concurrent background |
43a70507 N |
379 | writes. If there are more than this, new writes will by |
380 | synchronous. | |
aeb04e52 | 381 | |
ece5cff0 | 382 | bitmap/metadata |
aeb04e52 MCC |
383 | This can be either ``internal`` or ``external``. |
384 | ||
385 | ``internal`` | |
386 | is the default and means the metadata for the bitmap | |
387 | is stored in the first 256 bytes of the allocated space and is | |
388 | managed by the md module. | |
389 | ||
390 | ``external`` | |
391 | means that bitmap metadata is managed externally to | |
392 | the kernel (i.e. by some userspace program) | |
393 | ||
ece5cff0 | 394 | bitmap/can_clear |
aeb04e52 | 395 | This is either ``true`` or ``false``. If ``true``, then bits in the |
ece5cff0 | 396 | bitmap will be cleared when the corresponding blocks are thought |
aeb04e52 MCC |
397 | to be in-sync. If ``false``, bits will never be cleared. |
398 | This is automatically set to ``false`` if a write happens on a | |
ece5cff0 N |
399 | degraded array, or if the array becomes degraded during a write. |
400 | When metadata is managed externally, it should be set to true | |
401 | once the array becomes non-degraded, and this fact has been | |
402 | recorded in the metadata. | |
9e653b63 | 403 | |
664aed04 AP |
404 | consistency_policy |
405 | This indicates how the array maintains consistency in case of unexpected | |
406 | shutdown. It can be: | |
aeb04e52 | 407 | |
664aed04 AP |
408 | none |
409 | Array has no redundancy information, e.g. raid0, linear. | |
410 | ||
411 | resync | |
412 | Full resync is performed and all redundancy is regenerated when the | |
413 | array is started after unclean shutdown. | |
414 | ||
415 | bitmap | |
416 | Resync assisted by a write-intent bitmap. | |
417 | ||
418 | journal | |
419 | For raid4/5/6, journal device is used to log transactions and replay | |
420 | after unclean shutdown. | |
421 | ||
422 | ppl | |
423 | For raid5 only, Partial Parity Log is used to close the write hole and | |
424 | eliminate resync. | |
425 | ||
426 | The accepted values when writing to this file are ``ppl`` and ``resync``, | |
427 | used to enable and disable PPL. | |
aeb04e52 | 428 | |
ec164d07 SP |
429 | uuid |
430 | This indicates the UUID of the array in the following format: | |
431 | xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | |
432 | ||
aeb04e52 MCC |
433 | |
434 | As component devices are added to an md array, they appear in the ``md`` | |
435 | directory as new directories named:: | |
436 | ||
bb636547 | 437 | dev-XXX |
aeb04e52 MCC |
438 | |
439 | where ``XXX`` is a name that the kernel knows for the device, e.g. hdb1. | |
bb636547 N |
440 | Each directory contains: |
441 | ||
442 | block | |
aeb04e52 MCC |
443 | a symlink to the block device in /sys/block, e.g.:: |
444 | ||
bb636547 N |
445 | /sys/block/md0/md/dev-hdb1/block -> ../../../../block/hdb/hdb1 |
446 | ||
447 | super | |
448 | A file containing an image of the superblock read from, or | |
449 | written to, that device. | |
450 | ||
451 | state | |
2d78f8c4 | 452 | A file recording the current state of the device in the array |
aeb04e52 MCC |
453 | which can be a comma separated list of: |
454 | ||
455 | faulty | |
456 | device has been kicked from active use due to | |
457 | a detected fault, or it has unacknowledged bad | |
458 | blocks | |
459 | ||
460 | in_sync | |
461 | device is a fully in-sync member of the array | |
462 | ||
463 | writemostly | |
464 | device will only be subject to read | |
465 | requests if there are no other options. | |
466 | ||
467 | This applies only to raid1 arrays. | |
468 | ||
469 | blocked | |
470 | device has failed, and the failure hasn't been | |
471 | acknowledged yet by the metadata handler. | |
472 | ||
473 | Writes that would write to this device if | |
474 | it were not faulty are blocked. | |
475 | ||
476 | spare | |
477 | device is working, but not a full member. | |
478 | ||
479 | This includes spares that are in the process | |
480 | of being recovered to | |
481 | ||
482 | write_error | |
483 | device has ever seen a write error. | |
484 | ||
485 | want_replacement | |
486 | device is (mostly) working but probably | |
487 | should be replaced, either due to errors or | |
488 | due to user request. | |
489 | ||
490 | replacement | |
491 | device is a replacement for another active | |
492 | device with same raid_disk. | |
2d78f8c4 N |
493 | |
494 | ||
d6bc8ac9 | 495 | This list may grow in future. |
aeb04e52 | 496 | |
45dc2de1 | 497 | This can be written to. |
aeb04e52 MCC |
498 | |
499 | Writing ``faulty`` simulates a failure on the device. | |
500 | ||
501 | Writing ``remove`` removes the device from the array. | |
502 | ||
503 | Writing ``writemostly`` sets the writemostly flag. | |
504 | ||
505 | Writing ``-writemostly`` clears the writemostly flag. | |
506 | ||
507 | Writing ``blocked`` sets the ``blocked`` flag. | |
508 | ||
509 | Writing ``-blocked`` clears the ``blocked`` flags and allows writes | |
510 | to complete and possibly simulates an error. | |
511 | ||
512 | Writing ``in_sync`` sets the in_sync flag. | |
513 | ||
514 | Writing ``write_error`` sets writeerrorseen flag. | |
515 | ||
516 | Writing ``-write_error`` clears writeerrorseen flag. | |
517 | ||
518 | Writing ``want_replacement`` is allowed at any time except to a | |
519 | replacement device or a spare. It sets the flag. | |
520 | ||
521 | Writing ``-want_replacement`` is allowed at any time. It clears | |
522 | the flag. | |
523 | ||
524 | Writing ``replacement`` or ``-replacement`` is only allowed before | |
525 | starting the array. It sets or clears the flag. | |
526 | ||
527 | ||
528 | This file responds to select/poll. Any change to ``faulty`` | |
529 | or ``blocked`` causes an event. | |
bb636547 | 530 | |
4dbcdc75 N |
531 | errors |
532 | An approximate count of read errors that have been detected on | |
533 | this device but have not caused the device to be evicted from | |
534 | the array (either because they were corrected or because they | |
535 | happened while the array was read-only). When using version-1 | |
536 | metadata, this value persists across restarts of the array. | |
537 | ||
538 | This value can be written while assembling an array thus | |
539 | providing an ongoing count for arrays with metadata managed by | |
540 | userspace. | |
541 | ||
014236d2 N |
542 | slot |
543 | This gives the role that the device has in the array. It will | |
aeb04e52 | 544 | either be ``none`` if the device is not active in the array |
014236d2 | 545 | (i.e. is a spare or has failed) or an integer less than the |
aeb04e52 | 546 | ``raid_disks`` number for the array indicating which position |
014236d2 N |
547 | it currently fills. This can only be set while assembling an |
548 | array. A device for which this is set is assumed to be working. | |
549 | ||
93c8cad0 N |
550 | offset |
551 | This gives the location in the device (in sectors from the | |
552 | start) where data from the array will be stored. Any part of | |
b6fec069 | 553 | the device before this offset is not touched, unless it is |
93c8cad0 N |
554 | used for storing metadata (Formats 1.1 and 1.2). |
555 | ||
83303b61 N |
556 | size |
557 | The amount of the device, after the offset, that can be used | |
558 | for storage of data. This will normally be the same as the | |
559 | component_size. This can be written while assembling an | |
560 | array. If a value less than the current component_size is | |
d7027458 | 561 | written, it will be rejected. |
83303b61 | 562 | |
06e3c817 | 563 | recovery_start |
aeb04e52 | 564 | When the device is not ``in_sync``, this records the number of |
06e3c817 DW |
565 | sectors from the start of the device which are known to be |
566 | correct. This is normally zero, but during a recovery | |
b6fec069 | 567 | operation it will steadily increase, and if the recovery is |
06e3c817 DW |
568 | interrupted, restoring this value can cause recovery to |
569 | avoid repeating the earlier blocks. With v1.x metadata, this | |
570 | value is saved and restored automatically. | |
571 | ||
572 | This can be set whenever the device is not an active member of | |
573 | the array, either before the array is activated, or before | |
aeb04e52 MCC |
574 | the ``slot`` is set. |
575 | ||
576 | Setting this to ``none`` is equivalent to setting ``in_sync``. | |
577 | Setting to any other value also clears the ``in_sync`` flag. | |
06e3c817 | 578 | |
6e0d2d03 NK |
579 | bad_blocks |
580 | This gives the list of all known bad blocks in the form of | |
581 | start address and length (in sectors respectively). If output | |
582 | is too big to fit in a page, it will be truncated. Writing | |
aeb04e52 | 583 | ``sector length`` to this file adds new acknowledged (i.e. |
6e0d2d03 NK |
584 | recorded to disk safely) bad blocks. |
585 | ||
586 | unacknowledged_bad_blocks | |
587 | This gives the list of known-but-not-yet-saved-to-disk bad | |
aeb04e52 | 588 | blocks in the same form of ``bad_blocks``. If output is too big |
6e0d2d03 NK |
589 | to fit in a page, it will be truncated. Writing to this file |
590 | adds bad blocks without acknowledging them. This is largely | |
591 | for testing. | |
592 | ||
664aed04 AP |
593 | ppl_sector, ppl_size |
594 | Location and size (in sectors) of the space used for Partial Parity Log | |
595 | on this device. | |
06e3c817 | 596 | |
bb636547 | 597 | |
b6fec069 | 598 | An active md device will also contain an entry for each active device |
aeb04e52 | 599 | in the array. These are named:: |
bb636547 N |
600 | |
601 | rdNN | |
602 | ||
aeb04e52 | 603 | where ``NN`` is the position in the array, starting from 0. |
bb636547 | 604 | So for a 3 drive array there will be rd0, rd1, rd2. |
aeb04e52 MCC |
605 | These are symbolic links to the appropriate ``dev-XXX`` entry. |
606 | Thus, for example:: | |
607 | ||
bb636547 | 608 | cat /sys/block/md*/md/rd*/state |
aeb04e52 MCC |
609 | |
610 | will show ``in_sync`` on every line. | |
bb636547 N |
611 | |
612 | ||
613 | ||
b6fec069 | 614 | Active md devices for levels that support data redundancy (1,4,5,6,10) |
bb636547 N |
615 | also have |
616 | ||
617 | sync_action | |
618 | a text file that can be used to monitor and control the rebuild | |
619 | process. It contains one word which can be one of: | |
aeb04e52 MCC |
620 | |
621 | resync | |
622 | redundancy is being recalculated after unclean | |
623 | shutdown or creation | |
624 | ||
625 | recover | |
626 | a hot spare is being built to replace a | |
627 | failed/missing device | |
628 | ||
629 | idle | |
630 | nothing is happening | |
631 | check | |
632 | A full check of redundancy was requested and is | |
633 | happening. This reads all blocks and checks | |
634 | them. A repair may also happen for some raid | |
635 | levels. | |
636 | ||
637 | repair | |
638 | A full check and repair is happening. This is | |
639 | similar to ``resync``, but was requested by the | |
640 | user, and the write-intent bitmap is NOT used to | |
641 | optimise the process. | |
bb636547 N |
642 | |
643 | This file is writable, and each of the strings that could be | |
644 | read are meaningful for writing. | |
645 | ||
aeb04e52 MCC |
646 | ``idle`` will stop an active resync/recovery etc. There is no |
647 | guarantee that another resync/recovery may not be automatically | |
648 | started again, though some event will be needed to trigger | |
649 | this. | |
650 | ||
651 | ``resync`` or ``recovery`` can be used to restart the | |
652 | corresponding operation if it was stopped with ``idle``. | |
653 | ||
654 | ``check`` and ``repair`` will start the appropriate process | |
655 | providing the current state is ``idle``. | |
bb636547 | 656 | |
72a23c21 NB |
657 | This file responds to select/poll. Any important change in the value |
658 | triggers a poll event. Sometimes the value will briefly be | |
aeb04e52 MCC |
659 | ``recover`` if a recovery seems to be needed, but cannot be |
660 | achieved. In that case, the transition to ``recover`` isn't | |
72a23c21 NB |
661 | notified, but the transition away is. |
662 | ||
a99ac971 NB |
663 | degraded |
664 | This contains a count of the number of devices by which the | |
aeb04e52 MCC |
665 | arrays is degraded. So an optimal array will show ``0``. A |
666 | single failed/missing drive will show ``1``, etc. | |
667 | ||
a99ac971 NB |
668 | This file responds to select/poll, any increase or decrease |
669 | in the count of missing devices will trigger an event. | |
670 | ||
bb636547 | 671 | mismatch_count |
aeb04e52 MCC |
672 | When performing ``check`` and ``repair``, and possibly when |
673 | performing ``resync``, md will count the number of errors that are | |
674 | found. The count in ``mismatch_cnt`` is the number of sectors | |
675 | that were re-written, or (for ``check``) would have been | |
bb636547 | 676 | re-written. As most raid levels work in units of pages rather |
c79a8d85 | 677 | than sectors, this may be larger than the number of actual errors |
bb636547 N |
678 | by a factor of the number of sectors in a page. |
679 | ||
9b1d1dac PC |
680 | bitmap_set_bits |
681 | If the array has a write-intent bitmap, then writing to this | |
682 | attribute can set bits in the bitmap, indicating that a resync | |
683 | would need to check the corresponding blocks. Either individual | |
684 | numbers or start-end pairs can be written. Multiple numbers | |
685 | can be separated by a space. | |
aeb04e52 MCC |
686 | |
687 | Note that the numbers are ``bit`` numbers, not ``block`` numbers. | |
9b1d1dac PC |
688 | They should be scaled by the bitmap_chunksize. |
689 | ||
aeb04e52 MCC |
690 | sync_speed_min, sync_speed_max |
691 | This are similar to ``/proc/sys/dev/raid/speed_limit_{min,max}`` | |
08a02ecd | 692 | however they only apply to the particular array. |
aeb04e52 MCC |
693 | |
694 | If no value has been written to these, or if the word ``system`` | |
08a02ecd N |
695 | is written, then the system-wide value is used. If a value, |
696 | in kibibytes-per-second is written, then it is used. | |
aeb04e52 | 697 | |
08a02ecd | 698 | When the files are read, they show the currently active value |
aeb04e52 | 699 | followed by ``(local)`` or ``(system)`` depending on whether it is |
08a02ecd N |
700 | a locally set or system-wide value. |
701 | ||
702 | sync_completed | |
703 | This shows the number of sectors that have been completed of | |
704 | whatever the current sync_action is, followed by the number of | |
705 | sectors in total that could need to be processed. The two | |
aeb04e52 | 706 | numbers are separated by a ``/`` thus effectively showing one |
08a02ecd | 707 | value, a fraction of the process that is complete. |
aeb04e52 MCC |
708 | |
709 | A ``select`` on this attribute will return when resync completes, | |
c6207277 N |
710 | when it reaches the current sync_max (below) and possibly at |
711 | other times. | |
712 | ||
08a02ecd N |
713 | sync_speed |
714 | This shows the current actual speed, in K/sec, of the current | |
715 | sync_action. It is averaged over the last 30 seconds. | |
716 | ||
aeb04e52 | 717 | suspend_lo, suspend_hi |
08a02ecd N |
718 | The two values, given as numbers of sectors, indicate a range |
719 | within the array where IO will be blocked. This is currently | |
720 | only supported for raid4/5/6. | |
721 | ||
aeb04e52 | 722 | sync_min, sync_max |
28a83978 | 723 | The two values, given as numbers of sectors, indicate a range |
aeb04e52 MCC |
724 | within the array where ``check``/``repair`` will operate. Must be |
725 | a multiple of chunk_size. When it reaches ``sync_max`` it will | |
28a83978 | 726 | pause, rather than complete. |
aeb04e52 | 727 | You can use ``select`` or ``poll`` on ``sync_completed`` to wait for |
28a83978 | 728 | that number to reach sync_max. Then you can either increase |
aeb04e52 | 729 | ``sync_max``, or can write ``idle`` to ``sync_action``. |
28a83978 | 730 | |
aeb04e52 | 731 | The value of ``max`` for ``sync_max`` effectively disables the limit. |
0baac4db C |
732 | When a resync is active, the value can only ever be increased, |
733 | never decreased. | |
aeb04e52 | 734 | The value of ``0`` is the minimum for ``sync_min``. |
0baac4db C |
735 | |
736 | ||
08a02ecd | 737 | |
bb636547 N |
738 | Each active md device may also have attributes specific to the |
739 | personality module that manages it. | |
740 | These are specific to the implementation of the module and could | |
741 | change substantially if the implementation changes. | |
742 | ||
aeb04e52 | 743 | These currently include: |
bb636547 N |
744 | |
745 | stripe_cache_size (currently raid5 only) | |
746 | number of entries in the stripe cache. This is writable, but | |
a37376f3 | 747 | there are upper and lower limits (32768, 17). Default is 256. |
aeb04e52 | 748 | |
bb636547 N |
749 | strip_cache_active (currently raid5 only) |
750 | number of active entries in the stripe cache | |
aeb04e52 | 751 | |
8b3e6cdc DW |
752 | preread_bypass_threshold (currently raid5 only) |
753 | number of times a stripe requiring preread will be bypassed by | |
754 | a stripe that does not require preread. For fairness defaults | |
755 | to 1. Setting this to 0 disables bypass accounting and | |
756 | requires preread stripes to wait until all full-width stripe- | |
757 | writes are complete. Valid values are 0 to stripe_cache_size. | |
5a6265f9 SL |
758 | |
759 | journal_mode (currently raid5 only) | |
760 | The cache mode for raid5. raid5 could include an extra disk for | |
761 | caching. The mode can be "write-throuth" and "write-back". The | |
762 | default is "write-through". | |
a596d086 MD |
763 | |
764 | ppl_write_hint | |
765 | NVMe stream ID to be set for each PPL write request. |