13 Raid on Linux

13.1 Building a RAID set

Generally, system RAID sets will be built at install time but larger RAID's - such as a RAID 10 for a database - may be built later. In this examle I will build a RAID 10 using sda1, sdb1, sdc1, sde1, sdf1, and sdg1.

1. Check all the drives are partitioned. Repartition if necessary
   $ sudo fdisk -l 
   Disk /dev/sda: 146.8 GB, 146815733760 bytes 
   255 heads, 63 sectors/track, 17849 cylinders 
   Units = cylinders of 16065 * 512 = 8225280 bytes 
    
      Device Boot   Start    End   Blocks  Id  System 
   /dev/sda1  *        1  17849 143372061  fd  Linux raid autodetect  
   <etc>
2. Create the RAID1 devices
   $ sudo mdadm -create /dev/md11 -level=1 -chunk=1024 -raid-devices=2 /dev/sda1 /dev/sde1 
   $ sudo mdadm -create /dev/md12 -level=1 -chunk=1024 -raid-devices=2 /dev/sdb1 /dev/sdf1 
   $ sudo mdadm -create /dev/md13 -level=1 -chunk=1024 -raid-devices=2 /dev/sdc1 /dev/sdg1
3. Create the RAID0 across the above RAID1's to createa RAID10
   $ sudo mdadm -create /dev/md10 -level=0 -chunk=1024 -raid-devices=3 /dev/md11 /dev/md12 /dev/md13
4. Format the new device
   $ sudo mke2fs -j -L /mp001 /dev/md10
5. Generate a new mdadm.conf
   $ cd 
   $ echo 'DEVICE /dev/sd*[0-9]' >tt 
   $ sudo mdadm -examine -scan -config=tt >mdadm.conf 
   $ sudo cp mdadm.conf /etc/mdadm.conf

13.2 Replacing a Failed RAID1 Drive with a spare

By:

Stephen Carville

Rev:

09/01/2004

For illustration, assume drive /dev/sdh and /dev/sdi make up a raid, /dev/sdi has failed and will be replaced by drive /dev/sdj.

1. Determine which drive has failed. this information can be inferred from /proc/mdstat and mdadm
   $ cat /proc/mdstat
   $ sudo mdadm -D /dev/md0
2. If any partitions on the failed drive were not removed by the system, remove them
   $ sudo mdadm -r /dev/md1 /dev/sdi2
3. Partition the replacement drive like the failed drive. If any differences are necessary, make the new partitions bigger
4. If the replacement and failed drives have the same geometry, use sfdisk to duplicate it
   $ sudo sfdisk -d /dev/sdh >part 
   $ sudo sfdisk /dev/sdj <part
5. If the hardware allows hot swapping, swap the new drive and the failed drive. (This step can be skipped if desired but be sure to adjust drive names below).
6. Add the new drive partitins to the RAID. Be certain that you are adding the correct partitions!
   $ sudo mdadm -a /dev/md0 /dev/sdi1 
   $ sudo mdadm -a /dev/md1 /dev/sdi2 
   <etc>
7. Wait for the drive to be completly restored. The sync process happens in the background at about 500 to 600 Kilobytes per second.
8. If possible, reboot to check that the new RAID is working correctly.

13.3 Replacing a failed RAID1 drive without a spare

1. Determine which drive has failed. this information can be inferred from /proc/mdstat and mdadm
   $ cat /proc/mdstat
   $ sudo mdadm -D /dev/md0
2. If any partiton on the failed drive were not removed by the system, remove them
   $ sudo mdadm -r /dev/md1 /dev/sdi2
3. Dump the partition for the good drive
   $ sudo sfdisk -l /dev/sdh
4. If the replacement drive is the same geometry
   $ sudo sfdisk -d /dev/sdh >part
5. Replace the failed drive
6. Partition the replacement drive like the failed drive using sfdisk if possible. If any differences are necessary, make the new partitions bigger.
7. Add the new drive partitions to the RAID. Be certain that you are adding the correct partitions!
   $ sudo mdadm -a /dev/md0 /dev/sdi1 
   $ sudo mdadm -a /dev/md1 /dev/sdi2 
   <etc>
8. Wait for the drive to be completly restored.
9. If possible, reboot to check that the new RAID is working correctly.

13.4 Stopping a RAID

This probably won't be necesary very often but when it is...

1. Unmount the RAID drive. Do not forget to do this.
   $ sudo umount /dev/md10
2. Try stopping the RAID. This usually works.
   $ sudo mdadm -stop /dev/md10

13.4.1 If stopping the RAID doesn't work

1. Determine the partitions that make up up the RAID
   $ sudo mdadm -D /dev/md10
2. If the RAID is a set of other RAID drives - eg a RAID 10 then it may be sufficient to destryteh superblock on the individual RAID sets
   $ sudo mdadm -zero-superblock /dev/md11 
   $ sudo mdadm -zero-superblock /dev/md12 
   $ sudo mdadm -zero-superblock /dev/md13 
   $ sudo reboot
3. If the RAID is made up of physical partitions or the above does not work, use fdisk change the partiton from type FD (Linux raid autodetect) to type 83 (Linux) and reboot. Fdisk will complain that it cannot update the partition table in the kernel but you can ignore that.