Howto: Create RAID5 and Expand/Grow it using mdadm

me09

View Full Version :
Howto: Create RAID5 and Expand/Grow it using mdadm without loosing data.
PumpActionAugust 4th, 2007, 10:53 AMI
write this guide because i found a lot of info on the subject but still
i had some problems because a lot of the info was very out-of-context
it dealt with very specific problems and no clue if I had/should do
anything before or after. With this guide I try to cover the whole
process, thats why I've included a plan over what I have to to before I
actually comes with the specific commands to type in terminal. This
guide got some bloat that you might not need, to skip it; search this
page for "blabla" and take it from there. For more information about
what those commands are suppose to do read the bloat:
I make this guide while experimenting to get the process right for creating and expanding a raid 5 array.
My situation is: I got three 320gb sata disks which is empty, and
another three which is full, I want to combine all six sata disks in
one raid 5 array.
I got no way to back it all up and format all disk and set up a 6 disk
raid-5 from scratch, so if my idea was to set up a raid5 with 3 disks
move over some data and, format the now empty disk and expand the array
to cover the new disk. Then repeat the process until its all six disks
in one array. This is of course not critical data that, that why I
don't got backups. Loosing it is only inconvenient, I am aware of this
SO SHOULD YOU BE IF YOU FOLLOW THIS GUIDE. IF DATA ARE IMPORTANT HAVE
BACKUPS AND DONT BLAME ME IF YOU LOOSE DATA. I'm no Ubuntu/Linux/RAID
expert, this guide is more of a collection of other guides aimed at
solving one specific set of problems. I write this guide while testing
everything i do on six small partitions spread over the three empty
disks, it is my advise that you to "try-it-before-you-buy-it".
In advance I have made some preparations which i won't cover in this guide. This is:
Prepare some files with a checksum (sfv). which i use as a quick check to see if my files are ok after expansion.
I also know my partitions very well, almost to the point that i could point them out on the platters :P
Also my system is up and running with a working internet connection and has access to apt-get
This howto will try to cover the whole process of setting up a raid5
array and expanding/growing it on a system already up and running. This
guide will NOT show you how to use a softraid5 array as basis for your
operating system or anything like that. This simply shows you how to
take a bunch of disks turn them into a raid5 and how to expand it later
without loosing data.
I write this guide because i found a lot of info on the subject but
still i had some problems because a lot of the info was very
out-of-context it dealt with very specific problems and no clue if I
had/should do anything before or after. With this guide I try to cover
the whole process, thats why I've included a plan over what I have to
to before I actually comes with the specific commands to type in
terminal.
What I've got:
Three full 320gb harddisk
Three empty 320gb harddisk
Ubuntu Feisty Fawn (7.04) 32-bit installed on some other completely separate disk.
Access to gparted, cksfv, mdadm either already installed or via apt-get install and a working internet connection
The todo list human mode:
Create raid5 with 3 empty disks/partitions
Copy data to raid5 array
Expand raid5 array and its filesystem with one (or more) disk/partition
This simple plan is a bit more complicated for the computer. Here is it's version:
The todo list computer mode:
----Make softraid section----
Prepare partitions to be used in array and set raid flag, format and set raid flag
Create raid array with mdadm
Format (make filesystem) raid array
Mount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Copy files
----Expand section----
Unmount the brand new and flashingly good looking supercool new ready to be used raid 5 array
Prepare and the new partition to be added
Put the new partition under softraiddriver control
Tell driver to grow on to the new disk
Check filesystem integrity
Expand filesystem so you get more space to play with
mount the expanded array
Check the integrety of your files
The terminal commands, no more blabla:
This is where it get serious, these are the commands I've used. MAKE
SURE YOU CHANGE EVERYTHING THAT NEEDS TO BE CHANGED like device names
and path, number of raid devices and such. tore is my username, md0 is
the raid-array I use, level=5 is raid5 vs. for example raid1 which is
level=1, raid-devices=3, the number of partitions dedicated for this
specific array. /home/tore/raid is where i mount my array.
REMEMBER THIS GUIDE COME WITH NO WARRANTY, IF YOU LOOSE DATA DON'T BLAME ME. Try-it-before-you-buy-it if you can.
EDIT: I've edited the guide, I split the guide into three sections.
(Prepare partition - make raid5 and format it - expand array &
filesystem)
This guide have been tested with success on Ubuntu Feisty Fawn (7.04) 32-bit.
If you test it on another system, tell me and i add a line above.
---Prepare the disk/partition---
What preparations needs to be done?
First you must know which partition/disk to use. I got no specific way
of helping you here, best tips is to use gparted to get a graphic view
of disks and partitions.
gparted can be installed by:
sudo apt-get install gparted
To run gparted type:
sudo gparted
When you have sorted out which partitions/disks to use there are four things to do:
1.remove it from fstab
2.format it to ext3
3.set raid flag
4.unmount partitions
When you have figured out which partitions to use you can proceed to
removing it from fstab. I assume you know how to do this, but a quick
refresher if you don't remember the details:
Back up your fstab:
sudo cp /etc/fstab /etc/fstab.backup
edit your original fstab with gedit:
sudo gedit /etc/fstab
put a # in front of the line containing the partition you want to use. Save&Exit
Now its time for format the disks/partition to be used in the array.
This is also a very straight forward kind of operation. Just run
gparted again.
sudo gparted
Since gparted is graphical I cant give you a command line guide. But if
you could find your way in fstab you should be able to navigate you way
to your partitions in gparted with ease. Once there just right click it
and format to ext3. Remember to be sure to pick the right partition, if
you pick the wrong one there is no going back.
After the partition has been formated right click it and “manage flags”, then hook-on the raid flag. Click "OK" & Exit.
The last step, make sure to unmount the partitions to be used in the
array. Now they should be out of the fstab, this could be done via a
reboot or:
sudo umount name_of_device
eksample:
sudo umount /dev/sda1

When this is done on all disks/partitions that are suppose to be a part
of the initial array you are finished with preparing the
disks/partitions.
----Make softraid section----
Install mdadm (softraid driver/application):
sudo apt-get install mdadm (awnser a few questions, i choose: none - yes)
Create raid device with partitions you prepared in gparted(modify this
line for correct number of raid-devices and make sure to use the right
devices):
sudo mdadm --create /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1
The prosess of creating the array takes time, monitor it and do not
continue this guide until its done(type it over again to update):
cat /proc/mdstat
Make a ext3 filesystem that spans the array:
sudo mkfs.ext3 /dev/md0
Mount the array (remember to choose were):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Take ownership (replace tore with your username and correct the path)
sudo chown -R tore:tore /home/tore/raid
The copy some files with valid sfv and check to see if everything is ok, and to have a refrence point after expansion.
cd /home/tore/raid
cksfv -r -q
----Expand section----
Unmount the array, i don't know if this is necessary, but doing so means less stuff to go wrong.
sudo umount /dev/md0
Then you need to go and prepare the disk(s) to be added. (The prepare section of the guide)
Then tell mdadm that it can play with it:
sudo mdadm --add /dev/md0 /dev/sda2
Then tell mdadm to stop playing and use it to expand/grow your raid5
array(make sure that the number of raid devices is correct, it should
be the number of disks/partitions the grown/expanded array will have):
sudo mdadm --grow /dev/md0 --raid-devices=4
View and wait for the new array to reshape:
cat /proc/mdstat
Check the filesystem (required to resize it):
sudo e2fsck -f /dev/md0
Resize the filesystem on the new array so that you can use it for something useful.
sudo resize2fs /dev/md0
Mount the expanded/grown array with it's resized filesystem(remember to change the mount point):
sudo mount -t ext3 /dev/md0 /home/tore/raid
Go check your files:
cd /home/tore/raid
cksfv -r -q
_____________________
Need to expand more? Repeat the expand section. Remember to change
number of raid devices and choose the correct device to add. Also, you
can add more then one disk at the time, I jump strait from 3 to 6.
_____________________
Was this not the right guide for you? Try one of these (my guide is composed from some of these):
http://scotgate.org/?p=107
http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch26_:_Linux_Software_RAID
http://tldp.org/HOWTO/Software-RAID-HOWTO.html
http://ubuntuforums.org/showthread.php?t=408461
http://www.howtoforge.com/linux_resizing_ext3_partitions
http://linux.about.com/library/cmd/blcmdl8_resize2fs.htm
http://www.mail-archive.com/linux-raid@vger.kernel.org/msg08539.html
http://ubuntuforums.org/showthread.php?t=509388&highlight=mdadm
http://gd.tuwien.ac.at/linuxcommand.org/man_pages/mdadm8.html
EDIT1: Fix a buch of typos.
EDIT2: Split the guide into three sections, added some info on fstab, fix more typos.
tuckieSeptember 21st, 2007, 01:30 PMJust
wanted to say thanks for the writeup, the bulk of it I knew, but its
nice to have it all in one place. I'm going to be using this on Monday.
tuckieSeptember 23rd, 2007, 06:14 PMQuick
question, anyone know if it is necessary to umount the raid volume
before growing? I'd rather not have to go through 7 hours of downtime.
tuckieSeptember 24th, 2007, 06:21 PMwell im growing it without umounting the volume. Wish me luck.  Only 13 hours remaining!
matthodgeOctober 8th, 2007, 09:42 AMHow did it go tuckie? RAID up and running without unmounting?
tuckieOctober 8th, 2007, 11:55 PMwell
if you looked at my other posts you'd get a good idea :( Essentally,
growing the raid went fine, but I had to unmount it to grow the ext3
partition. Pretty much 5 hours after everything was up and working, one
of the drives dropped from the array and somehow the partition got
corrupted in the process (2tb of data gone!).
--Live and learn and always have backups.
               
               
               

本文来自ChinaUnix博客，如果查看原文请点：http://blog.chinaunix.net/u/17006/showart_652943.html