General RAID question.

[Matt Harris]

Evenin' all.

I'm still livin' in the glorious world of IDE drives, God bless 'em, but am hopefully upgrading soon, and it'll all be SATAII and RAID.

RAIDs of particular interest to me as I have to do alot of backing up, normally of whole hard drives.

The drives will be set out as follows.
320gb for OS, Software, general stuff.
750gb for media and such.
then four 500gb drives.
A and B. With a mirror of each.

I'm new to the whole RAID thing and havent been able to find anything that explains if if can do what I want it to do.

I want to set up the 500gb drives like this.
AA AB BA BB.
AB and BB being backups of their respective drives.
I know thats RAID1, a mirror drive, but can it be done, so that RAID1 is enabled on AA to AB, and on BA to BB.
Or is it only available once?

Thanks for any replies.

Cheers,
Matt.

[TheMadDutchDude]

Why would you want to do that?

Just create a 4x500GB back up drive (RAID1), this will give you 500GB of space, although they will all have identical data stored on them. Indeed, very high security on all of your data.

[Matt Harris]

I need it to be set up as two drives, A and B. B for when A's full, which is quicker then you'd imagine.
Once A's full, I start saving on B, but I'll still need access to A.
And I want both of them to have their own backup.

[TheMadDutchDude]

Mhmm, complicated but hey.

I don't have enough knowledge in this area to be able to assist you fully although I'm sure someone else does. If I had another two drives I'd give it a go, but unfortunately I don't.

I'm sure you can do it, just await for some assistance and all shall be good! Apologies for little assistance given.

[NeilX90]

Yes, you can have multiple RAID 1 installations on your disks, but you may well need a seperate RAID controller card depending on your motherboard. Windows (any version) couldn't give a stuff about RAID as it's all done at a lower level.

Raid controllers start from about £30 and go up to over £1000, and as many use a high-speed PCI-E slot, you'll need to check you're not running on older hardware. I think for just 4 drives, you should easily find a controller for under £100 if your m/b doesn't natively support multiple RAID arrays.

Adaptec controllers would be a sound choice for home/small office use - good value for money.

[Vic]

I want to set up the 500gb drives like this.
AA AB BA BB.
AB and BB being backups of their respective drives.

So what you're looking at is two mirrors of 500GB each.

You might want to look at alternative ways of doing this - you could, for example, use 3x500GB drives to give yourself a 1TB RAID-5 array, where any drive failing will leave you with an intact (albeit unprotected) filesystem. That's pretty much the same resilience as your dual-RAID-1 configuration, but gives you a single 1TB space to partition as you see fit.

If you wanted to improve your fault tolerance, you could add a fourth drive as a RAID-6 array. Any drive failing will degrade the array to the RAID-5 one above. But that's probably overkill unless you're intending to sut this away in a datacentre for years at a go...

I can't advise on hardware to do this, I'm afraid - I only use software RAID these days. CPU load is minimal - all the delay is in disk I/O. But I don't know if Windows does decent software RAID - that's something else I no longer use...

HTH

Vic.

[Matt Harris]

Cheers for all the feedback guys, I really appreciate it.
Like I said, new to the whole RAID thing, though I like to think I have a basic grasp of it.

Basically I want a mirror each of two drives.

My business is photography, so if I lose a drive, I've lost years worth of work, hence the somewhat overkill backup system.
I was recommended to use two 1tb drives, with one running as a backup, but that'd probably be even more trouble in the long run.
At least if a 500gb drive dies, you can just swap it out with another 500gb, cause lets face it, they don't break the bank anymore.
Where as 1tb drives still cost a bit, although alot less then the last time I upgraded my drives.

I had a good google on the RAID array built into the motherboard I'm looking at, the Asus P5Q-Pro.
Intel's site says its able to run two volumes of RAID1 from its controller array, which should work...right?
I dont need a complex striping setup, or saving data over all disks, I just need a direct backup of each 500gb drive.

Thanks again for all the help guys.

[NeilX90]

Matt, in which case - the RAID 5 suggested by Vic is the way to go. With only 3 drives, you can protect all of your data - if one fails, you simply replace it. It's cheaper than running 4 drives and will give you the same amount of storage and protection.

For ultimate data protection, RAID 6 is perfect but needs 4 drives. What are the chances of two drives failing before you can buy a new one ? Ermmm.. slim to nil.

In your position, I'd run 3 x 500Gb drives in RAID 5, with a total of 1Tb available.

Cheers

[Matt Harris]

The way I'm reading it, which is probably wrong, RAID5 copies the data 3 times, if you have 3 disks. right? Once to each...

[Vic]

The way I'm reading it, which is probably wrong, RAID5 copies the data 3 times, if you have 3 disks. right? Once to each...

Not really...

RAID-5 uses 3 disks (or multiples thereof), but it's not a simple mirroring; there's is a rotating parity over the diskset.

So one block from each of the disks goes to make up TWO blocks of data in the array.

Thus 3x500GB drives in a RAID-5 array is 1TB of space, and you can lose any one of those drives without losing any data.

Vic.

[Matt Harris]

This may sound a bit dim, but how can I fit 1gb of data, with a full backup of all that data, onto 1.5tb of drives...

[NeilX90]

It's not a dim question at all.

Rather than bore you with the actual details, you *don't* have a physical back-up of 1Tb of data. What happens is that all 3 drives store information to enable the remaining two disks to rebuild any data lost on any one failed drive. It is not a backup per se - it is a level of redundancy that allows one disk to fail. When that failure occurs, you now have no backup or redundancy at all and will need to replace the failed drive asap.

If you opt for RAID 6, you will still only be able to store 1TB, but now two disks can fail and you will lose nothing.

If you opt to RAID-1 two sets of drives - AA and BA, mirrored to AB and BB.... if AA and AB fail... bang, there goes half your data. With RAID 6 *any* 2 disks can fail and you've lost nothing.

Any clearer ? :D

[Vic]

If you opt for RAID 6, you will still only be able to store 1TB, but now two disks can fail and you will lose nothing.

Absolutely correct.

I would, however, question whether RAID-6 is appropriate for most people; IME, even in datacentre applications, RAID-5 with an extra disk that can be made into a hot spare is plenty, and it allows you to use the fourth disk for other purposes for short periods. That can get you out of the poo when your customer suddenly changes his mind about box config, but doesn't want to sacrifice the uptime guarantee he demanded :-)


If you opt to RAID-1 two sets of drives - AA and BA, mirrored to AB and BB.... if AA and AB fail... bang, there goes half your data. With RAID 6 *any* 2 disks can fail and you've lost nothing.

Exactly. RAID-1 is about protecting against single failures. The original proposal of two RAID-1 arrays is not proof against two disks going down.

Vic.

[Nox]

Given the critical nature of your data, I would be using 500GB disks in external caddies, and keeping them away from the computer - bare in mind that if your computer gets fried, you run the risk of loosing all disks anyway... what you're doing is only protecting against a physical failure of one of the drives, not something caused by... lightning (get a surge protector) or a malfunctioning psu (bye bye all hd's if you're unlucky) or whatever.

oh and atleast 2, incase one gets dropped :D I would not be putting all my eggs in one basket, erm sorry, disks in a computer.

Nox

[Matt Harris]

I kinda understand RAID-5 and 6 now, but I'm still a bit hazy.

Right now for my work, I'm running two WD 500gb MyBooks. I'm was gonna run them as extra backups too, using Ghost or the such like.
Update them every few days automatically.

For what I need to achieve, I still think RAID-1 is the most straight foward set-up.
Two individual sets will be so much easier to search too, instead of a huge 1tb partition. Seeing as I shoot anywhere between 200-500 images per shoot, and my cataloging isnt exactly organized.
I'm looking at it was the main drive, with a mirror through RAID-1, and an extra copy of that on an external drive. Plus DVD backups straight from the memory cards.

Unless someone can convince me of a real, overwhelming reason to use RAID-5 or 6.
To me they seem much more about spreading the data around efficiently, then acting as redundancies.

Thanks again guys, I appreciate all the replies.

[NeilX90]

Matt,

RAID is *all* about redundancy - it's certainly not about efficency !

Put it this way - RAID 5 offers the same level of protection using one less disk than twin RAID-1. It is cheaper. If you buy 4 disks, you have one spare to instantly swap in. If you use twin RAID-1 you will have to buy another.

You can also partition a 1Tb RAID-5 array into 2x500Gb volumes, should you so wish. However, at the end of the day, it's your money !

[Matt Harris]

Neil, put that way, it definitely sounds better.
I've spent the better part of the weekend trying to find a concise explanation of how it works. All I kept finding was the same default thing.

Would it be worth running a 4 disk RAID-5 array?
Most examples I see are of a 4 disk layout.
Or the basic three, with two 500gb partitions, like I'm used to? Any benefits of 4 drives?

Thanks again.

[NeilX90]

I think the simple answer here is that most people who are using RAID levels 5 and 6 are doing so professionally - they are providing data security for companies, web servers etc. and they generally talk to each other in a language that few of us mortals can understand.

This is about as simple as I can make the explanation of how it works and whilst not technically 100% accurate, it should make things clear(er) !

You have 3 drives, each of which is 500GB, however only 1Tb is available to use. Let us imagine you've filled the drives to capacity and have 1Tb of stored data. The data will be spread across each of the three disks - let's say 333Gb on each one. That leaves 167Gb on each drive free. What the RAID controller does is use that spare space to provide data which the controller can read to rebuild the data from the failed disk onto the remaining two.

So...

Disk 1 has information on how to rebuild data if disks 2 or 3 fail.
Disk 2 has information on drives 1 + 3
Disk 3 has information on drives 1 + 2

Let us say Disk 2 fails. There is enough information on disks 1 and 3 to allow the RAID controller to recover the lost data from Disk 2 and rebuild it on Drives 1 + 3. Now there is no spare space to store recovery information which is why you have to replace that failed disk to get redundancy again (that was the bit that's not quite technically accurate - if you only have, say, 1Mb of data stored, the death of one drive still means you have no redundancy, even though that 1Mb could be easily stored on any of the remaining disks multiple times - it just isn't !). Anwyay, as soon as you replace Disk 2, the RAID controller will spread the data across all 3 again, along with the recovery info which is called 'parity blocks'.

Phew.

The reason you might see mention of 4-disk arrays is that performance is increased for each additional disk in a given array, useful for corporate servers etc, and obviously the amount of storage increases. (The total storage capacity of a RAID 5 array is one disk less than the total number of disks in the array). You could have a 12-disk RAID 5 array if you wanted, but it still has the same basic redundancy level - if one of those 12 drives fail, you won't have any redundancy. You could argue that the more disks you add to an array increases the likelihood of it failing !

A further reason to increase the number of drives in an array like RAID 5 is that of cost-efficiency. It might not be obvious at first, but is easily explained. In a 3-disk RAID 5 array you have lost 33% of your total storage capacity, 500Gb out of 1.5TB. If you increase your array to four drives, you have lost 25% of your total storage, 500GB out of 2Tb. If we take things to the extreme, a 100-drive RAID 5 array loses only 1% of its total storage capacity, but would cost a small fortune. Corporate types will have a formula which allows them to work out the optimum number of drives comparing total cost to storage efficiency.

RAID 6 allows two drives to fail before any need replacing and, as Vic said, is very likely overkill. A cheap, 3-disk RAID 5 array should do you nicely.

As I mentioned earlier, Windows cannot tell that you are using a RAID array - it's handled by the controller at a lower level; the controller presents the three disks to Windows as a single 1Tb hard drive which you can then partition as normal to suit your purposes.

Very interesting comment earlier about putting them in external enclosures rather than in the PC itself - a very good point if your data is ultra-critical and as you make your living from it I guess it is ! Depends how much you trust your other backups, but if I made my living from digital images I would be paranoid about data security.