Glock Talk

Glock Talk (http://glocktalk.com/forums/index.php)
-   Tech Talk (http://glocktalk.com/forums/forumdisplay.php?f=86)
-   -   What would I gain from a "real" RAID? (http://glocktalk.com/forums/showthread.php?t=1493338)

CitizenOfDreams 06-30-2013 17:11

What would I gain from a "real" RAID?
 
I know "fake" software RAIDs are laughed at in the IT community, but I have been happy with mine (4 drives in RAID10, Intel ICH10R). It had adequate speeds and no noticeable CPU load. The array properly rebuilt itself when I faked a drive failure.

Now I am planning to build another computer, so I have the following choices in storage configuration:

1. Intel motherboard with software 3Gb/s RAID;

2. A $300 more expensive Asrock motherboard with "half-hardware" 6Gb/s LSI SAS2308 onboard controller;

3. A separate "half-hardware" RAID controller (which one?);

4. A real hardware RAID controller (does not fit my budget).

So, which option should I choose, and what exactly will I gain from a better RAID controller? I am planning to use the machine for video editing and light gaming (think Starcraft, not Halo 6).

harrygunner 06-30-2013 20:05

Really should think about RAID with today's multi-terabyte drives. We're getting to the point where the "Unrecoverable Bit Error" (UBE) of drives is near the size of drives.

So re-silvering (rebuilding) a RAID group after a drive failure requires nearly the number read/writes as the drives are spec'ed for. If a UBE occurs or two or more drives fail, you've effectively lost your data.

Personally, I'd go either go RAID1 or forget RAID. In all cases, back up your data. Tape drives cost a bit, but you can store multiple copies off-site and recover data much faster than over Internet. Pipe data through compression, then encryption before writing to tape and it's secure.

As far as hardware RAID controllers, the main downside is if anything proprietary is written by the controller and either that version becomes unavailable or the company goes out of business, you'll have trouble if the controller fails.

CitizenOfDreams 07-02-2013 22:14

Thanks for the input!

RAID10 is what I want. A combination of capacity and speed, plus some level of fault tolerance. And yes, I know the importance of backups (learned it the hard way, like most of us).

For now, I am going to settle on onboard "fake RAID", for budgetary reasons. Maybe one day I will uprade to a real controller, but I would like to play with one before I commit.

Bushflyr 07-03-2013 00:50

Quote:

Originally Posted by harrygunner (Post 20419053)
So re-silvering (rebuilding) a RAID group after a drive failure requires nearly the number read/writes as the drives are spec'ed for.

Sorry, but you have no idea WTF you're you're talking about. I've rebuilt my RAID after failure about a dozen times, 5 and 6,
there's no way possible that it's even coming close to wearing out the drives.


Quote:

Tape drives cost a bit, but you can store multiple copies off-site and recover data much faster than over Internet.
Tape!?! TAPE ?!!? Sorry bro, you're stuck in the last century. Tape drives are beyond dead tech. :upeyes::rofl:

Quote:

As far as hardware RAID controllers, the main downside is if anything proprietary is written by the controller and either that version becomes unavailable or the company goes out of business, you'll have trouble if the controller fails.
At least you got one thing right.

Which is why I recommend software RAID. You give up a little performance, which most of us never notice, and gain the comfort of never having to rely on hardware that will likely be obsolete and difficult to replace when it fails. I've had hardware that I need go obsolete then fail so many times it isn't even funny. We still use 1gb PCMCIA cards at work that are stupid expensive compared to modern tech but we can't change because they're TSO'd. :upeyes:

Detectorist 07-03-2013 10:07

Tape? My Grandparents told me stories about that. lol

harrygunner 07-03-2013 15:30

I read this paper a while back http://www.cs.cmu.edu/~bianca/fast07.pdf I just like to read stuff and I don't like to lose data. That paper had me thinking what to do given I have dozens of multi-terabyte drives with critical data on them.

Here's two articles written by people who appear to be unconcerned about failures while rebuilding a RAID5 array.

They has links to other articles and calculations.

http://www.pizzaandcode.com/posts/780

http://subnetmask255x4.wordpress.com...-impacts-raid/

Some posted comments about still being concerned.

Quote:

KD Mann: In addition, the RAID-5 rebuild process itself is hard on all the drives in the array, not just the “failing” drive, and UREs will increase with drive temperature across the entire array — further increasing the chances of a catastrophic failure.

Finally, with 2, 3 and soon 4TB drives, the amount of time requires to complete a RAID-5 rebuild can now stretch into DAYS! Desktop-class disks weren’t built to withstand that kind of abuse even for a few hours…

Given the ratio of URE probability to disk capacity, RAID-5 is dead, there’s no getting around it. Best to just get used to the idea and get on with life….

Disclaimer: Yes, I do work for a RAID manufacturer, and yes, I have (in the past) designed RAID hardware and software.

And about tape drives, the idea is to backup to devices that have different failure modes and lower failure rates. Backing data to devices that have similar failure rates as the original devices is delusional.

And equally important, faster recovery times. Backing up to the "cloud" sounds cool, but if you need to recover those terabytes, it's going to take a while. If you can perform the calculations to predict probabilities of failures of disk arrays, calculating the time to download your backup is easy.

Linux3 07-04-2013 15:40

Quote:

Originally Posted by Detectorist (Post 20426317)
Tape? My Grandparents told me stories about that. lol

Oh my child. Tape is still very much in use.

If you want to back up a 50Tbyte virtual drive it is the only way.
If you want to digitize, FX and store a movie, it's the way to fly.
It is the standard for long term storage.

http://www.fujitsu.com/global/servic...ts/tape/lt250/

Detectorist 07-04-2013 18:36

That's awesome, Linux3.

The last time I dealt with tapes was way back in 1988ish. IBM reel to reel 3450? density. I was the SSO and the shipping company lost a shipment of them...

dwhite53 07-05-2013 06:25

I went through this too building my wife's latest machine. Wanted something to make sure her data was secure.

Windows 7 has software mirroring but it has to be set up from the original install, we were moving her hard drive to the new machine so that was out.

I then did a search for "real time backup" and found a few options. I ended up with this:

http://www.codessentials.com/products/yadisbackup.html

You can set it to mirror whatever files from your main drive to a second, third or however many drives. It's not true mirroring but it will save your data. Any time you modify or create a file, Yadis! copies it to the backup drive. It has to be set to delete saved files when they're deleted from the main drive.

I installed this on my laptop too removing the cd drive and installing a second hard drive. Doesn't affect batter life noticeably.

All the Best,
D. White

srhoades 07-05-2013 18:06

Quote:

Originally Posted by dwhite53 (Post 20430786)
I went through this too building my wife's latest machine. Wanted something to make sure her data was secure.

Windows 7 has software mirroring but it has to be set up from the original install, we were moving her hard drive to the new machine so that was out.

I then did a search for "real time backup" and found a few options. I ended up with this:

http://www.codessentials.com/products/yadisbackup.html

You can set it to mirror whatever files from your main drive to a second, third or however many drives. It's not true mirroring but it will save your data. Any time you modify or create a file, Yadis! copies it to the backup drive. It has to be set to delete saved files when they're deleted from the main drive.

I installed this on my laptop too removing the cd drive and installing a second hard drive. Doesn't affect batter life noticeably.

All the Best,
D. White

Does it save versions? What happens if I save over a file and it copies that over across all the drives?

dwhite53 07-06-2013 12:30

Quote:

Originally Posted by srhoades (Post 20432574)
Does it save versions? What happens if I save over a file and it copies that over across all the drives?

Yadis will do versioning though I don't use it and am thus not familiar with it.

Pretty easy to just download, install, and configure. Select a few files from your machine to save to an SD card or flash drive and see how it works.

As fast as computers are now the overhead of the machine making copies of files to another drive is minimal. For most people RAID is just too much.

I've read that if you do hardware RAID and your controller/motherboard die you can have issues accessing your data unless you use another board with the exact same hardware. RAID only has to do what is described in it's ANSI/ISO/IEEE spec's. How the hardware manages it is up to the hardware manufacturer.

Using Windows built in RAID eliminates this problem however it only does the same thing as Yadis! does with a lot more overhead.

All the Best,
D. White

CitizenOfDreams 07-06-2013 18:25

Quote:

Originally Posted by dwhite53 (Post 20434389)
Using Windows built in RAID eliminates this problem however it only does the same thing as Yadis! does with a lot more overhead.

I have used Windows software RAID (stripe) with good success. The CPU usage was somewhat noticeable, but I didn't have much of a CPU back then.

Unfortunately, Windows does not do RAID10 in software.

Drjones 07-08-2013 14:49

Quote:

Originally Posted by CitizenOfDreams (Post 20418470)
So, which option should I choose, and what exactly will I gain from a better RAID controller? I am planning to use the machine for video editing and light gaming (think Starcraft, not Halo 6).


Apart from a RAID 0 for a possible performance increase, I'm not sure what you'd gain from RAID.

I mean, there's the obvious redundancy advantage, but as some here have pointed out, there are real dangers to proprietary RAID controllers.

Dunno if you saw my other thread about the NAS I want to build, but for now, I've decided against RAID, mostly because I know that the non-enterprise-level RAID stuff is not that great.

Many/most video/graphics editing programs are designed to have a "scratch" drive anyway, right? This wouldn't apply to a RAID config, I wonder if you might not get *worse* performance, depending exactly what software you use & how you use it.

Dunno about your level of tolerance for messing with stuff like this, but if I were you, I'd forget the RAID, have multiple internal drives, and make positive that you have 2-3 different backups that run as often as you need them to.

CitizenOfDreams 07-08-2013 19:10

Quote:

Originally Posted by Drjones (Post 20440103)
Apart from a RAID 0 for a possible performance increase, I'm not sure what you'd gain from RAID.

Mainly, not having to restore data from backup when one of the hard drives goes "click-click-click". Also, storing "semi-important" data that's not worth backing up (e.g. downloaded videos to watch and forget).

Quote:

Dunno if you saw my other thread about the NAS I want to build, but for now, I've decided against RAID, mostly because I know that the non-enterprise-level RAID stuff is not that great.
I think I do remember that thread. That's where you had to reformat your array after a firmware upgrade? My software RAID (Intel Rapid Storage) didn't seem to have those problems. It kept working after a BIOS upgrade, an HDD firmware upgrade, and a driver upgrade.

Quote:

Many/most video/graphics editing programs are designed to have a "scratch" drive anyway, right? This wouldn't apply to a RAID config, I wonder if you might not get *worse* performance, depending exactly what software you use & how you use it.
I don't see a problem there. Programs either use %TEMP% (which would be on a separate SSD along with the OS), or let you set the scratch disk manually.

Quote:

Dunno about your level of tolerance for messing with stuff like this, but if I were you, I'd forget the RAID, have multiple internal drives, and make positive that you have 2-3 different backups that run as often as you need them to.
I don't see how RAID10, even a fake RAID10, can be less reliable than a standalone drive. :dunno:

Drjones 07-08-2013 21:21

Quote:

Originally Posted by CitizenOfDreams (Post 20440930)
Mainly, not having to restore data from backup when one of the hard drives goes "click-click-click". Also, storing "semi-important" data that's not worth backing up (e.g. downloaded videos to watch and forget).

True, I just figure that ANYTHING worth keeping around at all, is worth backing up. Maybe not making multiple backups, and off-site backups, but backups nonetheless.



Quote:

I think I do remember that thread. That's where you had to reformat your array after a firmware upgrade? My software RAID (Intel Rapid Storage) didn't seem to have those problems. It kept working after a BIOS upgrade, an HDD firmware upgrade, and a driver upgrade.

Nope, this thread: http://glocktalk.com/forums/showthread.php?t=1492327



Quote:

I don't see how RAID10, even a fake RAID10, can be less reliable than a standalone drive. :dunno:
Oh, I never said it would be less reliable. Just that it's a little more work, and again some of the comments I've read in various places on the internet make me a little nervous about consumer grade RAID.

nostalgiaguy 07-08-2013 21:58

In regards to the very first post.......

You lost me at ...." I know "fake" software RAIDs... "

After that I just mumbled to myself abut getting old and not understanding anything anymore.

Damn whippersnappers. Get off my lawn!! LOL

CitizenOfDreams 07-08-2013 22:20

Quote:

Originally Posted by nostalgiaguy (Post 20441376)
In regards to the very first post.......

You lost me at ...." I know "fake" software RAIDs... "

You know what a hard drive is? No? Magnetic drum storage? Tape? Punch card? How about abacus?

CitizenOfDreams 07-09-2013 04:20

Quote:

Originally Posted by Drjones (Post 20441295)
True, I just figure that ANYTHING worth keeping around at all, is worth backing up. Maybe not making multiple backups, and off-site backups, but backups nonetheless.

In ideal world, yes, all data should be backed up. In the real world, however, backup storage and bandwidth are not unlimited. So you have to range your data from "irreplaceable, backup at all costs" to "somewhat hard to recreate, backup whenever the budget allows" to "commonly found, no big deal if it gets lost".

Scary stuff there. There is something fundamentally wrong about a RAID that does not report a dead array member.

Quote:

Oh, I never said it would be less reliable. Just that it's a little more work, and again some of the comments I've read in various places on the internet make me a little nervous about consumer grade RAID.
As long as you not replacing regular backups with RAID, I don't see a problem. As far as work, rebuilding a RAID is far less work than restoring a backup (and all your data stays accessible in the process).

dwhite53 07-09-2013 07:35

My machine is configured with a single, two partition, hard drive. C:\ is for operating system and program installation. Nothing gets saved to the C:\ drive.

D:\ is for all personal data. Everything gets saved to the D:\ drive. I've got D:\ configured with Yadis! to back-up real time to a second hard drive. I periodically (like most, not often enough) do a full imaging of C:\.

Having an OS/program drive and a separate data drive has worked well for me for years.

All the Best,
D. White

CitizenOfDreams 07-09-2013 09:29

Quote:

Originally Posted by dwhite53 (Post 20442027)
Having an OS/program drive and a separate data drive has worked well for me for years.

Yep, that's exactly how I keep it. Partition C: is for the OS and temporary files only, so it can be formatted and reinstalled at any time without worrying about user data.


All times are GMT -6. The time now is 14:23.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Copyright ©2013, Glock Talk, All Rights Reserved.