The importance of storage

Larry Jordan#

Author: Larry Jordan#

Published 1st July 2015

by Larry Jordan Issue 102 - June 2015

"What\'s the best computer to edit video?\" It\'s a great question - it\'s just the wrong question. Yes, CPU speed is important and the Internet is buzzing with the latest specs on GPUs. But, today, virtually any currently shipping computer is more than adequate to edit video; even 4K.
The critical component that no one wants to talk about - because it just isn\'t very sexy - is storage. Yet, if you don\'t have the right storage, the fastest computer in the world is not going to work very well. So, let me share some thoughts on what you need to consider when purchasing storage.
SIZE MATTERS - BUT SIZE ISN\'T ENOUGH
A few years ago, we debated how many gigabytes to get. Today, we are obsessed with total terabytes. In a year or two, exabytes will be affordable and we\'ll happily start debating these. The key rule you must keep in mind is that however much storage you buy, it will never be enough.
In Hollywood, there\'s a saying that an actor "can\'t be too rich or too thin.\" With storage, the rule is that a hard disk is "either empty or full.\" For some reason unknown to the brightest minds on the planet, a half-full hard disk is almost unheard of. Plug-in a brand-new hard disk and, within a couple of days, you are scrambling to delete files to make more space.
Sigh¦
CODECS COUNT
The reason is that video files don\'t live in a vacuum. The codec that you shoot and edit determines how much space your files will require. For example, the table below illustrates five different codecs all shooting 720p files. The image size, the frame rate is the same, but the file size is not.

Why the difference? Most video formats are compressed and the codec determines the amount of compression. Some formats, like HDV and AVCHD, are massively compressed; removing more than 95% of the original image. Others like RED and ProRes are much less compressed, retaining more of the original image.
So, as you start to budget for storage, you need to first ask yourself which codecs you are likely to be shooting, whether you plan to transcode or render media and what formats you plan to use for final output.
All of these decisions have an impact on the amount of storage you\'ll need.

YOU NEED CONNECTIONS
Having massive storage is part of the solution, but you also need to give it a fast connection to your computer. In the past, it was a choice between FireWire and USB 2. Neither of those choices are good ones today.
USB 2 is too slow for any video editing; except for transferring files from one drive to another. FireWire is fine for SD, but struggles to keep up with HD. I no longer advise it, unless you are running on an older, slower system without access to faster connections.
Mac users are best advised to use Thunderbolt, with a maximum throughput of 2.2 GB /second. However, as you\'ll learn in a minute, this number is illusory at best and deceptive at worst. PC users can choose cards that support eSATA, USB3, miniSAS or Thunderbolt. I\'m not a PC expert, so I don\'t have a specific recommendation except to say that USB3 is a safe, middle-of-the-road choice.
THE INTERFACE TRAP
Here\'s where you need to be careful. Manufacturers are very fond of saying that their equipment "supports Thunderbolt 2 with speeds up to 2.2 GB/second.\" This is a true statement for all equipment that is approved for Thunderbolt. It is true, but not accurate.
A "spinning-media\" hard disk sends data across the wire about 120 MB/second. So, although the "pipe\" that connects it runs at 2.2 GB/second, the MOST you can get out of one hard disk is 120 MB/second. Period. Your data is running inside a very large pipe, but it isn\'t running very fast. This is true of ALL high-speed connections - Thunderbolt, USB3, eSATA, etc. The connection is no longer the gating factor on speed, it\'s the speed of your hard drives.
"Oh!,\" I hear you say, "what about solid state drives (SSD)?\" SSD drives can be faster than spinning media, but not all SSD drives are equally fast. I\'ve tested SSD drives that were only marginally faster than spinning media. If you are spending lots of money for SSD drives, buy from a reputable company that publishes the actual speeds of their units, not simply states that it supports a particular protocol.
In general, a four SSD array will get close to 1 GB / second. An eight SSD array will get close to 2 GB / second. But there\'s no free lunch. SSD drives, though potentially MUCH faster than spinning media, hold far less and cost far more. Personally, I use SSD drives for the drive containing the operating system and applications, with an external RAID containing spinning hard drives for all my media. (I like SSDs, but I need storage capacity more than speed.)

RAIDs - TODAY\'S SOLUTION
A RAID is a collection of hard drives - either SSD or spinning media - that combine their efforts to create faster speeds and greater storage than a single hard drive alone can support. RAIDs are designed to provide three key benefits:
* Faster speed than a single hard drive
* Greater storage than a single hard drive
* Data "redundancy,\" meaning that if one of the drives in the RAID dies, all your data is safe.
RAIDs are grouped by RAID "level.\" There are different levels, but here are the most popular. You buy the RAID level that delivers the performance and features that you need:
RAID 0 - Fast, cheap, no data redundancy. Requires a minimum of two hard drives inside the RAID enclosure. Most often used when speed combined with low cost are paramount. Combines the speed and storage of the drives in the RAID.
RAID 1 - Complete data redundancy. Requires a minimum of two hard drives inside the RAID enclosure. Often called "mirroring,\" each drive is a complete copy of the other. Most often used for backing up servers or on-set for DIT media work. No faster than the slowest drive in the system.
RAID 5 - Very fast, data redundancy. Requires a minimum of three drives; most often found with four or five drives inside. When one drive dies, your data is still safe. This is the most popular mid-priced RAID for video editing and generally connected to just one computer system.
RAID 6 - Fast, extra data redundancy. Requires a minimum of four drives. This version protects your data in the event two hard drives die at the same time. It used to be more expensive than RAID 5, but, generally, the same physical size. Like the RAID 5 this is most often connected to just one computer. Not as fast as a RAID 5.
RAID 10 - Very fast, totally redundant. Requires a minimum of four drives, but is often created using two matched RAID 0\'s. This creates a RAID 1 by combining two RAID 0\'s into one unit. This provides the speed equivalent of a RAID 0, with the data redundancy of RAID 1. As RAIDs continue to drop in price, this can be a less-expensive way to create systems that rival the performance of a RAID 50. Like RAID 5 and RAID 6, this is most often used connected to one computer.
RAIDs vary in price from a few hundred dollars for a RAID 0 to thousands of pound for a 16-drive RAID 10. As with many things in life, greater storage, faster performance or greater redundancy all cost more.
SUMMARY
As you continue your career in media, you\'ll discover that you will spend FAR more money on storage than you will ever spend on your computer. For this reason, it makes sense to better understand how storage fits into your editing workflow. There are many excellent hardware choices out there - you just need to ask the right questions to get the one that \'s best for you.

Related Listings

Related Articles

Related News

Related Videos

© KitPlus (tv-bay limited). All trademarks recognised. Reproduction of this content is strictly prohibited without written consent.