Protecting your assets, safe storage and long life archives


A vast amount of information exists out on the Internet that can be very helpful but also misleading and confusing on how best to protect digital content over time both from degradation of content and from piracy.
Thinking back on my twenty plus years in the industry, it is my personal opinion the first creative professionals to experience problems with securing digital content were graphic designers and digital photographers moving from film to flash type memory cards, and musicians moving from 2” tape and DATs to hard disk recording – these professionals were the first digital pioneers to experience problems when digital storage technology failed.
Fast forward to today where consumer handheld devices now have the ability to record HD and even 3D movies, and home users and creative professional alike are fuelling a massive explosion in unstructured digital content on a scale never before seen. So what do you do with ‘the’ data? How important is ‘the’ data to you? What value will ‘the’ data have in years to come? The data is just ones and zeros. A campaign I ran with Glyph Technologies back in 1997 had a slogan “not just one’s and zero’s – it’s your art, your life”.
A good example would be a post-production company working for a client, typically the post-house would only be concerned about data availability whilst doing the job required, once finished and the project delivered, the company hands over the responsibility of the data to the client. The client may or may not realise the fallibility of the single HDD that they have just been given by the post-house. How long could or should the client need to keep the data? And who can predict the value of that data in the future? But we all know this right, so what technology should we use?
700 million Hard Disk Drives (HDD) were sold last year alone and the capacities are ever increasing with 4TB now available in a 3.5” single disk form factor – 4TB would be a huge amount of data just to disappear for most creative professional. As these larger capacity and low cost SATA disks are integrated into large RAID solutions it creates new problems for manufactures to solve. For example the time taken for a 3TB or 4TB HDD in a RAID set to rebuild if it fails is significantly longer than a 1TB or 2TB mechanism, with some high capacity systems taking 48hrs to rebuild. RAID manufactures are addressing this by implementing more redundant RAID systems that feature RAID 6 and faster RAID controllers that speed up rebuild times by utilising the new Intel Jasper Forest CPUs in their construction.
Today, writing data to a HDD is very fast and whilst powered on very reliable too, however when powered off the health checking and Error Correction Code (ECC) on the HDD is also off. If HDDs are powered off for long periods of time the fluid used to lubricate the spindle motor can stick and cause the spindle motor to fail and the disk platters will also suffer from magnetic decay over time causing irretrievable data bit errors, resulting in data corruption that cannot be fixed. This means that a HDD with stored data, if left unmaintained, could suffer degradation in as little as 12 months, verses five years for a disk that powered up and in use (spinning).
With the above said HDDs are still a fast, reliable and inexpensive way to move large amounts of data around, but what about security? You’ve just done a shoot, all very important, highly valuable data that you do not want appearing on YouTube – what can be done?
Some external drive manufacturers have single disk products that have a keypad or finger print recognition technology, but the implementation is pretty basic and typically more hype than technical substance. A neat box, the FiT by AXUS Microsystems, is a desktop RAID for 2.5” or 3.5” HDDs that uses AES (Advanced Encryption Standard) technology with a key length of 256-bit (AES 256bit) to secure data against unauthorised access. By using such devices your RAID is protected against disk failures and the data stored across on all the disks is encrypted with the encryption/decryption algorithm performed in real-time without any performance degradation. Should the enclosure with the disks or just the disks be stolen they are useless without the encryption key – in the case of the AXUS FiT the encryption key is a USB stick that connects to RAID allowing the data and the ability to access that data to be transported and/or stored separately.
Encryption of data can also been undertaken by the HDD itself. Since 2007 HitachiGST has had the ability to encrypt/decrypt data on the disk, using an encryption ‘key’ that is actually generated off a pulse from the spindle motor, creating a truly unique encryption key that is impossible to replicate. To date this technology has little real-world implementation but the scope for integration in to disk-based storage systems in the future is vast.
Digital Data Tape in the form of Liner Tape Open (LTO) natively supports encryption on LTO-4 and LTO-5 tape drives. If you are not familiar with LTO data tape it is a digital tape cartridge of approximately 10 x 10 x 2cm in size with a shelf life of 30-years and the latest generation (LTO-5) can store 1.5TB of uncompressed data. Tape library manufacturers like Spectra Logic have a ‘BlueScale OS’ that allows you to create and manage encrypted sets of tapes within the library using industry standard AES 256-bit encryption keys. Thus if the data tapes are lost or stolen the data on the tapes is secure.
I’ve talked briefly about drives and data security but what about long term archive? This could be anything from graphics files, documents, still images or audio and video content. What strategies can be used to secure the preservation of digital assets for future generations?
Hard disk drives are not a good idea for long terms archive with very few manufactures of disk storage systems offering support or maintenance beyond five years. So what about Solid State Drives (SSD)? Are they not more reliable and more robust and therefore offer a long-term solution for drive-based data retention? Unfortunately not, for several reasons namely capacity of the disks is still low and the cost is still high, and the fact that SSDs also have write cycle and cell life span limitations. Wear level algorithms and cleaver ECC can help resolve these issues to a certain extent, but essentially the memory chip on the SSDs are built on cells and these have electrical charge applied to them which equals the 1 or 0 data bits for the information stored on the SSD. Over time memory cells start to wear and ‘leakage’ starts to occur, in this process the data bits can get flipped from 1 or 0 causing data corruption. If you are using SSD for offline archive scheduling in a power on cycle to allow error checking is good and essential practice. The last point to note about SSDs is that they are more vulnerable to damage as a result of voltage spikes.
What do I think is a happy medium? A hybrid workflow that includes both disk and tape is usually a common outcome. A workflow for a production company may feature a capture devise that uses SSDs for field recording but also some form of on location LTO archive – appliances and solutions by Atempo, Cache-A and XenData can be used to create source masters of the all important rushes. Back at the production company’s facility data can be moved from the field recording devices to the disk/RAID-based edit space or managed shared storage. Once editing is complete the finished project can be delivered on HDD or, as I suspect more commonly in time, delivered on LTO tape using the new Liner Tape File System (LTFS) format – an IBM developed format that allows a data tape to appear as a logical drive to the operating system.
Increasingly production companies are turning to digital data tape to secure their assets longer term with a view to repurposing and using them to generate further revenues in the future. Many post-production facilities are also offering archive services to their clients. Using tape for these long term archive applications is unsurprising considering that the bulk of research into the disk verse tape debate shows that disk systems cost on average 25 times more annually to power and cool than a similar tape system, and in some case the cost of energy alone for the average disk based solution exceeds the entire TCO of the average tape based solution.
This topic has such an immense depth that the above barely scratches the surface. Technology is advancing faster than ever before and in many cases diminishing budgets demand that IT managers spend their resources wisely. Often, throwing more disks at a problem is only a short term fix and can work out more expensive in the longer term than a more appropriate solution for data management and retention. Making sure you understand the RAID level you are working with and the implications that has to your workflow is vital, especially if there are tight deadlines involved. When backing up and archiving it is important to consider that a 1.5TB LTO5 tape costs less than £60, whereas a good quality external 1TB drive costs in the region of £100 or even more. Not only is the tape option cheaper but compared to the expected 3-5 year lifespan of a hard disk drive a LTO data tape cartridge has a 30-year media life and is far more environmentally friendly.
The rich media and content creation market is brimming with talented people producing outstanding work so it is as relevant as ever to remember that data is not just one’s and zero’s, it’s your art, your life.
Global Distribution, based in Cambridge is a value-added, specialist distributor of data storage and infrastructure solutions for data-intensive computing environments within the creative, post-production, broadcast, CCTV and high-performance computing markets. Dedicated to delivering the best new storage products and solutions to these markets, Global provide active support and development that is second to none.
Founder Keith Warburton started out in the industry by building and testing broadcast consoles before going on to design and install recording studios and subsequently develop and supply hard disk-based recording solutions for the Atari and Apple Macintosh platforms. During the mid-nineties seeing the convergence between audio and video Keith founded Global Distribution with a mission statement to work with leading manufacturers to make available the best technology for creative industries. Sixteen years on Global has a wealth of knowledge and a clear focus on understanding the ever-changing requirements of end-users, and are excellently placed to provide storage technologies that not only fulfill the needs of Global’s resellers and partners but ultimately their clients’ needs.

Tags: global distribution | archive | storage | assets | axus | lto | digital data tape | iss058 | N/A
Contributing Author N/A

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Global Distribution with mLogic at IBC 2013

    Global Distribution with mLogic at IBC 2013

  • Guided tour of the Global Distribution Stand at BVE 2016

    Guided tour of the Global Distribution Stand at BVE 2016

  • Global Distribution at BVE 2012

    Global Distribution at BVE 2012

  • Global Distribution at IBC2011

    Global Distribution at IBC2011

  • NOA Archive Solutions at IBC 2014

    NOA Archive Solutions at IBC 2014

  • Object Based Storage Solutions from Object Matrix at NAB 2017

    Object Based Storage Solutions from Object Matrix at NAB 2017

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • SGL Broadcast at IBC 2015

    SGL Broadcast at IBC 2015

  • NOA Audio Solutions at IBC 2013

    NOA Audio Solutions at IBC 2013

  • SGL at NAB 2013

    SGL at NAB 2013

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • Editshare at NAB 2012

    Editshare at NAB 2012

  • Nexsan at NAB 2012

    Nexsan at NAB 2012

  • Arkivum at BVE 2012

    Arkivum at BVE 2012

  • SGL at NAB 2014

    SGL at NAB 2014

  • ERA - Cloud Services - at BVE 2015

    ERA - Cloud Services - at BVE 2015

  • Primestream on BroadcastShow LIVE at IBC 2013

    Primestream on BroadcastShow LIVE at IBC 2013

  • Softron Media Services on BroadcastShow LIVE at IBC 2013

    Softron Media Services on BroadcastShow LIVE at IBC 2013

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Digital Vision on BroadcastShow LIVE at IBC 2013

    Digital Vision on BroadcastShow LIVE at IBC 2013

  • Cinegy at IBC 2013

    Cinegy at IBC 2013

  • Cinegy: Multiviewer at NAB 2013

    Cinegy: Multiviewer at NAB 2013

  • SGL at NAB 2012

    SGL at NAB 2012

  • TMD at BVE 2012

    TMD at BVE 2012

  • FOR-A at IBC2011

    FOR-A at IBC2011

  • Netia at IBC2011

    Netia at IBC2011

  • Facilis Technology Shared Storage at IBC 2018

    Facilis Technology Shared Storage at IBC 2018

  • All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

    All IP Workflow from Quantum with Xcellis Workflow Storage solutions at NAB 2018

  • Object based Storage Solution from Object Matrix at IBC 2017

    Object based Storage Solution from Object Matrix at IBC 2017

  • Glyph External Storage including the Studio and Atom raid at IBC 2017

    Glyph External Storage including the Studio and Atom raid at IBC 2017

  • Storage DNA at BVE 2016

    Storage DNA at BVE 2016

  • Fibrenetix with StorageDNA at IBC 2014

    Fibrenetix with StorageDNA at IBC 2014

  • Tiger Technology at BVE 2015

    Tiger Technology at BVE 2015

  • Fibrenetix with Quadrus at IBC 2014

    Fibrenetix with Quadrus at IBC 2014

  • Facilis at NAB 2014

    Facilis at NAB 2014

  • GB Labs Space at NAB 2014

    GB Labs Space at NAB 2014

  • ERA at BVE 2014

    ERA at BVE 2014

  • ERA Avere at BVE 2014

    ERA Avere at BVE 2014

  • Front Porch Digital on BroadcastShow LIVE at IBC 2013

    Front Porch Digital on BroadcastShow LIVE at IBC 2013

  • Sonnet Technologies on BroadcastShow LIVE at IBC 2013

    Sonnet Technologies on BroadcastShow LIVE at IBC 2013

  • EditShare on BroadcastShow LIVE at IBC 2013

    EditShare on BroadcastShow LIVE at IBC 2013

  • Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

    Harmonics Tom Lattie on BroadcastShow LIVE at IBC 2013

  • Aframe Cloud Video at IBC 2013

    Aframe Cloud Video at IBC 2013

  • Facilis at IBC 2013

    Facilis at IBC 2013

  • Facilis at NAB 2013

    Facilis at NAB 2013

  • Object Matrix at BVE 2013

    Object Matrix at BVE 2013

  • Facilis at BVE 2012

    Facilis at BVE 2012

  • Real Life Kit at ProVideo2011

    Real Life Kit at ProVideo2011

  • Suitcase TV at IBC2011

    Suitcase TV at IBC2011

  • Sonnet Technology at IBC2011

    Sonnet Technology at IBC2011

  • ATTO Technology at IBC2011

    ATTO Technology at IBC2011


Related Shows
  • StorageDNA LIVE at BVE 2016

    StorageDNA LIVE at BVE 2016


Articles
Making the Workflow Flow
Bruce Devlin - new The toughest things about being the Standards Vice President (SVP) is that everyone expects standards to be the most important thing. In all the systems that I’ve designed and deployed over the years, I've yet to find any production workflow that is 100% standards based. True, the core technologies may well be standards based, but the overall workflow is made up of many technology pieces from open source code, through de-facto delivery specifications based upon SMPTE or Trade Association Specifications that in turn depend on full, International Standards to work. I can already hear some folks saying "In the good old days, everything used standards", but I beg to disagree.
Tags: iss138 | pye museum | pye-philips | smpte | ietf | ieee | w3c | aes | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF
How Broadcasts Can Simplify The Delivery Of Live Video Streams To Affiliates
Phillipe Gonon In today’s fast paced environment broadcasters need to quickly, easily, and simultaneously share high-quality live content with multiple affiliates or other broadcast facilities. However, today it’s typical for each third-party transmitter to be connected to its own receiver with its own platform and management system. This results in major TV channels having several web interfaces and/or screens to receive and distribute feeds coming in from third-party devices.
Tags: iss138 | aviwest | stremaing | rtmp | rtsp | rtp | hls | Phillipe Gonon
Contributing Author Phillipe Gonon Click to read or download PDF
The World of 12G 4K/UHD Processing
Varun Patel Technology moves fast and when it comes to audio and video broadcasting, there is always room for improvements in audio and video quality and ease of production. One of the main goals of broadcasters and content providers today is to create an immersive experience for the viewers, giving them the feeling of being part of the viewed content. The 4K/UHD buzz has been in the consumer world for some time but what does that actually mean for the content producer?
Tags: iss138 | lynx technik | ott | greenmachine | yellobrik | conversion | Varun Patel
Contributing Author Varun Patel Click to read or download PDF
How the Womens World Cup was Boosted by Broadcast
John Griffiths As we write this article, England’s Lionesses have just beaten Norway 3-0 and are smashing viewing figure records, with 6.9 million viewers tuning in to watch them play Cameroon on BBC One. Rewind to the previous World Cup in 2015, and England’s group and early stage knockout games tempted up to 2.5 million viewers for each match. It’s safe to say that women’s football is finally having its moment; what was perhaps once considered a niche sport is finally gaining momentum in the mainstream space with broadcasters giving it the attention it deserves.
Tags: iss138 | wwc | world cup | spicymango | diversity | John Griffiths
Contributing Author John Griffiths Click to read or download PDF
Review: JVC GY-HC900 Camera
Andy McKenzie JVC has a long history of developing relatively low price-point equipment, dating right back to its success in the 1970s with the VHS tape format. JVC products sometimes impact the broadcast mainstream but generally find greatest success in niche market segments such as news production where price and portability take priority over the no-comprise quality and robustness demanded by the leading studio-based and OB content producers.
Tags: iss138 | jvc review | camera | sdhc | sdxc | mpeg-2 | nab | gy-hc900 | Andy McKenzie
Contributing Author Andy McKenzie Click to read or download PDF