Ask the experts with TMD looking at LTFS


Tony Taylor TV-Bay Magazine
Read ezine online
by Tony Taylor
Issue 80 - August 2013
Not another acronym! What is LTFS?
LTFS stands for Linear Tape File System. It is a file system for data archives, developed by IBM but published as an open standard and now widely adopted by all the leading vendors of large-scale data archiving.

If it is a data product, why am I reading about it in a broadcast magazine?

Because we have lots of data to archive. Indeed, so important is the broadcast and media archive market that LTFS was not launched at an IT event but at NAB, on 12 April 2010.

Still not sure I get the point of data archives. Why do I even need to keep my content?

Taking the last point first, for most people the need to keep content is clear: you may make money from it in the future. You have the opportunity to sell programmes to new channels and new delivery platforms which we have not even heard about.

Have a look around the more obscure corners of your EPG and see how many broadcasters are serving up content from any back catalogue they can get hold of: programmes which are 40 or 50 years old in some cases.

It is not just whole programmes you can sell: you might get some business in clips from your content. What if you have a particularly spectacular shot, or the first appearance of an actor who goes on to be very famous? Good to be able to drag it out of the archive and sell it for a modest fee.

Some people like major broadcasters and audiovisual archives are committed to keeping large amounts of content as an important part of our social and cultural history. We take it for granted today that we can watch clips from the 1948 London Olympics, or the coronation in 1953.

What is wrong with archiving the original material?
In broadcast, formats come and go very quickly indeed. If you have material on 2 quad or 1 B-format tape, then it is virtually inaccessible today unless you are prepared to pay a lot of money for an expert restoration company to transfer it. In the last decades of the 20th century we saw new video formats launched at virtually every NAB, most of which are unavailable today.

One of the most pressing reasons for anyone to implement an asset management system today is to maintain access to modern content: it will get increasingly hard to source BetaSP and DigiBeta VTRs in the coming years.

So it makes sense to digitise and archive as data. Especially as the IT industry has vastly more R&D resources than we do, so will come up with a well-designed standard that we can just adopt for ourselves.
I thought that was LTO tape?
Indeed, LTO tape is an excellent solution, a well-designed standard that we can just adopt for ourselves. Many media archives already use LTO tape libraries. The family is now up to LTO-6, with each new generation doubling (roughly speaking) the storage density, helping with the huge amounts of data we keep generating.

So problem solved, then?
Well no. LTO is a storage system, a way of keeping the data. But it is not an archive in itself. That would be like saying we can just buy a Raid array and it will be an archive it wont.

What you need is some way of describing what is on the tape, so we can make sense of it later. At the moment that is some sort of archive software, which may be part of the asset management platform or it might be in a separate hierarchical storage application. It knows what is on each tape. When LTO-7 comes along two tapes may become one, and the archive software would keep track of that.

The problem, though, is one of longevity. 50 years from now when you want to recover something you have not seen in decades, you will probably be using an asset management system which is eight generations on from where we are today. Will that still know where your stuff is? Is that a good way to manage platform migration?

And that is where LTFS comes in?
LTFS, remember, is the Linear Tape File System. The important point here is that it is a file system. Specifically, it is a self-describing file system: it defines the organisation of both essence data and metadata on each tape.

So a tape written in the LTFS format can be used independently of any external database or storage system. You simply plug an LTFS-format tape into a drive and it can be read without problems, either by the drives own application or by any asset management system. The files on the tape are fully described.

So assets are truly portable, between systems and over time. You can keep a tape for 50 years or more and it can be loaded into whatever we will be calling asset management in 2060. Or you can take an LTFS tape from your asset management system, plug it into a different system somewhere else, and it instantly becomes useful.

It does not completely populate the asset management system, because that would involve making it specific to a particular system, which is what we are trying to avoid. The file system is deliberately simple to guarantee portability.

That also means that, should your main archive be destroyed by fire or flood, you could pick up your offsite backup LTFS tapes and be back on air, in any facility which can give you space, as soon as you load the tapes
What happens if new storage mechanisms come along?
Some vendors are talking about Maid storage: the massive array of idle disks. Others suggest that holographic storage may be the way of the future.

But LTFS is a file system, so it has to be open to any changes in storage technology, because we know that LTO will develop in the future. That means it could just as easily be applied to Maid or holography or whatever else comes along. It is a file system: it describes what is on the storage unit, whether that is a tape or a disk or anything else.

Most people for now will be using it with LTO drives and robots. But its simplicity also makes it very flexible.

If it is as good as it sounds, it has to be expensive, complex or both
LTFS is a widely supported format in the IT industry, developed originally by IBM and maintained by a broad consortium of manufacturers. It is an open standard, freely licensed to anyone who wants to implement it.

Being part of the IT industry, it benefits from the huge R&D budgets available, so products supporting it are already available, including LTO drives and robots. It means that broadcast vendors can concentrate our investment where it is really important: on the applications which run on this commodity hardware.

At TMD we are already implementing our first large-scale archive project which is using LTFS to future-proof a huge amount of content. Others are sure to follow.


Tags: iss080 | ltfs | guide | tape | backup | archive system | tmd | data archives | ibm | Tony Taylor
Contributing Author Tony Taylor

Read this article in the tv-bay digital magazine
Article Copyright tv-bay limited. All trademarks recognised.
Reproduction of the content strictly prohibited without written consent.

Related Interviews
  • Tony Taylor from TMD talks about LTFS at IBC 2013

    Tony Taylor from TMD talks about LTFS at IBC 2013

  • SGL at NAB 2014

    SGL at NAB 2014

  • SGL at NAB 2013

    SGL at NAB 2013

  • Guided Tour of the Polecam Stand at BVE 2016

    Guided Tour of the Polecam Stand at BVE 2016

  • Guided tour of the Global Distribution Stand at BVE 2016

    Guided tour of the Global Distribution Stand at BVE 2016

  • Guided tour of the Holdan Stand at BVE 2016

    Guided tour of the Holdan Stand at BVE 2016

  • TMD talk workflow on BroadcastShow LIVE at IBC 2013

    TMD talk workflow on BroadcastShow LIVE at IBC 2013

  • PAG talk about flying with batteries at IBC 2014

    PAG talk about flying with batteries at IBC 2014

  • Triveni Digital at NAB 2012

    Triveni Digital at NAB 2012

  • Workflow Solutions from Pronology at NAB 2017

    Workflow Solutions from Pronology at NAB 2017

  • Pronology at IBC 2014

    Pronology at IBC 2014

  • PacTV Truck at NAB 2014

    PacTV Truck at NAB 2014

  • Blue Lucy Media at IBC 2013

    Blue Lucy Media at IBC 2013

  • Marquis Broadcast show Backup Versioning at NAB 2018

    Marquis Broadcast show Backup Versioning at NAB 2018

  • Facilis Technology at BVE 2013

    Facilis Technology at BVE 2013

  • TMD Mediaflex-UMS at IBC 2017

    TMD Mediaflex-UMS at IBC 2017

  • TMD Corprate Update from IBC 2017

    TMD Corprate Update from IBC 2017

  • Mediaflex UMS Platform from TMD at NAB 2017

    Mediaflex UMS Platform from TMD at NAB 2017

  • New CEO and news update from TMD at NAB 2017

    New CEO and news update from TMD at NAB 2017

  • TMD at IBC 2016

    TMD at IBC 2016

  • TMD Mediaflex-UMS at IBC 2016

    TMD Mediaflex-UMS at IBC 2016

  • TMD at NAB 2016

    TMD at NAB 2016

  • TMD Mediaflex at NAB 2016

    TMD Mediaflex at NAB 2016

  • TMD at BVE 2016

    TMD at BVE 2016

  • TMD news update at IBC 2015

    TMD news update at IBC 2015

  • TMD Mediaflex at IBC 2015

    TMD Mediaflex at IBC 2015

  • TMD Chameleon at NAB 2015

    TMD Chameleon at NAB 2015

  • TMD Mediaflex at NAB 2015

    TMD Mediaflex at NAB 2015

  • TMD at BVE 2015

    TMD at BVE 2015

  • TMD summarises their work this year at IBC 2014

    TMD summarises their work this year at IBC 2014

  • TMD Mediaflex at IBC 2014

    TMD Mediaflex at IBC 2014

  • TMD at IBC 2014

    TMD at IBC 2014

  • TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

    TMD talk asset management solutions on BroadcastShow LIVE at IBC 2013

  • Tony Taylor from TMD talks about Mediaflex CI

    Tony Taylor from TMD talks about Mediaflex CI

  • Tony Taylor from TMD talks about Post Production

    Tony Taylor from TMD talks about Post Production

  • TMD at NAB 2013: MediaFlex Reporting Module

    TMD at NAB 2013: MediaFlex Reporting Module

  • TMD at NAB 2013: MediaFlex Systems

    TMD at NAB 2013: MediaFlex Systems

  • TMD at NAB 2013: Content Intelligence

    TMD at NAB 2013: Content Intelligence

  • TMD at IBC 2012

    TMD at IBC 2012

  • TMD at NAB 2012

    TMD at NAB 2012

  • TMD at BVE 2012

    TMD at BVE 2012

  • TMD at IBC2011

    TMD at IBC2011


Related Shows
  • Talking Streaming with Stream UK: BVE Day 3

    Talking Streaming with Stream UK: BVE Day 3


Articles
Streaming - Shut up and Sit Down
Matt Lees Twilight Imperium is a board game all about strategy and backstabbing. Known for its length of gameplay, the team behind Shut Up and Sit Down decided to deliver a mammoth 10-hour live broadcast, streamed to Twitch. Matt Lees is here to tell us how it went down.
Tags: iss136 | streaming | blackmagic | atem | 4k | canon c100 | sony pxw-z90 | Matt Lees
Contributing Author Matt Lees Click to read or download PDF
State of the Nation - Drama
Dick Hobbs - new If you can remember as far back as the 1980s, then you probably remember a Saturday night drama called Hill Street Blues, written and developed by Steven Bochco. Made for NBC in the States, it was widely shown around the world, including in the UK.
Tags: iss136 | drama holby city | night manager | netflix | bbc | ai | streaming | podcast | Dick Hobbs - new
Contributing Author Dick Hobbs - new Click to read or download PDF
Looking for the Silver Lining
Harry Grinling According to the World Meteorological Organisation, there are 10 different types of cloud, each of which can be divided further into sub-types. They range from the cirrus, the thin floaty clouds which generally serve only to make the sky look beautiful to the towering, all-embracing cumulonimbus which can deliver fearful quantities of rain – the biggest cumulonimbus clouds can contain 50 million tonnes of water.
Tags: iss136 | cloud | lto | archive | storage | Harry Grinling
Contributing Author Harry Grinling Click to read or download PDF
The way forward in a changing world
Alan Wheable The broadcast and media industries are evolving in all areas to adapt to the challenges of competition from OTT, changing user viewing habits, technological advances in image resolution, dynamic range and colour rendition as well as embracing new video over IP infrastructures.
Tags: iss136 | omnitek | aims | ip | smpte | 2110 | 4k qc | ultra xr | nmos | ultra tq | Alan Wheable
Contributing Author Alan Wheable Click to read or download PDF
NAB Intelligence
Bruce Devlin - new It's that time of year again. 06:30 on road bikes on the Las Vegas Strip with a merry bunch of folks who believe they look good in lycra. As we pedal West up the hill towards Red Rock there will be stories of the year just gone and questions about what to look for at the show. I, for one, will be attending the Devoncroft event to check up on who is spending what and where. I will then look forward to the topics that might be buzzing around the halls. I predict there will be many displays of Machine Learning hiding under the banner of Artificial Intelligence. Some of these will show better picture quality, others will show personalisation engines, some will be improved search tools and yet more will be synthesising pictures and motions to replace reality with and Artificial reality that's more compelling for the viewer.
Tags: iss136 | nab | class | ai | smpte | st2110 | imf | Bruce Devlin - new
Contributing Author Bruce Devlin - new Click to read or download PDF