IBM Research goes Hollywood with Emmy win
Linear tape technology to revolutionize media storage
In a day where digitized movies, TV shows, and YouTube videos are available at the swipe of a screen, consumers more than ever have the luxury of accessing large amounts of data when and where they want it. But storing this content economically and sustainably has been a major challenge for media and entertainment companies who need quick and easy access to their data for mass distribution. In fact, in the next five years, digitized media will account for 24 percent of the growth in the media and entertainment industry1.
The surprising answer for this digital problem? Tape
IBM Research satisfied this long-standing storage deficiency in the media & entertainment industry with the Linear Tape File System. Whereas companies previously couldn't store mountains of movies and digital media at low cost with standardized access, The Linear Tape File System offers an inexpensive hardware package that allows companies to store about 50 hours of XDCAM HD video on a 1.5TB cartridge. The archive and retrieval cost is that of setting a movie reel on a shelf - zero.
Disk has a bit error rate three orders of magnitude higher than tape – meaning you will go through one thousand times more data on tape before you come across one error.
The research team, with more than three decades of expertise in tape storage, came up with the idea to create a cross-platform, open source system for tape, and worked quickly to launch the Linear Tape File System. After a very successful demo of the prototype to customers at the National Association of Broadcasters (NAB) 2009 show, IBM's Systems & Technology Group worked with Research to make it a product offering. Just over a year later, it earned a "Pick Hits" Award at the NAB 2011 show.
When FOX Networks Group used the Linear Tape File System, IBM earned its first Emmy Award for "improving the ability of media companies to capture, manage and exploit content in digital form, fundamentally changing the way that audio and video content is managed and stored."
Workflow integration for instant and easy access
In the past, for a company to store content on tape, it meant that their data was 'stuck' in one application or operating system and wasn't accessible across different vendors' platforms. A single vendor would typically hold on to the data and companies couldn't easily work with the content for editing and production purposes
If footage from a sports event was lost or damaged, the content could never be reproduced. In fact, because of this, sports television producers would copy the same video footage onto two separate hard disk drives and deliver them to the studio via two different couriers to ensure that the content would arrive safely for editing and distribution.
Also, the same hard disk drives weren't guaranteed to start up decades later to retrieve archived data.
“ The idea of a file system that was cross-platform and interoperable was key; we wanted people to have an interface they were familiar with, similar to a disk with file folders, drag and drop and double-click. But we also wanted to make sure it wasn't tied to only Windows or only Unix. ”
David Pease, Storage Systems Research Manager
With no tape system interchange standard established to enable rapid, easy to use data sharing and retrieval, IBM researchers felt that the technology was available to make such a methodology a reality.
The Linear Tape File System breathed life back into tape systems by allowing content producers to operate on a file-based workflow - similar to how a user accesses files on their personal computer - where digital files could easily be integrated across Mac, Windows or Linux, speeding operations and providing more flexibility. This way, files could be uploaded and shared automatically and virtually, versus physically, and could potentially be accessible decades later.
Ending vendor lock-in
The Linear Tape File System gives companies a standardized, open format that provides easy interface into their new file-based workflows as well as flexibility across platforms. By mimicking a removable external disk, it can be considered a large-scale cousin to DVD-RW or USB memory sticks with directory tree access and file-level drag-and-drop capability, also allowing information to be passed from one system or employee to another.
The next step for the team is to merge the Linear Tape File System with IBM's General Parallel File System to create a seamless storage tiering system. It would eliminate the manual data transfer process between disk and tape while allowing components of each system to either idle or activate depending on usage. Systems like this are vital to industries beyond media and entertainment, including finance and banking, video surveillance, medical imaging and even oil exploration, where multiple sensors pick up images on the order of 10 terabytes in size per file, and have to be kept for analysis over time.
Share this story
Explore this topic
Meet the researchers
Senior Technical Staff Member,
Manager, Exploratory Storage Systems,
IBM Research - Almaden