PNNL, SGI Think Big

The Department of Energy’s Pacific Northwest National Laboratory (PNNL) and Silicon Graphics have partnered to develop new storage technology for data-intensive computing.

The two will conduct research into “active storage,” an effort to shift computation and transformation of data from client computers to storage devices. Eng Lim Goh, SGI’s chief technology officer, claims the effort “holds the promise of dramatic productivity breakthroughs for a broad range of computing disciplines saddled by large data.”

The alliance, which is part of a long-term collaboration between PNNL and SGI, includes options for more than 2.5 petabytes of storage over the next two years. As part of the first phase, SGI Professional Services will deliver a 380 terabyte file system this summer to the William R. Wiley Environmental Molecular Sciences Laboratory located at PNNL.

PNNL scientists will be able to take raw data sets stored on the file server and conduct computations to identify data signatures and patterns before the data is transferred to client systems.

“By developing methods to perform computing inside the file system, we will be able to reduce the amount of redundant data transfers, which routinely undermines productivity and lengthens the time to solution,” states Scott Studham, PNNL associate director for advanced computing.

The new file system is expected to sustain write rates in excess of 8Gbps and demonstrate single client write rates higher than 600Mbps. The system will leverage Lustre, an open source, object-oriented file system with development lead by Cluster File System Inc., with funding from the Department of Energy. Lustre is used on four of the top five supercomputers, including the PNNL cluster based on 1,900 Intel Itanium 2 processors.

SGI also plans to evaluate how the research effort can contribute to the evolution of the company’s SGI InfiniteStorage CXFS shared file systems.

Pushing the Boundaries

“The scientific and HPC community continues to push the boundaries of physical and software computing systems by creating and consuming ever larger data structures,” states William Hurley, senior analyst at the Enterprise Storage Group’s Enterprise Application Group.

The SGI product combination “puts SGI ahead of many storage and system vendors offering hardware or software … that are meant to provide a simplified, yet resilient means to access distributed data,” Hurley told Enterprise Storage Forum.

Hurley says the use of the Lustre and CXFS file system as a base Data Abstraction Layer provides common, shared access to a large data set while reducing the operating burden on distributed network and compute resources.

“The desire of many large and small organizations to achieve this operating condition for their file and relational data is high,” he says. “Only a handful of companies are delivering this type of functionality today.”

“Giving commercial organizations the ability to deploy a Data Abstraction Layer provides app developers, business analysts and IT operations staff a standardized data medium that expedites app development and reduces data errors that cause inaccurate reporting outcomes, while easing the burden on physical system resources and their management,” Hurley says.

Back to Enterprise Storage Forum

Paul Shread
Paul Shread
eSecurity Editor Paul Shread has covered nearly every aspect of enterprise technology in his 20+ years in IT journalism, including an award-winning series on software-defined data centers. He wrote a column on small business technology for, and covered financial markets for 10 years, from the dot-com boom and bust to the 2007-2009 financial crisis. He holds a market analyst certification.

Latest Articles

Storage Software Q&A With Chris Schin of HPE

Storage software technology continues to undergo rapid shifts. As enterprises' data needs multiply, storage providers have scaled their software products, so customers can optimize...

What Is Virtual Memory? Ultimate Guide on How It Works

Virtual Memory allows a computer more memory than physically available. Learn how it works & how it differs from physical memory. Click here now.

What is Blob Storage? Definition, Types, & How It Works

Microsoft Azure Blob Storage makes stored data available to businesses over the internet. While data can be of any type, blob storage is particularly...