Armada: a parallel I/O framework for computational grids

Ron Oldfield and David Kotz

Department of Computer Science
Dartmouth College

Objectives.

Develop an I/O framework that allows data-intensive applications to efficiently access geographically distributed data sets.

Problem Definition.

An exciting trend in high-performance computing is the use of geographically-distributed networks of heterogeneous systems and devices known as "computational grids". Of particular interest is the development of data-intensive grid applications. Data-intensive grid applications require access to large (terabyte-petabyte) remote data sets, and often require significant preprocessing of filtering before the computation can take place. Such applications exist in many areas of scientific computing including seismic processing, climate modeling, particle physics, and others

Technical Approach.

The Armada framework for parallel I/O provides a solution for data-intensive applications in which the application programmer and data-set provider deploy a network of application-specific and data-set-specific functionality across the grid. Using the Armada framework, grid applications access remote data by sending data requests through a graph of distributed application objects. A typical graph consists of two distinct portions: one that describes the layout of the data (as defined by the data provider), and one that describes the interface and processing required by the application. Armada appends the two portions to form a single graph, restructures the resulting graph to distribute computation and data flow, and assigns modules to grid processors so that data-reducing modules execute near the data source and data-increasing modules execute near the data destination. The figures below show the original and restructured graphs for an application accessing a replicated and distributed data set.

Figure 1: A typical Armada graph consists of a portion from the data provider that describes the layout of the data and a portion from the application that describes required preprocessing.

Figure 2: Armada restructured the graph to provide end-to-end parallism and it moved the filter code close to the data source. The grey blobs represent the three LANs used by the application.

A key feature of Armada is the ability to restructure the application graph. Unlike existing database systems that restructure graphs based on well-known relational operators, Armada's restructuring algorithm uses programmer-defined properties to allow restructuring of a wide range of application classes (not just relational databases).

Perhaps the most critical part of the Armada system is the placement algorithm. Modules that make up the graph execute on processors near the client, processors near the data, or intermediate network processors. Our approach is to treat placement as a hierarchical graph-partitioning problem. We first partition the graph into domains in such a way to minimize the data transferred between domains. Next, we partition the modules in each domain to available processors provided by each domain's local resource manager.

Open Problems.

Armada is not a fully-featured I/O system. In particular, it lacks support for data management, security, and fault tolerance, which are necessary in a production grid system. While there are active research groups working in each of these areas, there is still much debate over proper solutions. In the Conclusions and Future Work section of the dissertation (oldfield:thesis), we discuss the issues involved in building support for each of these features in the Armada I/O system. In addition, we present potential improvements to the placement algorithm, and describe how to adapt Armada to work efficiently on tightly-connected clusters of workstations.

Papers

oldfield:armada:: Ron Oldfield and David Kotz. Armada: A parallel file system for computational grids. In Proceedings of the First IEEE/ACM International Symposium on Cluster Computing and the Grid, pages 194-201, Brisbane, Australia, May 2001. IEEE Computer Society Press.
oldfield:armada-wip:: Ron Oldfield and David Kotz. The Armada framework for parallel I/O on computational grids. Work-in-progress report at the Conference on File and Storage Technologies, January 2002.
oldfield:emulab-tr:: Ron Oldfield and David Kotz. Using the Emulab network testbed to evaluate the Armada I/O framework for computational grids. Technical Report TR2002-433, Dept. of Computer Science, Dartmouth College, Hanover, NH, September 2002.
oldfield:framework:: Ron Oldfield and David Kotz. Armada: a parallel I/O framework for computational grids. Future Generation Computing Systems (FGCS), 18(4):501-523, March 2002.
oldfield:restruct:: Ron Oldfield and David Kotz. Improving data access for computational grid applications. Cluster Computing, 9(1):79-99, January 2006, DOI 10.1007/s10586-006-4899-7.
oldfield:thesis:: Ron Oldfield. Efficient I/O for Computational Grid Applications. PhD thesis, Dept. of Computer Science, Dartmouth College, May 2003. Available as Dartmouth Computer Science Technical Report TR2003-459.
See also later version oldfield:thesis-tr.
oldfield:thesis-tr:: Ron Oldfield. Efficient I/O for computational grid applications. Technical Report TR2003-459, Dept. of Computer Science, Dartmouth College, May 2003.
See also earlier version oldfield:thesis.

Talks

oldfield:armada-talk-dartmouth2003:: Ron Oldfield and David Kotz. High-performance I/O for computational grid applications. Department of Computer Science graduate student colloquium, Dartmouth College, Hanover, NH, January 2003.
oldfield:armada-talk-dartmouth98:: Ron Oldfield and David Kotz. The Armada parallel file system. Department of Computer Science graduate student colloquium, Dartmouth College, Hanover, NH, October 1998.
oldfield:armada-talk-sandia1998:: Ron Oldfield. The Armada parallel file system. Invited talk at Sandia National Laboratories, Albuquerque, NM, October 1998.
oldfield:armada-talk-sandia2000:: Ron Oldfield. Flexible Parallel I/O for Computational Grids. Invited talk at Sandia National Laboratories, Albuquerque, NM, March 2000.
oldfield:armada-talk-sandia2003:: Ron Oldfield. High-performance I/O for computational grid applications. Invited talk at Sandia National Laboratories, Albuquerque, NM, January 2003.
oldfield:datagrids-talk-microsoft:: Ron Oldfield and Reagan Moore. Summary of Existing Data Grids (Grids for a federation of data, Applications that use the grid). Invited talk for the plenary session of the Fourth Grid Forum, Microsoft, Redmond, WA, July 2000.
oldfield:seismic-talk-dartmouth96:: Ron Oldfield, Curtis Ober, and David Womble. Parallel I/O for seismic imaging. Department of Computer Science colloquium, Dartmouth College, Hanover, NH, September 1996.
oldfield:seismic-talk-sc95:: Ron Oldfield. Generating a large synthetic seismic dataset on the Paragon Supercomputer. Invited talk at birds-of-a-feather session of SC'95, San Diego, CA, November 1995.
oldfield:thesis-defense:: Ron Oldfield. Efficient I/O for computational grid applications. Department of Computer Science colloquium, Dartmouth College, Hanover, NH, May 2003.
oldfield:thesis-proposal-talk:: Ron Oldfield and David Kotz. Flexible parallel I/O for computational grids. Ph.D. thesis proposal for the Department of Computer Science, Dartmouth College, Hanover, NH, May 2000.

Other papers and talks

Some other papers, possibly overlapping list above:

File translated from T_EX by T_TH, version 3.13.
On Mon Mar 2 00:46:18 EST 2009 .