Dartmouth College Computer Science
Technical Report series
TR search TR listserv
|By author:||A B C D E F G H I J K L M N O P Q R S T U V W X Y Z|
|By number:||2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, 1998, 1997, 1996, 1995, 1994, 1993, 1992, 1991, 1990, 1989, 1988, 1987, 1986|
Most current multiprocessor file systems are designed to use multiple
disks in parallel, using the high aggregate bandwidth to meet the growing
I/O requirements of parallel scientific applications. Most multiprocessor
file systems provide applications with a conventional Unix-like
interface, allowing the application to access those multiple disks
transparently. This interface conceals the parallelism within the file
system, increasing the ease of programmability, but making it difficult
or impossible for sophisticated application and library programmers to
use knowledge about their I/O to exploit that parallelism. In
addition to providing an insufficient interface, most current
multiprocessor file systems are optimized for a different workload than
they are being asked to support.
In this work we examine current multiprocessor file systems, as well as
how those file systems are used by scientific applications. Contrary to
the expectations of the designers of current parallel file systems, the
workloads on those systems are dominated by requests to read and write
small pieces of data. Furthermore, rather than being accessed
sequentially and contiguously, as in uniprocessor and supercomputer
workloads, files in multiprocessor file systems are accessed in regular,
structured, but non-contiguous patterns.
Based on our observations of multiprocessor workloads, we have designed
Galley, a new parallel file system that is intended to efficiently
support realistic scientific multiprocessor workloads. In this work, we
introduce Galley and discuss its design and implementation. We describe
Galley's new three-dimensional file structure and discuss how that
structure can be used by parallel applications to achieve higher
performance. We introduce several new data-access interfaces, which
allow applications to explicitly describe the regular access patterns we
found to be common in parallel file system workloads. We show how these
new interfaces allow parallel applications to achieve tremendous
increases in I/O performance. Finally, we discuss how Galley's new file
structure and data-access interfaces can be useful in practice.
Bibliographic citation for this report: [plain text] [BIB] [BibTeX] [Refer]
Or copy and paste:
Nils Nieuwejaar, "Galley: A New Parallel File System for Parallel Applications." Dartmouth Computer Science Technical Report PCS-TR96-300, November, 1996.
Notify me about new tech reports.
Search the technical reports.
To receive paper copy of a report, by mail, send your address and the TR number to reports AT cs.dartmouth.edu
Copyright notice: The documents contained in this server are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Technical reports collection maintained by David Kotz.