Disk-directed I/O for an Out-of-core Computation


David Kotz. Disk-directed I/O for an Out-of-core Computation. Technical Report number PCS-TR95-251, Dartmouth Computer Science, January 1995. ©Copyright David Kotz. Later revised as kotz:lu.


New file systems are critical to obtain good I/O performance on large multiprocessors. Several researchers have suggested the use of collective file-system operations, in which all processes in an application cooperate in each I/O request. Others have suggested that the traditional low-level interface (read, write, seek) be augmented with various higher-level requests (e.g., read matrix), allowing the programmer to express a complex transfer in a single (perhaps collective) request. Collective, high-level requests permit techniques like two-phase I/O and disk-directed I/O to significantly improve performance over traditional file systems and interfaces. Neither of these techniques have been tested on anything other than simple benchmarks that read or write matrices. Many applications, however, intersperse computation and I/O to work with data sets that cannot fit in main memory. In this paper, we present the results of experiments with an “out-of-core” LU-decomposition program, comparing a traditional interface and file system with a system that has a high-level, collective interface and disk-directed I/O. We found that a collective interface was awkward in some places, and forced additional synchronization. Nonetheless, disk-directed I/O was able to obtain much better performance than the traditional system.

Citable with [BibTeX]

Projects: [starfish]

Keywords: [pario]

Available from the publisher: [page]

Available from the author: [bib] [pdf] [ps.gz]
This pdf was produced by the publisher and its posting here is permitted by the publisher.

[Kotz research]