BIB-VERSION:: CS-TR-v2.0
ID:: ncstrl.dartmouthcs//TR2007-596
ENTRY:: June 14, 2007
ORGANIZATION:: Dartmouth College, Computer Science
TITLE:: When One Pipeline Is Not Enough
TYPE:: Technical Report (paper)
REVISION:: 1
AUTHOR:: Cormen, Thomas H.
AUTHOR:: Natarajan, Priya
AUTHOR:: Davidson, Elena Riccio
DATE:: June 2007
RETRIEVAL:: For a paper copy, email <reports@cs.dartmouth.edu>
RETRIEVAL:: For a paper copy, write to
       Technical Report Librarian
       Department of Computer Science
       Dartmouth College
       6211 Sudikoff Laboratory
       Hanover, NH 03755-3510
       USA
RETRIEVAL:: PDF at http://www.cs.dartmouth.edu/reports/TR2007-596.pdf
ABSTRACT:: 
Pipelines that operate on buffers often work well to mitigate the high
latency inherent in interprocessor communication and in accessing data
on disk.  Running a single pipeline on each node works well when each
pipeline stage consumes and produces data at the same rate.  If a
stage might consume data faster or slower than it produces data, a
single pipeline becomes unwieldy.
  We describe how we have extended the FG programming environment to
support multiple pipelines in two forms.  When a node might send and
receive data at different rates during interprocessor communication,
we use disjoint pipelines that send and receive on each node.  When a
node consumes and produces data from different streams on the node, we
use multiple pipelines that intersect at a particular stage.
Experimental results for two out-of-core sorting algorithms---one
based on columnsort and the other a distribution-based
sort---demonstrate the value of multiple pipelines.
END:: ncstrl.dartmouthcs//TR2007-596