Pipes Filters Architecture Pattern Source PatternOriented Software Architecture

Pipes & Filters Architecture Pattern Source: Pattern-Oriented Software Architecture, Vol. 1, Buschmann, et al

Problem • You are designing a system that needs to perform several transformations on some data • The transformations can be performed sequentially • It should be easy to: – Reorder the sequence of the transformations – Add new kinds of transformations – Remove transformations • There are different sources of input data (files, network, programs) • There are different destinations for output data (files, network, programs)

Solution • • Organize the system using Pipes & Filters Input data originates from Data Source components Output data is consumed by Data Sink components Each transformation required to convert the input into the output is implemented by a Filter component • Data Sources, Filters, and Data Sinks arranged in a pipeline • Pipes connect adjacent components, the output of one component becoming the input to the next component

Solution • Sources, Filters, and Sinks can be Active or Passive • Active components run on their own thread of control • Passive components execute only when invoked, directly or indirectly, by an Active component

Dynamic Behavior : Scenario I • Active Source / Passive Filter / Passive Sink

Dynamic Behavior : Scenario II • Passive Source / Passive Filter / Active Sink

Dynamic Behavior : Scenario III • Passive Source / Active Filter / Passive Sink

Dynamic Behavior : Scenario IV Buffering Pipe • Passive Source / Multiple Active Filters / Passive Sink

Implementation • Divide the system into a sequence of processing stages • Define the data format to be passed along each pipe • Decide how to implement each pipe connection – The simplest Pipe is direct Read/Write method calls between adjacent filters – If buffering or synchronization is required between filters, actual Pipe objects will be needed to perform these functions

Implementation • Design and implement the filters – Passive filters can be implemented as regular methods – Active filters can be implemented as processes or threads • Design the error handling – In pipelines with one active component, standard error handling mechanisms can be used (exceptions, return codes, etc. ) – In pipelines with multiple active components, how will an error in one thread of control become visible to other threads? – If an error occurs, we could: • Tear down the entire pipeline and restart it from scratch, or • Restart only the parts that failed and resynchronize the pipeline • Setup the pipeline

Known Uses: UNIX Command Pipelines cat fall. txt win. txt | sort | gzip | mail fred@byu. edu

Known Uses: Image Processing

Known Uses: Compilers

Consequences • Very flexible – Filters can be reused and recombined in arbitrary ways – Individual filters can be easily replaced (e. g. , plug in a different kind of compression) • No intermediate files necessary between processing stages • Benefits from efficiencies inherent in parallel processing (multiple active components) • Some filters don’t produce any output until they've consumed all of their input (e. g. , sort), which is not conducive to parallelism • Data transfer and context switching between processes and threads can be expensive