Iteratee IO--Safe,Practical,Declarative,IncrementalInputProcessing

Date: 2009-12-17

Time: 11:30 - 12:20

Room: BBL 471, at Princetonplein 5.

Speaker: Oleg Kiselyov

Title: Iteratee IO -- safe, practical, declarative, incremental input processing

We introduce input processing with left-fold enumerator --Iteratee IO-- as a safe, declarative and practical alternative to Handle and Lazy IO for input processing. The approach is general and applies to processing data from various collections, from in-memory data structures to databases, files, sockets, etc. Binary and random IO is supported. The input data may come from a file or from an embedded (e.g., chunk-encoded or encrypted) stream; the depth of embedding is arbitrary. The approach is naturally incremental. It permits IO interleaving without any unsafe operations. The approach is algebraic and declarative.

We focus on:

• high-performance: block-based i/o, low-latency, incremental processing in constant memory;
• correctness: predictable resource usage, timely deallocation, preventing access to disposed resources;
• elegance: arbitrary nesting of processing layers; vertical and horizontal compositions of enumerators and iteratees; writing input processing without worrying about input buffer underflows, chunk- and UTF-8 decoding.

We achieve both high performance and encapsulation of input processing layers that can be freely composed. The technique has been validated in a production web-application server written in Haskell, database access library Takusen, TIFF and WAVE file readers. The performance of the Iteratee library on reading WAVE files can exceed the performance of the C-based libsnd library.

More information, including the complete code, is available from:

• http://okmij.org/ftp/Streams.html
• http://okmij.org/ftp/Haskell/Iteratee/README.dr
• http://johnlato.blogspot.com/2009/02/build-better-wave-reader-pt-2.html