One of the strengths of the &unix; operating system is its philosophy of having many small, simple tools which perform specific tasks well. Users can compose commands to peform more complex computations by combining these small tools at the command line, by way of a pipeline. This is especially true of text processing, where the tools can act as "filters", consuming some text from its input, and producing a modified version of the text on its output.

Back in the early days of &unix;, text processing typically consisted of manipulating flat text files. This data file format is simply a collection of ASCII-based records, separated by newline characters. The standard text processing tools, such as grep, awk and sed, all focus on this flat text file format.

There are, however, problems with dealing solely with flat text. Firstly, not all data easily transform into this format; kludges must be made to fit the data into records. As an example, the hierarchical structure of a directory tree would benefit from a textual representation that could accommodate the nesting of directory entries in other directory entries.

Secondly, some programs ignore the inherent record-based nature of their output and instead format it to be friendly for the user. To process this requires the use of text filtering programs solely to extract the relevant information from the formatted output. A prime example of this would be &unix;'s ls -l command. The output from this command contains a date field whose format can change according to its difference from the current date. Such conditional formatting makes it difficult to consistently parse the information.

This project aims to look at the use of structured text for standard program input and output, in particular the Extensible Markup Language (XML), as a means of solving these problems. Following on from this, an environment in which to run these programs will also be devised.

The idea of shifting the focus of &unix; process I/O from flat text to a more structured format has not been investigated greatly. Kernighan & Plauger (1976) introduced the idea of the "software tool" — the small program which performed its task, and only its task. Descriptions of the standard &unix; text processing tools and combining these using the shell are presented . However, all of these tools are designed for flat text processing.

The beginnings of structured, marked up document processing lie in the Standard Generalized Markup Language (SGML) . SGML found itself a following in the publishing industry, as well as a killer application in the form of the Hypertext Markup Language (HTML) . It wasn't until the formulation of a simpler, extended version of SGML, the Extensible Markup Language (XML), that a standard, structured document format was used for general computer interchange of information .

Recently, XML has been proposed as the data format for many tasks, such as Remote Procedure Call (RPC) messages , however XML hasn't yet been seriously considered for general process I/O.