Installing and Using Pipeline

• installing Pipeline,
• setting Pipeline's global parameters, and
• using Pipeline in a form.

Note that there is an online setup utility to configure pipeline.pl for site-specific locations and global parameters.

Installation

Okay, so how do I install it?

Take the following steps to make Pipeline usable:

• Change the top line of the pipeline.pl file to contain the correct path to your perl 5 executable if necessary. It is currently set at "/usr/bin/perl". You can find your perl executables in some UNIX shells by typing 'where perl'.
  
• Place the pipeline.pl file in your cgi-bin directory or otherwise make the script executable as a CGI script by your web server. You may need your server administrator's help with this.
  
• Install CGI.pm unless you already have it installed. Questions regarding this installation should be directed toward CGI.pm's maintainer.

See here for how to use Pipeline in forms.

Global Parameters

The global parameters to Pipeline are settings for Pipeline that are the same across all forms that use Pipeline. Each of these are (necessarily) specified in top of the pipeline.pl file itself or in a file that pipeline.pl knows about. You should adjust these parameters for your local system before trying to use Pipeline. At present, there are 4 sets of such parameters.

Default errorsto

Warning and error messages from Pipeline and modules may be sent to e-mail addresses and files specified in the form. If there is such a message and either no destination is given or all the destinations given are not valid according the fixed errorsto feature, the default errorsto value is used as the play to send the error to. An attacker may generate such a condition. The default errorsto can be set in the $DEFAULT_ERRORSTO variable in pipeline.pl.

I encourage everyone to provide such a value as it can catch errors and possibly catch attackers, especially when used in conjunction with the fixed errorsto feature.

Fixed parameters

If enabled, the "fixed parameters" restrict the allowable values for the Pipeline control fields that come from forms. One thing this does is to serve as a double-check on the way Pipeline is being used by your forms. A more important effect is that it restricts malicious hackers to using the modules, paths, and error destination values that you specify. Therefore, it is advisable to take advantage of this feature, and is enabled by default.

There are three sets of fixed parameters:

• one to control the modules that are used,
• one to control the paths to modules, and
• one to restrict where form configuration errors are reported to.

Restricting the modules that are executed

If the $FIXED_MODULES variable is set, then the modules that are allowed to be executed are specified. For a given module, either the full path to the module or just the name of the module is allowed. In the case of just the name being given, all files with that name that appear in the path are accepted. So, it is generally preferable in terms of security to give a full path.

If a file is given in the $MODULE_LIST_FILE variable, the valid modules are specified in a file in this format. Otherwise they need to be given in the pipeline.pl file in the variable @VALID_MODULES.

In other words:

If a form attempts to use a module that is not specified, then an error message is sent and the module is skipped.

A list of HFPM modules is available.

Restricting the path values

In a form, the _path field acts like the UNIX path shell variable, given directories to look for modules in. If the $FIXED_PATH variable is set to 1, then what can be on this path is restricted. If any other values in _path are encountered from the form, an error message is sent and form processing halts.

The valid paths are read from a specially formatted file if $PATH_LIST_FILE is a file, otherwise they are found in the @VALID_PATH variable in pipeline.pl.

Restricting the destination of error reports

Error reports and warnings from Pipeline and its modules can be sent to a number of e-mail addresses and files as specified in the form. The valid destinations for this can indicated and any other specified destinations rejected with an error message. This is done by setting (or leaving) the $FIXED_ERRORSTO variable in pipeline.pl to1. If the $ERRORSTO_LIST_FILE variable is set to a file, then that file (in a particular format) is where the list of valid destinations is found. Otherwise it is taken from the pipeline.pl in the @VALID_ERRORSTO variable.

In other words:

If an attacker is trying to compromise your site using Pipeline, error messages would be rather useful to him or her. This is one reason to limit where errors are reported. Make the attacker fly blind. If you also use the default errorsto feature, you can be sure that any error message will be sent to some valid destination regardless of the contents of the form.

List file format

In list files, each item (module, file path, or e-mail address) is listed on a separate line. Empty lines and lines beginning with a "#" are ignored.

Inline list format

When lists are specified in the pipeline.pl file, they are put between parentheses and separated by spaces. For example:

@VALID_MODULES= qw(set.pl copy.pl setdate.pl setifempty.pl);

specifies the items "set.pl", "copy.pl", "setdate.pl", and "setifempty.pl" for the variable @VALID_MODULES. Those who know Perl can probably think of other ways to do this.

Using Pipeline in a form

How do I use Pipeline in my forms?

First you need to install pipeline.pl. Then you need to create a form with the action being the URL of pipeline.pl and containing hidden _pipeline and _path fields. Hidden fields are fields of the form <INPUT TYPE=hidden NAME="name" VALUE="value">.

_path field

The _path field is analogous to the path variable in UNIX shells; it stores a list of directories to look in for modules if the modules to execute aren't given with full paths. For example if the _path field has the value "/home/smith/cgifpi/modules /usr/pkg/hfpm/modules", and the module "set.pl" is referenced, then module is looked for as "/home/smith/cgifpi/modules/set.pl" and if that isn't found it is then looked for as "/usr/pkg/hfpm/modules".

_pipeline field

The _pipeline field tells Pipeline what modules to run with what arguments and in what order. The programs to be run are separated with a pipe/vertical bar character ("|") with the first "word" in the section being the module and the remaining words being the arguments to the module. "Words" are separated by whitespace.

This is best illustrated by an example. If _pipeline contains "module1.pl arg1 arg2 | module2 | module3.pl arg1", "module1.pl" is executed first with "arg1" and "arg2' as arguments, then "module2" is run with no arguments and finally "module3.pl" is run with "arg1".

There are two quoting options available that "hide" characters for the purposes of dividing them in to words and modules. The first is the backslash character ("\") which quotes the next character. This can be used, for example, to hide a space or a pipe character. The second option is to enclose the text to be quoted in single-quotes ("'"). This can be used to force a range of text to be treated as a single "word". (In case you want to know, backquoting is done before single-quoting, enabling single-quotes to be quoted.)

For example, "set.pl $text 'some text that includes a | and a'\ \\" is interpreted as the module "set.pl" with the arguments "$text" and "some text that includes a | and a \".

_errorsto field

Sometimes the configuration of Pipeline as used in a form is not correct. Traditionally, it is hard to get at such error reports. However, Pipeline had features to allow the messages to get sent to e-mail addresses and/or appended to files along with the context of the error.

The locations to send the message are specified in the hidden _errorsto field in the form. Any number of e-mail addresses or files many be given and are separated by spaces. If an item starts with a letter or number, it is assumed to be an e-mail address otherwise it is taken as a file name.

If there is no _errorsto field but the default errorsto feature is employed, then the message is sent to that location. If there is no _errorsto field and the default errorsto has not been set, then you may be able to see the message on the Web browser.