Hyrax - BES Configuration

Building the BES from and it's data handlers source or installing from an RPM will provide the default installtion with data and a valid configuration. This is suitable for testing. The following details how you go about customizing it for your data.

Administration & Logging

The BES.ServerAdministrator parameter is the address used in various mail messages returned to clients. Set this so that whoever gets the email will be able to fix problems ad/or respond to user questions. Also set the log file and log level. If the BES.LogName is set to a relative path it will be treated as relative to the directory where the BES is started. That is, if the BES is installed in /usr/local/bin but you start it in your home directory using the parameter value below, the log file will be bes.log in your home directory.

BES.ServerAdministrator=webmaster@some.place.edu
BES.LogName=./bes.log
BES.LogVerbose=no

Because the BES is a server in its own right, you will need to tell it which network port and interface to use. Assuming you are running the BES and OLFS (i.e., all of Hyrax) on one machine, here's what you should do:

Setting the networking parameters

BES.ServerPort=10002
# BES.ServerUnixSocket=/tmp/opendap.socket

The BES.ServerPort tells the BES which TCP/IP port to use when listening for commands. Unless you need to use a different port, use the default. Ports with numbers less than 1024 are special, otherwise you can use any number under 65536. That said, stick with the default unless you know you need to change it.

In the default bes.conf file we have commented the ServerUnixSocket parameter since doing so now disables I/O over that device. If you need UNIX socket I/O, uncomment this line, otherwise leave it commented since the fewer open network I/O ports, the easier it is to make sure the server is secure.

If both ServerPort and ServerUnixSocket are defined, the BES listens on both the TCP port and the Unix Socket. Local clients, on the same machine as the BES, can use the unix socket for a faster connection. Otherwise, clients on other machines will connect to the BES using the BES.ServerPort value. Note that the OLFS always uses only the TCP socket, even if the UNIX socket is present.

Debugging Tip

Use the BES.ProcessManagerMethod parameter to control whether the BES acts like a normal Unix server or not. The default value of multiple causes the BES to accept many connections at once, like a typical server. The value single causes it to accept a single connection, process the commands sent to it and exit, greatly simplifying trouble shooting.

BES.ProcessManagerMethod=multiple

Controlling how compressed files are treated

Until we complete an assessment of this feature, we suggest you disable serving compressed files by setting the value of BES.Compressed.Script to null as shown below. When set to null, the values of the other parameters in this section of the configuration file have no effect.

BES.Compressed.Script=

Loading Handlers

Virtually all of the BESs functions are contained in modules that are loaded when the server starts up. Each module is shared-object library. If you're curious about how these are built, take a look at any of the handler (e.g., the dap-server software has several handlers and those are used by almost every BES).

There are two steps to the configuration:

1. List the module in the order you want it loaded using the BES.modules parameter
2. Use the BES.module.module name parameter to tie the name using in BES.modules with a specific shared-object module file. Use absolute paths.

Here's an example:

BES.modules=dap,cmd,ascii,usage,www,nc,h4,ff
BES.module.dap=/usr/local/lib/bes/libdap_module.so
BES.module.cmd=/usr/local/lib/bes/libdap_cmd_module.so
BES.module.ascii=/usr/local/lib/bes/libascii_module.so
BES.module.usage=/usr/local/lib/bes/libusage_module.so
BES.module.www=/usr/local/lib/bes/libwww_module.so
BES.module.nc=/usr/local/lib/bes/libnc_module.so
BES.module.h4=/usr/local/lib/bes/libhdf4_module.so
BES.module.ff=/usr/local/lib/bes/libff_module.so

In this example configuration the BES loads the following modules:

dap

The DAP implementation

cmd

The BES command set

ascii

A handler to build the ASCII data response using the data response from any one of the data handlers

usage

A handler which retrieves metadata from a data handler and formats it as a text/plain response

www

A handler which retrieves metadata from a data handler and builds the HTML form interface for that data source

nc

The data handler for NetCDF3 files

h4

The data handler for HDF4 files

ff

The data handler for files which can be read using FreeForm

Parameters for Specific Handlers

Currently there's only one handler that has a configuration parameter all its own.

Use HDF4.CacheDir to set the directory where the HDF4 handler will cache DAP metadata objects. The default value of /tmp is OK, but you might want to change it to a directory specific to just this purpose, such as <prefix>/var/bes/hdf4.

HDF4.CacheDir=/tmp

Pointing to data

There are two parameters that can be used to tell the BES where your data are stored. Which one you use depends on whether you are setting up the BES to work as part of Hyrax (and thus with THREDDS catalogs) or as a standalone server. In either case set the value of the ..RootDirectory parameter to point to the root directory of your data files (only one may be specified). Use BES.Catalog.catalog.RootDirectory if the BES is being used as part of Hyrax and BES.Data.RootDirectory if not. So, if you are setting up Hyrax, set the value of BES.Catalog.catalog.RootDirectory but be sure to set BES.Data.RootDirectory to some value or the BES will not start. (This will be fixed soon jimg 23:12, 22 February 2007 (PST))

BES.Catalog.catalog.RootDirectory=/full/path/data/root/directory
BES.Data.RootDirectory=/full/path/data/root/directory

Next configure the mapping between data source names and data handlers. When the BES is asked to perform some commands on a particular data source, it uses regular expressions to figure out which data handler should be used to carry out the commands. The value of the BES.Catalog.catalog.TypeMatch parameter is holds the set of regular expressions. The value of this parameter is a list of handlers and expressions. Note that these regular expressions are like those used by grep on Unix and it's somewhat cryptic, but once you see the pattern, it's not that bad. Below the TypeMatch parameter is being told that any data source with a name that ends in .nc should be handled by the nc (netcdf) handler (see BES.module.nc above), any file with a .hdf, .HDF or .eos suffix should be processed using the HDF4 handler (note that case matters) and that data sources ending in .dat should use the FreeForm handler.

BES.Catalog.catalog.TypeMatch=nc:.*\.nc;h4:.*\.(hdf|HDF|eos);ff:.*\.dat;

If you fail to configure this correctly, the BES will return error messages stating that the type information has to be provided. However, it won't tell you this when it starts, only when the OLFS (or some other software) actually makes a data request. This is because it's possible to use BES commands in place of these regular expressions, although the Hyrax won't.

Finally, you can configure the types of information that the BES send back when a client requests catalog information. The Include and Exclude parameters provide this mechanism, also using a list of regular expressions (with each element of the list separated by a semicolon). In the example below, files that begin with a dot are excluded.

The Include expressions are applied to the node first, followed by the Exclude expressions. For collections of nodes, only the Exclude expressions are applied.

BES.Catalog.catalog.Include=;
BES.Catalog.catalog.Exclude=^\..*;