Wiki Testing/ServerInstallationGuide3

From OPeNDAP Documentation

1 Installing Your Data

After installing the server, the data to be provided must be put in some location where it may be served to clients. The server navigates a directory tree rooted in the web server's document root directory. This is the directory (often called htdocs or the DocumentRoot) in which the web server first looks for web pages to serve. If you send your web server this URL:

http://yourmachine/page.html

It will look for the file page.html in the document root directory. Again, the location of this directory depends entirely on the web server you use and its configuration data. A server may also be enabled to follow links from the root directory tree, which means that you can store your data somewhere else, and make symbolic links to it from the root directory tree. By default, this is disabled for most servers, since it can become a security problem, but it is relatively easy to enable it. (The option is called FollowSymLinks for Apache users.) Further, there may be other options provided by the specific server used in a particular installation. In other words, there is really no way to avoid consulting the configuration instructions of the web server you use.

NOTE: You can also use the server's data_root configuration

parameter to tell the server where to look for data. The value of data_root is used as a prefix for the location of data. Using this you can configure a server for data that cannot also accessed using the web. The only drawback to serving data this way is that the directory browsing functions of the OPeNDAP server will not

work.

As noted, the location of the data depends not only on the configuration of the Web server, but also on the API used to access the data requested. For example, the netCDF server simply stores data in files with a pathname relative to the document root directory, htdocs, while the JGOFS server uses its data dictionary to specify the location of its data. Refer to the specific installation notes for each API for more information about the location of the data.

1.1 Special Instructions for the NetCDF, HDF, and DSP handlers

To install data for the NetCDF, HDF, or DSP formats, follow these steps:

NOTE: The data_root parameter can be substituted for the web server's DocumentRoot in the discussion that follows.


  1. Make a data directory somewhere in the document root tree. For Apache, this is the htdocs directory and all its subdirectories. If your web browser is configured to follow symbolic links (Apache: FollowSymLinks is enabled), you can just put links to the data files or to their directories in the document root tree. Do be aware that the reason symbolic links are not enabled by default is that there are potential security holes in its use. Consult your web server's documentation for more information.
  2. Put the data files into the data directory you've just made.
  3. Test the URL with a web browser. See [Quick Start Guide] for information about how to construct a URL if you know the machine name.

That's all.

1.2 Special Instructions for the OPeNDAP/JGOFS Server

The JGOFS data system is a distributed, object-based data management system for multidisciplinary, multi-institutional programs. It provides the capability for all JGOFS scientists to work with the data without regard for the storage format or for the actual location where the data resides.

A full description of the JGOFS data system and U.S JGOFS program can be found at [| http://www1.whoi.edu/ ].

1.2.1 About JGOFS Servers

In the JGOFS data system, scientific datasets are encapsulated in \new{data objects}. Data objects are simply names defined in the JGOFS \new{objects} dictionary which associate specific scientific datasets with a computer program, known as an access \new{method}, which can \subj{The JGOFS API reads "objects", not files.} read them. This mechanism provides a single, uniform access methodology for all scientific datasets served by this data server, regardless of whether they are simple single file or complex multi-file datasets.

Here is an example of a JGOFS object dictionary entry:

test0=def(/usr/local/apache/htdocs/data/t0)

In this example,

test0 
defines the data object name.
def 
specifies the access 'method' or program used to read the scientific dataset.
/usr/local/apache/htdocs/data/t0 
specifies the scientific dataset to read.

The OPeNDAP/JGOFS server requires only two components of a complete JGOFS installation to function, the `objects' dictionary and the access `methods' used to read local data files.

For those sites who wish to use the OPeNDAP/JGOFS server without installing a complete JGOFS data system we provide these required components. For sites which currently have a JGOFS data system installed the OPeNDAP/JGOFS servers can be configured to use the current installation.


Included with the OPeNDAP/JGOFS server is an example object dictionary, named .objects, and a binary version of the JGOFS default (def) method. The def method can read single or multi-file flat ASCII datasets. An example ASCII dataset is also provided to test the server installation.

NOTE:The OPeNDAP/JGOFS server currently requires the object dictionary to be named .objects.

The server uses two mechanisms for determining the location of the data objects dictionary and access method directory:


  1. The CGI dispatch script nph-dods defines two shell variables, JGOFS_OBJECT and JGOFS_METHOD. These are used by the server to determine the location of the objects dictionary and access method directory. Initially, the definition of these variables is set to the OPeNDAP server's directory (i.e., the directory that contains nph-dods).
  2. If the shell variables JGOFS_OBJECT and JGOFS_METHOD are not defined, the OPeNDAP/JGOFS server checks for the existence of a jgofs user in the /etc/passwd file. If a jgofs user exists it will set the JGOFS_METHOD location to ~jgofs/methods/, and the JGOFS_OBJECT location to ~jgofs/objects/.
  3. If the server cannot locate the objects directory using either of the above two mechanisms it will search the current working directory.

If the OPeNDAP/JGOFS server cannot resolve the filesystem paths for the objects dictionary or methods directory using any of these mechanisms it will report an error to the client and exit.

[Tip:] If you have an existing jgofs user but would like to use a different objects dictionary for the OPeNDAP server, then specifying the JGOFS_OBJECT and JGOFS_METHOD variables in the nph-dods script will override your existing JGOFS setup.

To use the .objects dictionary and def access method supplied with the OPeNDAP/JGOFS server, edit the nph-dods script. In the nph-dods script, update the lines specifying the JGOFS_OBJECT and JGOFS_METHOD variables. These variables must contain the directory name containing the .objects dictionary and def access method. Don't forget to remove the comment specifiers at the beginning of these lines.

[Tip:]{Though not required, copying these two files to the cgi-bin

location will keep all the OPeNDAP/JGOFS files in one convenient

location.

To make data accessible via the server, the entries in the objects dictionary must define an object name, and associate that name with an access method and a dataset to serve. Datasets served as JGOFS objects can be located anywhere within your computer's filesystem. However, the user and group used to run the WWW daemon must have permission to read these files.

[Another tip:]To find out which user/group httpd is running as, look in the httpd.conf file for lines like: User nobody Group nobody

To test your server installation using the provided dictionary, and test data, edit the .objects dictionary so that the test0 object points to the files located in the $prefix/share/dap-server/jgofs_data directory. 9 You are free to move these files to any location on your computer so long as the WWW daemon has permission to read these files, and the objects dictionary points to their location.

Using the supplied test datasets, try issuing the following URL:

http://yourmachine/cgi-bin/nph-dods/test0.dds

You will see something like this:

Dataset {
    Sequence {
        String leg;
        String year;
        String month;
        Sequence {
            String station;
            String lat;
            String lon;
            Sequence {
                String press;
                String temp;
                String sal;
                String o2;
                String sigth;
            } Level_2;
        } Level_1;
    } Level_0;
} test0;

You can go further and ask for some data:

http://yourmachine/cgi-bin/nph-dods/test0.asc

You should see a comma-separated list of data values.

1.2.2 Attribute Data for OPeNDAP/JGOFS

The JGOFS data access API does not support a wide variety of attribute data. The OPeNDAP server willlook for a file that contains \new{ancillary data} file named \var{object}.das, in the same directory which contains the JGOFS methods (i.e., the directory referenced by $JGOFS_METHODS).

NOTE:In the past the server looked in the same directory as the data.

For example, suppose that you have installed the OPeNDAP/JGOFS server, as described above, and a partial listing of the data objects dictionary contained the following definitions:

test0=def(/home/httpd/htdocs/data/test0.data)
s87_xbt=def(/home/httpd/htdocs/data/xbt.catalog)

To find attribute data for the JGOFS object `test0', the server will search for the file:

$JGOFS_METHODS/test0.das


The file itself should simply be a text version of the DAS you wish the server to supply to clients. For example:

Attributes {
    leg {
       String Description "The number of the voyage leg.";
    }
    year {
       String Range "Data between 1982 and 1992";
    }
}

1.2.3 Data Types for OPeNDAP/JGOFS Data

The JGOFS data access system returns string data when using the default (def) method. That is, all data is returned to the client as character strings. The OPeNDAP/JGOFS server supports a variety of other data types, allowing the data provider (i.e., you) to specify the variable type to use when returning data to the remote client. To accomplish this, the server will look for an ancillary DAS file (described in( Section3.2.2 )) for the object, and search that file for attribute containers for each variable. If the variable's container exists, it will then look for the DODS_Type, and missing_value attributes in that container.

If DODS_Type exists it will set the output type of the variable. If the missing_value attribute is set, it well replace any nd, or missing data values in the output stream for that variable to the value specified in the missing_value attribute. If the missing_value attribute is not specified, the server will set the missing value field to the largest value possible for specified type. The server assumes that the data provider knows the proper data type to use to contain the full range of possible data values for the object. If the JGOFS variable values cannot be converted to the specified data type, then a missing_value value will be returned.

Suppose that you have installed the OPeNDAP/JGOFS server, as described above, and a partial listing of the data objects dictionary contained the following definitions:

test0=def(/home/httpd/htdocs/data/test0.data)

The ancillary DAS file for this dataset could contain these new attributes:

Attributes {
    leg {
        String DODS_Type "Int32";
        Int32 missing_value -99;
    }
    press {
        String DODS_Type "Float32";
        Float32 missing_value -1.99.;
    }
    temp {
        String DODS_Type "Float64";
    }
}

The server will use the values specified in the DODS_Type attribute for any variable container provided, to define the output variable type. In this example, the variable named leg will be returned as a 32-bit integer (Int32), with missing values set to -99. Similarly, variable press will be returned as a 32-bit floating point number, with missing values set to -1.99. The variable temp will be returned as a 64-bit floating point number, but missing values will be returned as the largest possible 64-bit floating point number (1.797693e+308).

1.2.4 Limiting Access to a OPeNDAP/JGOFS Server

Generally speaking, it is straightforward to configure OPeNDAP servers with limited access. However, the data dictionary structure of the JGOFS data access API provides an additional way to differentiate between public and private datasets. To do this, you would maintain different data object dictionaries which encapsulate different scientific datasets. One dictionary would define data objects for general public access, and other dictionaries could define data objects for specific user communities, where access is restricted.

To accomplish this the site would maintain different versions of the nph-dods dispatch script. For example, using nph-dods for public access, this file would set the JGOFS_OBJECT variable to the public-access dictionary, and another dispatch script, say nph-dods-globec would set the JGOFS_OBJECT variable to a restricted-access dictionary. The WWW daemon is then configured to permit general access to nph-dods and restricted access to nph-dods-globec.

1.3 Special Instructions for the FreeForm Server

The OPeNDAP FreeForm data handler can serve data stored in a wide variety of formats. To read data, it uses a \new{format file} that describes the structure of the data file to be read. Armed with a knowledge of the file structure, the server can read a data file, and send it along to the client.

The FreeForm server has its own documentation: see [OPeNDAP FreeForm ND Server Manual] . You will find a complete description of the server, as well as instructions on how to write a format file for your data.