Hyrax: Server Side Functions

This is an introduction to Server-side Functions.

It is both lecture and hands-on and works best if you have Eclipse, Emacs or another suitable programming editor running.

DAP2: Constraints

Background on DAP2 and Constraints

• The DAP2 interface is via the URL.
• The URL includes the
  • Name of the data set (/data/nc/fnoc1.nc)
  • Name of the desired response (DAS, …)
  • Optional Constraint Expression
• Constraint expression chooses which parts of the dataset (granule) are returned
  • If the dataset has several variables, chooses among those
  • If the variables are arrays, subsets those
  • If the are Structures, chooses the fields
  • If there are Sequences, selects values
  • Also runs functions if they are available

Constraint Expression Syntax

• URL: http://host/dataset.<ext>?<ce>
• If <ext> is ‘dds’ or ‘dods’ the <ce> is used to contrain the response
• <ce> is a <projection>[&<selection>]*
• <projection>

  - is a comma separated list of zero or more:

  • Variables to be returned.
  • Array hyperslabs
  • Structure fields
  • Function calls

CE Projection Examples

Example 1 - Choosing Variables and subsetting arrays

• Using the http://localhost:8080/data/nc/fnoc1.nc dataset:

- ?u,v Chooses the variables u and v

- ?u[0][0:5][0:5] Chooses u and subsamples the array returning a 6 by 6 hyperslab

• Try these in a browser - you do not need to use the virtual machine’s server; you can substitute test.opendap.org for localhost

- e.g.,http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?u[0][0:5][0:5]

- I used ‘.ascii’ instead of ‘.dods’ so the result is easy to see in a browser

Example 2 - Retrieving Map variable data from a Grid

Selecting a top level Grid variable

http://test.opendap.org/opendap/data/nc/coads_climatology.nc.dds?SST

Dataset {
   Grid {
     Array:
       Float32 SST[TIME = 12][COADSY = 90][COADSX = 180];
     Maps:
       Float64 TIME[TIME = 12];
       Float64 COADSY[COADSY = 90];
       Float64 COADSX[COADSX = 180];
   } SST;
} coads_climatology.nc;

Choosing a Map variable

http://test.opendap.org/opendap/data/nc/coads_climatology.nc.dds?SST.TIME

Dataset {
   Structure {
       Float64 TIME[TIME = 12];
   } SST;
} coads_climatology.nc;

CEs for data

Replace the ‘.dds’ in the previous URLs with .ascii or .dods (use ascii with a web browser or dods with an application that understands binary data)

Look at the results

Try various combinations

• Writing Server Functions: short version
• Programming with libdap
• A Real Server Function
• Write HelloWorld()
• Advanced Topics

CE Selection Examples

Selections are used to choose values from relational types

Try it

http://localhost:8080/opendap/data/ff/avhrr.dat.asc?day

http://localhost:8080/opendap/data/ff/avhrr.dat.asc?day&day=19

http://localhost:8080/opendap/data/ff/avhrr.dat.asc?&day=19

The first constraint is a projection that chooses just the variable ‘day’ while the second constraint includes a selection clause which chooses only those entries where the value of day is 19. The third constraint returns all the fields but only those rows where day is 19

CE Functions

• Constrain Expressions can also invoke functions.
• The functions can compute and return values.
• They can also return a Boolean value for use in the selection part of the constraint expression.
• Functions are packaged in BES Modules and loaded by the BES at start up.
• The BES library (which is used by all of our handlers) includes a base set of functions

Function Syntax

The <ce> may contain one or more function calls

- <function call>,<function call>

Functions can be composed.

- <function> “(“ <zero or more args> “)”

Function Documentation

• Use ?version() to get a listing of installed functions.
• Use ?<function>() to get help information specific to a particular function
• Go to docs.opendap.org*

Simple Example: The version() function

Example of a simple function call:

http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?version()

- Run this example in a browser or command shell using getdap

- This is a very simple function and it takes no arguments

- It’s intent is both as an example and to provide information about the functions leaded into the server.

getdap:

jimg : ~/src/hyrax_1.8_release $ getdap "http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?version()"
Dataset: function_result_fnoc1.nc
version, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
…

More Useful Functions

grid()

sample a Grid variable based on the values of its map vectors

geogrid()

just like grid(), but assume that the Grid holds geospatial data

linear_scale()

Scale data using y=mx+b

geogrid() function example

Use the geogrid() function to select values from the COADS Climatology data set: ?geogrid(SST,-5,-80,-40,-50)

http://test.opendap.org/opendap/data/nc/coads_climatology.nc.asc?geogrid(SST,-5,-80,-40,-50)

Comments

- Note that the lat/lon values from the call match those of the response

- The arguments are a little odd but help is available by calling the function with no arguments

- This function’s logic is pretty crude, but the same interface can be used in front of more sophisticated processing (i.e., GIS)

- Calling the function with no arguments returns version information and the URL of the function’s documentation.

- Note also that this made no choice about the time information in the SST variable…

Selecting Time

?geogrid(SST,40,-80,5,-50,\”2000<TIME\”)

Try it - be careful with the quotes

http://test.opendap.org/opendap/data/nc/coads_climatology.nc.asc?geogrid(SST,-5,-80,-40,-50,"2000<TIME")

The general rule for geogrid() is that the first five arguments are required (they specify the Grid variable and the TLBR lat/lon box) and then subsequent arguments are relational expressions which include constants and variables which are the names of the Grid’s map vectors. The result is the intersection of the zero or more relational expressions

Types of Server Side Functions

BaseType Functions

void(*btp_func)(int argc, BaseType *argv[], DDS &dds, BaseType **btpp)

A BaseType function takes four arguments: an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. ARGC is the length of ARGV.

Boolean Functions

void(*bool_func)(int argc, BaseType *argv[], DDS &dds, bool *result)

A boolean function takes four arguments, an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. ARGC is the length of ARGV.

Projection Functions

void(*proj_func)(int argc, BaseType *argv[], DDS &dds, ConstraintEvaluator &ce)

A projection function is a function that appears in the projection part of the CE and is executed for its side-effect. Usually adds a new variable to the DDS. These are run _during the parse_ so their side-effects can be used by subsequent parts of the CE.

Writing Server Side Functions

What we'll do

• Write a C++ function which uses one of the three server function type signatures
• Write a very simple subclass of the libdap::Function class and install the your function into the class using the setFunction() method.
• Write a very simple subclass of the BESAbstractModule and in the initialize() method, add an instance of your Function class to the libdap::ServerFunctionsList
• Build and install the module.

HelloWorld function

The function

Here is the example_ssf::helloWorld() function from the example_ssFunction project. The example_ssf::helloWorld() function is defined in the file ExampleServerSideFunctions.cc.

void helloWorld(  int argc,   libdap::BaseType *argv[],  libdap::DDS &dds, libdap::BaseType **btpp)
{
    Str *dapResult = new Str("helloWorldFunction_result");
    dapResult->set_value("Howdy Stranger...");
    *btpp = dapResult;
}

We can see from the method's type signature that it is a BaseType function. All it does is create a DAP String object, set it's value and return via the return value parameter btpp.

A child class of libdap::Function

Now we need a libdap::Function to provide an API by which the server can learn things about our new function. Here is the example_ssf::HelloWorld class as defined in ExampleServerSideFunctions.h.

class HelloWorldFunction: public libdap::ServerFunction {
public:
    HelloWorldFunction()
    {
        setName("helloWorld");
        setDescriptionString("Returns a DAP String object whose value is the string 'Hello World'.");
        setUsageString("helloWorld()");
        setRole("http://services.opendap.org/dap4/server-side-function/hellowWorld");
        setDocUrl("http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions");
        setFunction(example_ssf::helloWorld);
        setVersion("1.0");
    }
    virtual ~HelloWorldFunction()
    {
    }

};

The parent class of our new example_ssf::HelloWorld class is libdap::Function. Since libdap::Function is not an abstract we don't need to override any methods to make the child class work, all we do is give the child class state that's specific to our new function.

Now we have a function, and a class through whose API we can learn about our function. Now we just need to hook it up to the BES and Hyrax...

A child class of BESAbstractModule

In order to get our new function into Hyrax we need to have the BES load it at startup. This is done by writing a very simple BES module. For this example our module classed is called ExampleServerSideFunctions which is declared in the header file ExampleServerSideFunctions.h:

class ExampleServerSideFunctions: public BESAbstractModule {
public:
    ExampleServerSideFunctions() {}
    virtual ~ExampleServerSideFunctions() {}
    virtual void initialize(const string &modname);
    virtual void terminate(const string &modname);
    virtual void dump(ostream &strm) const;
};

and implemented in ExampleServerSideFunctions.cc

void ExampleServerSideFunctions::initialize(const string &modname) {
    BESDEBUG( "ExampleServerSideFunctions", "Initializing ExampleServerSideFunctions:" << endl );

    libdap::ServerFunctionsList::TheList()->add_function(new example_ssf::HelloWorldFunction());

    BESDEBUG( "ExampleServerSideFunctions", "Done initializing ExampleServerSideFunctions" << endl );
}

void ExampleServerSideFunctions::terminate(const string &modname) {
    BESDEBUG( "ExampleServerSideFunctions", "Removing ExampleServerSideFunctions module (this does nothing)." << endl );
}

extern "C" {
    BESAbstractModule *maker() {
        return new ExampleServerSideFunctions;
    }
}

Note carefully the method ExampleServerSideFunctions.initialize() which contains the call to the libdap:ServerFunctionsList.addFunction() method. It is this call that causes a HelloWorld class instance and thus the helloWorld() function to be registered in BES.