Hyrax: Server Side Functions: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
 
(40 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This is an introduction to Server-side Functions.
== Overview ==


It is both lecture and hands-on and works best if you have Eclipse, Emacs or another suitable programming editor running.  
Before you embark upon writing a server side function for DAP2 you should read through this [[DAP2: Constraint Expressions | overview of DAP2 constraint expressions and their use of functions]].  


== Background on DAP2 and Constraints ==
<font color="red">Here's another source for information on Server Side FUnctions and Constraint Expressions: http://docs.opendap.org/index.php/UserGuideOPeNDAPMessages#Constraint_Expressions
<br>
You might want to take some of your new text and fold it into this and then reference the User Guide only - I'm just angling here for any way to improve the User Guide ;-) jhrg 2/11/13</font>


* The DAP2 interface is via the URL.
== Types of Server Side Functions ==
* The URL includes the
<blockquote>Here we document all three types of Server Side Functions that DAP2 supports. In ''practice'' only the first kind of function is very widely used. These functions take a set of variables from the current dataset and return a new value wrapped in a DAP2 variable. The other kinds of functions are used to control constraint expression evaluation and to add new variables to the dataset's DDS, respectively. We have used these to build queryable interfaces for inventories.</blockquote>
** Name of the data set (/data/nc/fnoc1.nc)
; BaseType Functions
** Name of the desired response (DAS, …)
: <font size="2"><tt>void(*btp_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, libdap::BaseType **btpp)</tt></font>
** Optional Constraint Expression
: A BaseType function takes four arguments: an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. ARGC is the length of ARGV.
* Constraint expression chooses which parts of the dataset (granule) are returned
** If the dataset has several variables, chooses among those
** If the variables are arrays, subsets those
** If the are Structures, chooses the fields
** If there are Sequences, selects values
** Also runs functions if they are available


=== Constraint Expression Syntax ===
; Boolean Functions
: <font size="2"><tt>void(*bool_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, bool *result)</tt></font>
: A boolean function takes four arguments, an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. Unlike the previous type function, the result is a boolean type, and will be used by the constraint expression evaluator to determine the truth value for the expression. ARGC is the length of ARGV.


* URL: http://host/dataset.<ext>?<ce>
; Projection Functions
* If <ext> is ‘dds’ or ‘dods’ the <ce> is used to contrain the response
: <font size="2"><tt>void(*proj_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, libdap::ConstraintEvaluator &ce)</tt></font>
* <ce> is a <projection>[&<selection>]*
: A projection function is a function that appears in the projection part of the CE and is executed for its side-effect. Usually adds a new variable to the DDS. These are run ''during the parse'' so their side-effects can be used by subsequent parts of the CE.
* <projection>  
*: - is a comma separated list of zero or more:
** Variables to be returned.
** Array hyperslabs
** Structure fields
** Function calls


=== CE Projection Examples ===
== Writing Server Side Functions ==


==== Example 1 - Choosing Variables and subsetting arrays ====
* Using the http://localhost:8080/data/nc/fnoc1.nc dataset:
: - ?u,v Chooses the variables u and v
: - ?u[0][0:5][0:5] Chooses u and subsamples the array returning a 6 by 6 hyperslab
* Try these in a browser - you do not need to use the virtual machine’s server; you can substitute test.opendap.org for localhost
: - e.g.,http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?u[0][0:5][0:5]
: - I used ‘.ascii’ instead of ‘.dods’ so the result is easy to see in a browser




==== Example 2 - Retrieving Map variable data from a Grid ====
=== Example: HelloWorld ===


; Selecting a top level Grid variable:
; What we'll do
: http://test.opendap.org/opendap/data/nc/coads_climatology.nc.dds?SST
* Write a C++ function which uses one of the three server function type signatures, btp_func (BaseTypePointer_Function)
* Write a very simple subclass of the <font size="2"><code>libdap::Function</code></font> class and install the your function into the class using the <font size="2"><code>libdap::Function.setFunction()</code></font> method.
* Write a very simple subclass of the <font size="2"><code>BESAbstractModule</code></font> and in the <font size="2"><code>initialize()</code></font> method, add an instance of your Function class to the <font size="2"><code>libdap::ServerFunctionsList</code></font>
* Build and install the module.


Dataset {
    Grid {
      Array:
        Float32 SST[TIME = 12][COADSY = 90][COADSX = 180];
      Maps:
        Float64 TIME[TIME = 12];
        Float64 COADSY[COADSY = 90];
        Float64 COADSX[COADSX = 180];
    } SST;
} coads_climatology.nc;


Get the example_ssFunction project and follow along:
: <font size="2"><code>svn co https://scm.opendap.org/svn/trunk/example_ssFunction</code></font>


; Choosing a Map variable:
==== The <code>helloWorld()</code> function ====
: http://test.opendap.org/opendap/data/nc/coads_climatology.nc.dds?SST.TIME
Dataset {
    Structure {
        Float64 TIME[TIME = 12];
    } SST;
} coads_climatology.nc;


; CEs for data
Here is the <font size="2"><code>example_ssf::helloWorld()</code></font> function from [https://scm.opendap.org/svn/trunk/example_ssFunction/ the example_ssFunction project.] The <font size="2"><code>example_ssf::helloWorld()</code></font> function is defined in the file [https://scm.opendap.org/trac/browser/trunk/example_ssFunction/ExampleServerSideFunctions.cc ExampleServerSideFunctions.cc].
: Replace the .dds’ in the previous URLs with .ascii or .dods (use ascii with a web browser or dods with an application that understands binary data)
: Look at the results
<font size="2"><source lang="cpp">
: Try various combinations
void helloWorld( int argc,  libdap::BaseType *argv[],  libdap::DDS &dds, libdap::BaseType **btpp)
{
    Str *dapResult = new Str("helloWorldFunction_result");
    dapResult->set_value("Howdy Stranger...");
    *btpp = dapResult;
}
</source></font>
We can see from the method's type signature that it is a BaseType function. The <font size="2"><code>helloWorld()</code></font> function creates  a DAP String object, set it's value and return it via the return value parameter '''btpp'''.


* ''Writing Server Functions: short version''
==== An instance of libdap::ServerFunction ====
* ''Programming with libdap''
* ''A Real Server Function''
* ''Write HelloWorld()''
* ''Advanced Topics''


=== CE Selection Examples ===
Now we need a <font size="2"><code>libdap::ServerFunction</code></font> to provide an API by which the server can learn things about our new function. Since <font size="2"><code>libdap::ServerFunction</code></font> is not an abstract we don't need to override any methods to make the child class work, all we do is give the new instance state that's specific to our new function.


Selections are used to choose values from relational types
<font size="2"><source lang="cpp">
;Try it:
: http://localhost:8080/opendap/data/ff/avhrr.dat.asc?day
: http://localhost:8080/opendap/data/ff/avhrr.dat.asc?day&day=19
: http://localhost:8080/opendap/data/ff/avhrr.dat.asc?&day=19
The first constraint is a projection that chooses just the variable ‘day’ while the second constraint includes a selection clause which chooses only those entries where the value of day is 19. The third constraint returns all the fields but only those rows where day is 19


=== CE Functions ===
libdap:ServerFunction *helloWorld =
* Constrain Expressions can also invoke functions.
    new libdap::ServerFunction(
* The functions can compute and return values.
        "helloWorld", // The name of the function as it will appear in a constraint expression
* They can also return a Boolean value for use in the selection part of the constraint expression.
        "1.0",        // The version of the function
* Functions are packaged in BES Modules and loaded by the BES at start up.
        "Returns a DAP String whose value is 'Hello World'", // A brief description of the function
* The BES library (which is used by all of our handlers) includes a base set of functions
        "helloWorld()", // A usage/syntax statement
        "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URL that points two a web page describing the function
        "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URI that defines the role of the function
        example_ssf::helloWorld // A pointer to the helloWorld() function
    );
</source></font>


==== Function Syntax ====
Now we have
; The <ce> may contain one or more function calls
* a BaseType function, and
: - <function call>,<function call>
* a libdap::ServerFunction class instance through whose API we can learn about the<font size="2"><code>example_ssf::helloWorld</code></font> function.


; Functions can be composed.
Now we just need to hook it up to the BES and Hyrax...
: - <function> “(“ <zero or more args> “)”


==== A child class of BESAbstractModule  ====


==== Simple Example: The version() function ====
In order to get our new function into Hyrax we need to have the BES load it at startup. This is done by writing a very simple BES module. For this example our module class  is called <font size="2"><code>ExampleServerSideFunctions</code></font> which is declared in the header file [https://scm.opendap.org/trac/browser/trunk/example_ssFunction/ExampleServerSideFunctions.h ExampleServerSideFunctions.h]:
Example of a simple function call:
: http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?version()
: - Run this example in a browser or command shell using getdap
: - This is a very simple function and it takes no arguments
: - It’s intent is both as an example and to provide information about the functions leaded into the server.


getdap:
<font size="2"><source lang="cpp">
jimg : ~/src/hyrax_1.8_release $ getdap "http://test.opendap.org/opendap/data/nc/fnoc1.nc.ascii?version()"
class ExampleServerSideFunctions: public BESAbstractModule {
Dataset: function_result_fnoc1.nc
public:
version, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    ExampleServerSideFunctions() {}
    virtual ~ExampleServerSideFunctions() {}
    virtual void initialize(const string &modname);
    virtual void terminate(const string &modname);
    virtual void dump(ostream &strm) const;
};
</source></font>


==== More Useful Functions ====
;grid()
: sample a Grid variable based on the values of its map vectors
:geogrid()
: just like grid(), but assume that the Grid holds geospatial data
;linear_scale()
: Scale data using y=mx+b


and implemented in [https://scm.opendap.org/trac/browser/trunk/example_ssFunction/ExampleServerSideFunctions.cc ExampleServerSideFunctions.cc]:


==== geogrid() function example ====
<font size="2"><source lang="cpp">
Use the geogrid() function to select values from the COADS Climatology data set:
void ExampleServerSideFunctions::initialize(const string &modname) {
?geogrid(SST,-5,-80,-40,-50)
    BESDEBUG( "ExampleServerSideFunctions", "Initializing ExampleServerSideFunctions:" << endl );


    // Here we wrap the helloWorld() function in an instance of the libdap::ServerFunction class
    libdap:ServerFunction *helloWorld =
        new libdap::ServerFunction(
            "helloWorld", // The name of the function as it will appear in a constraint expression
            "1.0",        // The version of the function
            "Returns a DAP String whose value is 'Hello World'", // A brief description of the function
            "helloWorld()", // A usage/syntax statement
            "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URL that points two a web page describing the function
            "http://services.opendap.org/dap4/server-side-function/helloWorld", // A URI that defines the role of the function
            example_ssf::helloWorld // A pointer to the helloWorld() function
        );


    // Here we add our new instance of libdap::ServerFunction to the libdap::ServerFunctionsList.
    libdap::ServerFunctionsList::TheList()->add_function(helloWorld);


==== Function Documentation ====
    BESDEBUG( "ExampleServerSideFunctions", "Done initializing ExampleServerSideFunctions" << endl );
}


* Use ?version() to get a listing of installed functions.
void ExampleServerSideFunctions::terminate(const string &modname) {
* Use ?<function>() to get help information specific to a particular function
    BESDEBUG( "ExampleServerSideFunctions", "Removing ExampleServerSideFunctions module (this does nothing)." << endl );
* Go to docs.opendap.org*
}
 
extern "C" {
    BESAbstractModule *maker() {
        return new ExampleServerSideFunctions;
    }
}
</source></font>
Note carefully the method <font size="2"><code>ExampleServerSideFunctions.initialize()</code> which contains the call to the <font size="2"><code>libdap:ServerFunctionsList.addFunction()</code></font> method. It is this call that causes a HelloWorld class instance and thus the helloWorld() function to be registered in BES.
 
And that's pretty much all there is to the C/C++ programming. What remains is to build a little autotools project for your function code and set it up to install into the BES when ''make install'' is run.






<!--
<!--
== Types of Server Side Functions ==
== Writing the function ==


== Wrapping the function in a class ==
== Wrapping the function in a class ==

Latest revision as of 00:10, 13 February 2013

Overview

Before you embark upon writing a server side function for DAP2 you should read through this overview of DAP2 constraint expressions and their use of functions.

Here's another source for information on Server Side FUnctions and Constraint Expressions: http://docs.opendap.org/index.php/UserGuideOPeNDAPMessages#Constraint_Expressions
You might want to take some of your new text and fold it into this and then reference the User Guide only - I'm just angling here for any way to improve the User Guide ;-) jhrg 2/11/13

Types of Server Side Functions

Here we document all three types of Server Side Functions that DAP2 supports. In practice only the first kind of function is very widely used. These functions take a set of variables from the current dataset and return a new value wrapped in a DAP2 variable. The other kinds of functions are used to control constraint expression evaluation and to add new variables to the dataset's DDS, respectively. We have used these to build queryable interfaces for inventories.

BaseType Functions
void(*btp_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, libdap::BaseType **btpp)
A BaseType function takes four arguments: an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. ARGC is the length of ARGV.
Boolean Functions
void(*bool_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, bool *result)
A boolean function takes four arguments, an integer (argc), a vector of BaseType *s (argv), the DDS for the dataset for which these function is being evaluated (analogous to the ENVP in UNIX) and a pointer for the function's return value. Unlike the previous type function, the result is a boolean type, and will be used by the constraint expression evaluator to determine the truth value for the expression. ARGC is the length of ARGV.
Projection Functions
void(*proj_func)(int argc, libdap::BaseType *argv[], libdap::DDS &dds, libdap::ConstraintEvaluator &ce)
A projection function is a function that appears in the projection part of the CE and is executed for its side-effect. Usually adds a new variable to the DDS. These are run during the parse so their side-effects can be used by subsequent parts of the CE.

Writing Server Side Functions

Example: HelloWorld

What we'll do
  • Write a C++ function which uses one of the three server function type signatures, btp_func (BaseTypePointer_Function)
  • Write a very simple subclass of the libdap::Function class and install the your function into the class using the libdap::Function.setFunction() method.
  • Write a very simple subclass of the BESAbstractModule and in the initialize() method, add an instance of your Function class to the libdap::ServerFunctionsList
  • Build and install the module.


Get the example_ssFunction project and follow along:

svn co https://scm.opendap.org/svn/trunk/example_ssFunction

The helloWorld() function

Here is the example_ssf::helloWorld() function from the example_ssFunction project. The example_ssf::helloWorld() function is defined in the file ExampleServerSideFunctions.cc.

void helloWorld(  int argc,   libdap::BaseType *argv[],  libdap::DDS &dds, libdap::BaseType **btpp)
{
    Str *dapResult = new Str("helloWorldFunction_result");
    dapResult->set_value("Howdy Stranger...");
    *btpp = dapResult;
}

We can see from the method's type signature that it is a BaseType function. The helloWorld() function creates a DAP String object, set it's value and return it via the return value parameter btpp.

An instance of libdap::ServerFunction

Now we need a libdap::ServerFunction to provide an API by which the server can learn things about our new function. Since libdap::ServerFunction is not an abstract we don't need to override any methods to make the child class work, all we do is give the new instance state that's specific to our new function.

libdap:ServerFunction *helloWorld =
    new libdap::ServerFunction(
        "helloWorld", // The name of the function as it will appear in a constraint expression
        "1.0",        // The version of the function
        "Returns a DAP String whose value is 'Hello World'", // A brief description of the function
        "helloWorld()", // A usage/syntax statement
        "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URL that points two a web page describing the function
        "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URI that defines the role of the function
        example_ssf::helloWorld // A pointer to the helloWorld() function
    );


Now we have

  • a BaseType function, and
  • a libdap::ServerFunction class instance through whose API we can learn about theexample_ssf::helloWorld function.

Now we just need to hook it up to the BES and Hyrax...

A child class of BESAbstractModule

In order to get our new function into Hyrax we need to have the BES load it at startup. This is done by writing a very simple BES module. For this example our module class is called ExampleServerSideFunctions which is declared in the header file ExampleServerSideFunctions.h:

class ExampleServerSideFunctions: public BESAbstractModule {
public:
    ExampleServerSideFunctions() {}
    virtual ~ExampleServerSideFunctions() {}
    virtual void initialize(const string &modname);
    virtual void terminate(const string &modname);
    virtual void dump(ostream &strm) const;
};


and implemented in ExampleServerSideFunctions.cc:

void ExampleServerSideFunctions::initialize(const string &modname) {
    BESDEBUG( "ExampleServerSideFunctions", "Initializing ExampleServerSideFunctions:" << endl );

    // Here we wrap the helloWorld() function in an instance of the libdap::ServerFunction class
    libdap:ServerFunction *helloWorld =
        new libdap::ServerFunction(
            "helloWorld", // The name of the function as it will appear in a constraint expression
            "1.0",        // The version of the function
            "Returns a DAP String whose value is 'Hello World'", // A brief description of the function
            "helloWorld()", // A usage/syntax statement
            "http://docs.opendap.org/index.php/Hyrax:_Server_Side_Functions", // A URL that points two a web page describing the function
            "http://services.opendap.org/dap4/server-side-function/helloWorld", // A URI that defines the role of the function
            example_ssf::helloWorld // A pointer to the helloWorld() function
        );

    // Here we add our new instance of libdap::ServerFunction to the libdap::ServerFunctionsList.
    libdap::ServerFunctionsList::TheList()->add_function(helloWorld);

    BESDEBUG( "ExampleServerSideFunctions", "Done initializing ExampleServerSideFunctions" << endl );
}

void ExampleServerSideFunctions::terminate(const string &modname) {
    BESDEBUG( "ExampleServerSideFunctions", "Removing ExampleServerSideFunctions module (this does nothing)." << endl );
}

extern "C" {
    BESAbstractModule *maker() {
        return new ExampleServerSideFunctions;
    }
}

Note carefully the method ExampleServerSideFunctions.initialize() which contains the call to the libdap:ServerFunctionsList.addFunction() method. It is this call that causes a HelloWorld class instance and thus the helloWorld() function to be registered in BES.

And that's pretty much all there is to the C/C++ programming. What remains is to build a little autotools project for your function code and set it up to install into the BES when make install is run.