OPeNDAP Developer's Workshop 2010

From OPeNDAP Documentation
Revision as of 17:34, 6 October 2010 by Ndp (talk | contribs) (→‎Lunch)
⧼opendap2-jumptonavigation⧽

Where & When

  • Location: Troy, NY
  • Dates: 6-8 Oct

Meeting modus operandi

  • mockups
  • whiteboard designs
  • use cases
  • best to exploit face-to-face for what it affords.

Agenda

Wednesday

0830 - Coffee
0900 -


Server Side Functions

  • Registration
    • modularized
    • each function -> class
    • version
      • can you handle this
      • service
      • how do we package this
  • Discovery
    • Ask the OLFS/BES what functions for dataset
      • what functions
      • capabilities
  • Format
    • function(p1,p2,p3,p4...)
  • Usage
    • Run by handler during loading - slection
    • Run during constraint parsing
    • Run during transmission/serialization (constraint evaluation)
    • What about functions like "version"??


void
function_version(int, BaseType *[], DDS &, BaseType **btpp)

typedef void(*bool_func)(int argc, BaseType *argv[], DDS &dds, bool *result);
typedef void(*btp_func)(int argc, BaseType *argv[], DDS &dds, BaseType **btpp);
typedef void(*proj_func)(int argc, BaseType *argv[], DDS &dds, ConstraintEvaluator &ce);


Use Cases
  • Scientist has a dataset and wants to discover what functions are available to apply to that dataset
  • Data provider has a set of server-side functions that they wish to provide for data users. They want to add these modules to the OPeNDAP via dynamically loaded modules
  • Scientist has a dataset and wants to apply geogrid function to that dataset
  • Scientist wants to discover the the shape and size (metadata) of a particular function against a dataset
Plan: Define a class/hierarchy
  • Define a set of classes for server-saide functions
  • Have the BES instantiate those from .so files
  • Those instances are then passed into a ConstraintEvaluator instance which is then passed to a handler
  • The BES conf file(s) can include information so that some functions are always loaded and some are loaded only for specific handlers (e.g., FreeForm).
  • Take the server side functions out of libdap and add to BES modules
  • Add the server side functions to server side class
  • These classes will be able to handle things like get version, given a ddx(?) can this function be run against it (return true/false)
  • At load time the classes are created within the BES, the functions are registered with libdap. This maintains binary compatability.


Server Administration

Admin interface
  • admin connects to a listener
  • can restart a listener
  • can restart the BES with ADMIN
  • turn on/off debugging through admin interface
  • keep track of how many connections are allowed/active
  • re-write besdaemon/beslistener (2-3 hrs)
  • allowed size of response from BES (1 hr)
http://docs.opendap.org/index.php/Hyrax_Admin_Interface
BES design changes
  • drop the besdaemon from the architecture
  • listener is started instead of daemon
  • listener keeps track of connections made and socket connected to
  • listener keeps track of connections dropped, removes from list
  • limit the number of open connections
  • besdaemon will be able to have both hard and graceful restart
  • Out of band comm will enable access to logs and conf files in addition to the restarts
  • capability to send back configuration information
  • capability to receive new configuration information and write to disk and reload
  • we don't sweat the fact that the logs are shared - we can build a fancy interface out at the servlet level to filter by specific client using the PID info in the log.
  • Changes in the conf only are used when a process is started.
  • Soft shutdown, listener sends certain signal to child, when done handling current request, go down
  • Hard shutdown, listener sends certain signal to child, go down now
  • capability to stream back the log file
Authentication and Authorization
  • Will people be authenticating through ESG to get to Hyrax? Or will Hyrax need to have an authentication piece? Authorization to ESG AuthZ.
  • encrypting the data? Is it through the BES or through the middle tier? If middle tier then no need to encrypt the information through the BES.
  • client application to need to allow login to get data? Scientist gets a URL back to access data through OPeNDAP. To get that data, have to be logged in? Authorization needs to happen.
  • Certificate is on the client disk, when request made within matlab client, grabs certificate and sends along with data URL
Throttling the response
  • Function added to DDS and base types to be able to pre-compute the response size based on projections and selections
  • Should also work for sequences, at least be able to say that you can have x number of rows returned.



Lunch

  • James - Pizza
  • Dan - Corned Beef
  • Patrick - Pizza
  • Michael - Chikin Sandwich
  • Nathan -Club sandwich


more

1300 -
Active file system
Nicer install for Hyrax
RPM, yum, apt-get, something like what perl has, mac port, etc...
SQL Data Handler
Patrick explains all...


1800 - Dinner

Thursday

0830 - Coffee
0900 -
NcML Ingestion Break Out
BES Internal Caching
1230 - Lunch
1300 -
1400-1800
Strategy Breakout
  • Peter, Dan, James."
1400-1800
NcML Ingestion Break Out
  • Patrick, Micheal, Nathan
  • Instead of reading NCML file, taking an NCML document (1 hr)
  • Issues with adding new BES commands? While adding this new functionality document current interface for adding commands to the BES architecture.


1800 - Dinner


Friday

0830 - Coffee
0900 -
Module check
Making sure a loaded module matches the version of BES and any required modules


SSL authentication/authorization issues
Currently problem with SSL authentication and keeping SSL channel open for secure communication


Multiple catalogs
BES and OLFS integration


1230 - Lunch
1300 -


1400-1800
Strategy Breakout (cont)



1800 - Dinner and Flee!
Retrieved from "https://docs.opendap.org/index.php?title=OPeNDAP_Developer%27s_Workshop_2010&oldid=5832"