OPeNDAP Developer's Workshop 2010: Difference between revisions
From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
Line 67: | Line 67: | ||
==== Admin interface | ==== Server Administration ==== | ||
==== Admin interface ==== | |||
::* admin connects to a listener | ::* admin connects to a listener | ||
::* can restart a listener | ::* can restart a listener | ||
Line 77: | Line 78: | ||
:: http://docs.opendap.org/index.php/Hyrax_Admin_Interface | :: http://docs.opendap.org/index.php/Hyrax_Admin_Interface | ||
* drop the besdaemon | |||
* listener is started instead of daemon | |||
* listener keeps track of connections made and socket connected to | |||
* listener keeps track of connections dropped, removes from list | |||
* limit the number of open connections | |||
* besdaemon will be able to have both hard and graceful restart | |||
* Out of band comm will enable access to logs and conf files in addition to the restarts | |||
* capability to send back configuration information | |||
* capability to receive new configuration information and write to disk and reload | |||
* we don't sweat the fact that the logs are shared - we can build a fancy interface out at the servlet level to filter by specific client using the PID info in the log. | |||
* Changes in the conf only are used when a process is started. | |||
* Soft shutdown, listener sends certain signal to child, when done handling current request, go down | |||
* Hard shutdown, listener sends certain signal to child, go down now | |||
* capability to stream back the log file | |||
==== Authentication and Autorization ==== | |||
* Will people be authenticating through ESG to get to Hyrax? Or will Hyrax need to have an authentication piece? Authorization to ESG AuthZ. | |||
* encrypting the data? Is it through the BES or through the middle tier? If middle tier then no need to encrypt the information through the BES. | |||
* client application to need to allow login to get data? Scientist gets a URL back to access data through OPeNDAP. To get that data, have to be logged in? Authorization needs to happen. | |||
* Certificate is on the client disk, when request made within matlab client, grabs certificate and sends along with data URL | |||
==Throttling the response== | |||
* Function added to DDS and base types to be able to pre-compute the response size based on projections and selections | |||
* Should also work for sequences, at least be able to say that you can have x number of rows returned. | |||
Revision as of 17:11, 6 October 2010
Where & When
- Location: Troy, NY
- Dates: 6-8 Oct
Meeting modus operandi
- mockups
- whiteboard designs
- use cases
- best to exploit face-to-face for what it affords.
Agenda
Wednesday
- 0830 - Coffee
- 0900 -
Server Side Functions
- Registration
- modularized
- each function -> class
- version
- can you handle this
- service
- how do we package this
- Discovery
- Ask the OLFS/BES what functions for dataset
- what functions
- capabilities
- Ask the OLFS/BES what functions for dataset
- Format
- function(p1,p2,p3,p4...)
- Usage
- Run by handler during loading - slection
- Run during constraint parsing
- Run during transmission/serialization (constraint evaluation)
- What about functions like "version"??
void function_version(int, BaseType *[], DDS &, BaseType **btpp) typedef void(*bool_func)(int argc, BaseType *argv[], DDS &dds, bool *result); typedef void(*btp_func)(int argc, BaseType *argv[], DDS &dds, BaseType **btpp); typedef void(*proj_func)(int argc, BaseType *argv[], DDS &dds, ConstraintEvaluator &ce);
Use Cases
- Scientist has a dataset and wants to discover what functions are available to apply to that dataset
- Data provider has a set of server-side functions that they wish to provide for data users. They want to add these modules to the OPeNDAP via dynamically loaded modules
- Scientist has a dataset and wants to apply geogrid function to that dataset
- Scientist wants to discover the the shape and size (metadata) of a particular function against a dataset
Plan: Define a class/hierarchy
- Define a set of classes for server-saide functions
- Have the BES instantiate those from .so files
- Those instances are then passed into a ConstraintEvaluator instance which is then passed to a handler
- The BES conf file(s) can include information so that some functions are always loaded and some are loaded only for specific handlers (e.g., FreeForm).
- Take the server side functions out of libdap and add to BES modules
- Add the server side functions to server side class
- These classes will be able to handle things like get version, given a ddx(?) can this function be run against it (return true/false)
- At load time the classes are created within the BES, the functions are registered with libdap. This maintains binary compatability.
Server Administration
Admin interface
- admin connects to a listener
- can restart a listener
- can restart the BES with ADMIN
- turn on/off debugging through admin interface
- keep track of how many connections are allowed/active
- re-write besdaemon/beslistener (2-3 hrs)
- allowed size of response from BES (1 hr)
- drop the besdaemon
- listener is started instead of daemon
- listener keeps track of connections made and socket connected to
- listener keeps track of connections dropped, removes from list
- limit the number of open connections
- besdaemon will be able to have both hard and graceful restart
- Out of band comm will enable access to logs and conf files in addition to the restarts
- capability to send back configuration information
- capability to receive new configuration information and write to disk and reload
- we don't sweat the fact that the logs are shared - we can build a fancy interface out at the servlet level to filter by specific client using the PID info in the log.
- Changes in the conf only are used when a process is started.
- Soft shutdown, listener sends certain signal to child, when done handling current request, go down
- Hard shutdown, listener sends certain signal to child, go down now
- capability to stream back the log file
Authentication and Autorization
- Will people be authenticating through ESG to get to Hyrax? Or will Hyrax need to have an authentication piece? Authorization to ESG AuthZ.
- encrypting the data? Is it through the BES or through the middle tier? If middle tier then no need to encrypt the information through the BES.
- client application to need to allow login to get data? Scientist gets a URL back to access data through OPeNDAP. To get that data, have to be logged in? Authorization needs to happen.
- Certificate is on the client disk, when request made within matlab client, grabs certificate and sends along with data URL
Throttling the response
- Function added to DDS and base types to be able to pre-compute the response size based on projections and selections
- Should also work for sequences, at least be able to say that you can have x number of rows returned.
- 1230 - Lunch
- 1300 -
- Active file system
- Nicer install for Hyrax
- RPM, yum, apt-get, something like what perl has, mac port, etc...
- SQL Data Handler
- Patrick explains all...
- 1800 - Dinner
Thursday
- 0830 - Coffee
- 0900 -
- NcML Ingestion Break Out
- BES Internal Caching
- 1230 - Lunch
- 1300 -
- 1400-1800
- Strategy Breakout
- Peter, Dan, James."
- 1400-1800
- NcML Ingestion Break Out
- Patrick, Micheal, Nathan
- Instead of reading NCML file, taking an NCML document (1 hr)
- Issues with adding new BES commands? While adding this new functionality document current interface for adding commands to the BES architecture.
- 1800 - Dinner
Friday
- 0830 - Coffee
- 0900 -
- Module check
- Making sure a loaded module matches the version of BES and any required modules
- SSL authentication/authorization issues
- Currently problem with SSL authentication and keeping SSL channel open for secure communication
- Multiple catalogs
- BES and OLFS integration
- 1230 - Lunch
- 1300 -
- 1400-1800
- Strategy Breakout (cont)
- 1800 - Dinner and Flee!