IOOS Meeting 03.22.2010

DAP 3.x and 4.x

• TDS and new DAP standards

Claim

For DAP 3.x and 4.x to become widely adopted we will need to get the TDS to become compatible with these versions.

The TDS is clearly the dominant DAP service software. This position makes it difficult (if not impossible) for us to push the DAP standards forward without enabling the TDS. INn order for the TDS to produce DAP 3.x and 4.x output we will need to start supporting these updated protocols in the Java-DAP.

Assertion

If we don't do this, then changes to the DAP will be irrelevant.

• Do we want to make the DAP 3.2 DDX the default DDX response?
  • Are there any users of the DAP 2.0 DDX?

WCS Semantic Software Activities

Need code review to see if we can make it more robust and efficient.

• Why are updates so slow?
  • It takes a long time to verify if changes have been made to the imports.
  • Even when no changes have been made to the imports list, semantic operations are still run. This seems counterintuitive: Can we change that? If nothing has changed then no additional semantic operations should be undertaken. Currently the last modified times of the imports are stored in the repository. This seems to be a very slow and unreliable mechanism for reevaluating the imports.
  • We need to evaluate what do to when things have changed. Is it more expedient to remove and replace changed values from the repository, or should we just rebuild the whole thing?
  • Can we pre-compute a starting point repository that contains all of the inferencing rules and ontologies? This would allow us to avoid the long slow acquisition of all of these files every time we rebuild the repository.
  • Should we consider caching the import list and last modified times as directly accessible java objects so that we can write simple java procedural code to evaluate the imports list?

• At start up the catalog is empty prior to the completion of the semantic operations.
  • Should we cache the catalog?
  • Should we consider making the service simply create the appropriate inputs for the LocalFileCatalog and skip more tightly coupled integration?

• Let's make the WCS service a stand alone web app. A seperate jar file that runs in it's own context in the servlet engine.
  • Fix the local/universal ID problem.
  • Re-factor the code as a servlet in it's own context.

GeoTIFF Module

We need to beg/borrow/steal the time to build a GeoTIFF module for the BES. The idea is that like the netcdf_fileout it would return data in a GeoTIFF format. Obviously Grids would map well to GeoTIFF (assuming that the Grid has the requisite maps). We would need to either creatively map other data types/structures or simply deny GeoTIFF access for unmapped data types. Even if we never build releasable WMS or WCS services adding this would allow others to easily wrap those services around our server.

Just creating this kind of output has the potential to increase our user base immensely.

KML Module

Same arguments as GeoTIFF.

THREDDS Catalog Clients

IDV seems like it might be a client that can navigate catalogs. (Not surprising, considering it's a UNIDATA thing)

Several clients demonstarted at IOOS meeting that allowed users to navigate THREDDS catalogs in search of data:

• Environmental Data Connector
• ERDDAP
• IOOS NCV

Dap Capabilities Response

• Services
• Return Types/Formats
• Server Side Functions
• Catalog
  • What catalog metadata should we provide so that a complete enough picture of the holdings is available? The intent it to make it so that users don't need to delve into the granules to determine if the holding contains the information that they desire. See THREDDS Catalog Metadata section below.

THREDDS Catalog Metadata

• How do we propagate (some subset of) dataset metadata "up" into our THREDDS catalogs? People are requesting this.

• Can we use NcML to push new metadata down into collections vis-a-vis the inherited metadata section of a dataset scan? I mean add metadata terms to the thredds catalogs using NcML. OtherXML? Attributes? Both?

• How do we add new services (say for example WCS) to the services listed in our THREDDS catalogs along with the appropriate data access links for the appropriate datasets? Is this even realistic given the architecture of our WCS?

• This is requested functionality for National Catalog Viewer in development by IOOS. And is needed by June 2010. Can we pull that off?

DAP Asynchronous Responses

• Could this be implemented by altering the dap:blob element? Currently we propose using an href attribute to hold the content ID for the MIME part that holds the data:

<dap:blob href="someUUID" />

We might consider allowing an alternate representation:

<dap:blob 
       xlink:href="http://the.server/location/where/you/can/get/the/binary/part"  
       xlink:type="simple"
       available="TimeItWillBeAvailableInSomeISOFormat"  
   />

That would indicate to the user that the content will be available asynchronously.

To Do

• Look at "Fudge Messaging"