DAP4: Capabilities and Versioning: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
mNo edit summary
 
(26 intermediate revisions by 2 users not shown)
Line 38: Line 38:
The '''dataset''' section should include
The '''dataset''' section should include
* a link to the full dataset capabilities document (this document)
* a link to the full dataset capabilities document (this document)
* links for all resource types for each supported version
* links to all available OPeNDAP services for the dataset
* a list of extensions the dataset supports
* a list of extensions supported for this dataset (links?)
* a list of the types of constraint expressions supported for this dataset
* a link to a constraint expression document detailing the constraint supported for this dataset
* a list of the types of processing instructions supported for this dataset
* a list of the types of processing instructions supported for this dataset


==== Link Relationship Type ====
Dataset capabilities documents can be encoded in the following media types:


'''<nowiki>http://opendap.org/rel/dap/dataset</nowiki>'''
* '''application/vnd.opendap.org.capabilities+xml'''
* '''application/vnd.opendap.org.capabilities+json'''
* '''application/vnd.opendap.org.capabilities+pbuf'''
 
Definitions of these media types can be found in :


The URL of a link with this relation type can be used to GET the dataset capabilities document for the dataset.
* The DAP4 Capabilities XML schema (link) which is declared in the namespace '''<nowiki>http://opendap.org/ns/dap/capabilities</nowiki>'''.
* The DAP4 Capabilities JSON definition (link).
* The DAP4 Capabilities protobuf definition (link).


==== Possible Media Types ====
A link to a Dataset Capabilities document can be identified by the following link relationship type:


* '''application/vnd.opendap.org.capabilities+xml'''
* '''<nowiki>http://opendap.org/rel/dap/dataset</nowiki>'''
* '''application/vnd.opendap.org.capabilities+json'''
* '''application/vnd.opendap.org.capabilities+pbuf'''


More formal definitions of these media types can be found:
The URL of a link with this relation type can be used to GET the dataset capabilities document for the dataset.


* The XML representation is a document that follows the DAP Capabilities XML schema (definition??) and is declared in the namespace '''<nowiki>http://opendap.org/ns/dap/capabilities</nowiki>'''.
An example of an HTTP request for an XML encoded dataset capabilities document is given [[#Server Capabilities Document Request-Response|below]].
* The JSON representation is a document following the definition given here (???).
* The protobuf representation is a document following the definition given here(???).


An example of an HTTP request for an XML encoded dataset capabilities document is given [[Example Server Capabilities Response/Request|below]].
=== DAP4 DDX ===


=== Other Resource Types ===
A DAP4 DDX document describes the structure of a dataset and contains other metadata for that dataset. It must contain a link to the DAP4 Data and to a CE document that describes the constraints supported for this dataset.


Dataset capabilities documents will include links to the available OPeNDAP services for the dataset. The type of resource at the end of a link will be indicated by the link relation type. The DAP4 core resource types as well as some possible extensions are listed in this section. Details on each resource type are discussed in another section below.
DAP4 DDX documents can be encoded in the following media types:


For DAP4, the core set of resource types are:
* '''application/vnd.opendap.org.dap4.ddx'''
* '''application/vnd.opendap.org.dap4.ddx+pbuf'''


* [[#DAP4 DDX|DAP4 DDX]]
Definitions of these media types can be found in:
* [[#DAP4 Data|DAP4 Data]]
* [[#DAP4 Constraint Expression|DAP4 Constraint Expression]]
* [[#DAP4 Server Capabilities Document|DAP4 Server Capabilities Document]]


For DAP2, the core set of resource types are:
* The DAP4 DDX XML schema (link) which is declared in the namespace '''<nowiki>http://opendap.org/ns/dap4/ddx</nowiki>'''.
* The DAP4 DDX JSON definition (link).
* The DAP4 DDX protobuf definition (link).


* [[#DAP2 DAS|DAP2 DAS]]
A link to a DAP4 DDX document can be identified by the following link relationship type:
* [[#DAP2 DDS|DAP2 DDS]]
* [[#DAP2 DDX|DAP2 DDX]]
* [[#DAP2 Data|DAP2 Data]]
* [[#DAP2 Constraint Expression|DAP2 Constraint Expression]]


A number of extensions to the core DAP4 protocol have been discussed. Here are a few extensions and some resource types that we might consider:
* '''<nowiki>http://opendap.org/rel/dap4/ddx</nowiki>'''


* DAP4 Asynchronous Request/Response
The URL of a link with this relation can be used to GET a DAP4 DDX resource.  
** Asynchronous information (<nowiki>http://opendap.org/rel/dap/asynchronous</nowiki>)
* DAP4 Server-Side Processing
** Processing instructions (<nowiki>http://opendap.org/rel/dap4/processing</nowiki>)


== Rationale for the solution ==
=== DAP4 Data ===


* Allows servers full control of their URL space
A DAP4 Data resource contains a subset of data from the dataset. They may be encoded in any of the following media types:
*** DAP4 URLs are identified by their link type rather than a hard wired "file extension".
*** On the other hand, file extensions can also be supported.


* '''application/vnd.opendap.org.dap4.data'''
* '''application/vnd.opendap.org.dap4.data+ascii'''
* '''application/vnd.opendap.org.dap4.data+json'''
* '''application/vnd.opendap.org.dap4.data+ncstream'''


== Discussion ==
Definitions of these media types can be found in:


----
* The DAP4 Data encoding specification (link).
* The DAP4 Data ASCII/UTF-8 encoding specification (link).
* The DAP4 Data JSON definition (link).
* The DAP4 Data ncstream/cdmremote definition (link).


== DAP4 Resource Types ==
A link to a DAP4 DDX document can be identified by the following link relationship type:


=== DAP4 DDX ===
'''<nowiki>http://opendap.org/rel/dap4/data</nowiki>'''


A DAP4 DDX document contains the metadata that describes the dataset.
The URL of a link with this relation can be used to GET or POST a DAP4 data resource. (POST is used if a dataset session is desired, i.e., to freeze dynamically changing datasets, or if an asynchronous response is expected.)


==== Link Relationship Type ====
=== DAP4 Constraint Expression ===


'''<nowiki>http://opendap.org/rel/dap4/ddx</nowiki>'''
A DAP4 Constraint Expression document contains a constraint that is to be applied to the dataset. They may be encoded in any of the following media types:


The URL of a link with this relation can be used to GET a DAP4 DDX resource.  
* '''application/vnd.opendap.org.dap4.ce''' (for GET, URL encoding)
* '''application/vnd.opendap.org.dap4.ce+xml''' (for POST, request body encoding)


==== Possible Media Types ====
Definitions of these media types can be found in:


* '''application/vnd.opendap.org.dap4.ddx'''
* The DAP4 CE URL encoding specification (link).
* '''application/vnd.opendap.org.dap4.ddx.pbuf'''
* The DAP4 CE XML schema (link) which is declared in the namespace '''<nowiki>http://opendap.org/ns/dap4/ce</nowiki>'''.
* The DAP4 Data JSON definition (link).
* The DAP4 Data ncstream/cdmremote definition (link).


A representation of this media type contains a DAP4 DDX document declared in the namespace <nowiki>http://opendap.org/nc/dap4/ddx</nowiki>.
A link to a DAP4 CE document can be identified by the following link relationship type:


=== DAP4 Data ===
* '''<nowiki>http://opendap.org/rel/dap4/ce</nowiki>'''
==== Link Relationship Type ====
==== Possible Media Types ====


* '''application/vnd.opendap.org.dap4.data'''
The URL of a link with this relation can be used to GET a DAP4 constraint expression document.
* '''application/vnd.opendap.org.dap4.data+ascii'''
* '''application/vnd.opendap.org.dap4.data+json'''
* '''application/vnd.opendap.org.dap4.data+ncstream'''


A representation encoded in one of these media types contains DAP4 data.  
Generally constraint expression are only used as part of a data request. This link relation type is given in case stand-alone constraint expression resources are found useful.


=== DAP4 Constraint Expression ===
<blockquote>NOTE: Could this document describe the types of constraints that can be applied to particular variables in a dataset? Is such a thing needed? If so, should it be a separate resource type.</blockquote>
==== Link Relationship Type ====
==== Possible Media Types ====


* '''application/vnd.opendap.org.dap4.ce'''
=== DAP4 Server Capabilities Documents ===
* '''application/vnd.opendap.org.dap4.ce+xml'''


A representation encoded in one of these media types contains a DAP4 constraint expression.
Do we need/want a server-level capabilities document? Or is a dataset-level one enough?


=== DAP4 Processing Instructions ===
==== Link Relationship Type ====
==== Possible Media Types ====


* '''application/vnd.opendap.org.dap4.processing+ferret'''
Server capabilities documents are encoded in the same media types that are defined for the DAP4 Dataset Capabilities Documents (see [[#DAP4 Dataset Capabilities Documents|above]]).
* '''application/vnd.opendap.org.dap4.processing+grads'''
* '''application/vnd.opendap.org.dap4.processing+jython'''
* '''application/vnd.opendap.org.dap4.processing+jvm'''


A representation encoded in one of these media types contains processing instructions for the DAP4 server.
A link to a Server Capabilities document can be identified by the following link relationship type:


=== DAP4 Server Capabilities Documents ===
* '''<nowiki>http://opendap.org/rel/dap/server</nowiki>'''
==== Link Relationship Type ====
==== Possible Media Types ====


== DAP2 Resource Types ==
The URL of a link with this relation type can be used to GET the server capabilities document for the server.


=== DAP2 DAS ===
=== DAP2 DAS ===
==== Link Relationship Type ====
==== Possible Media Types ====


'''application/vnd.opendap.org.dap2.das'''
'''application/vnd.opendap.org.dap2.das'''
Line 165: Line 150:


=== DAP2 DDS ===
=== DAP2 DDS ===
==== Link Relationship Type ====
==== Possible Media Types ====


'''application/vnd.opendap.org.dap2.dds'''
'''application/vnd.opendap.org.dap2.dds'''
Line 173: Line 156:


=== DAP2 DDX ===
=== DAP2 DDX ===
==== Link Relationship Type ====
==== Possible Media Types ====


'''application/vnd.opendap.org.dap2.ddx'''
'''application/vnd.opendap.org.dap2.ddx'''
Line 181: Line 162:


=== DAP2 Data ===
=== DAP2 Data ===
==== Link Relationship Type ====
==== Possible Media Types ====


* '''application/vnd.opendap.org.dap2.data'''
* '''application/vnd.opendap.org.dap2.data'''
Line 190: Line 169:


=== DAP2 Constraint Expression ===
=== DAP2 Constraint Expression ===
==== Link Relationship Type ====
==== Possible Media Types ====


'''application/vnd.opendap.org.dap2.ce'''
* '''application/vnd.opendap.org.dap2.ce'''


A representation of this media type contains a DAP2 constraint expression. Generally used as a DAP2 data URL query string. See the DAP2 specification for details.
A representation of this media type contains a DAP2 constraint expression. Generally used as a DAP2 data URL query string. See the DAP2 specification for details.
=== Extension Resource Types ===
A number of extensions to the core DAP4 protocol have been discussed. Here are a few extensions and some resource types that we might consider:
=== DAP4 Asynchronous Request/Response ===
See [[DAP4: Asynchronous Request-Response Proposal]]
=== DAP4 Server-Side Processing Instructions ===
* '''application/vnd.opendap.org.dap4.processing+ferret'''
* '''application/vnd.opendap.org.dap4.processing+grads'''
* '''application/vnd.opendap.org.dap4.processing+jython'''
* '''application/vnd.opendap.org.dap4.processing+jvm'''
* '''<nowiki>http://opendap.org/rel/dap4/processing</nowiki>'''
== Rationale for the solution ==
In an attempt to take advantage of the architecture of the web, this proposal is based on some RESTful ideas. REST targets networks of hypermedia resources connected to each other by the hyperlinks those resources contain. The links in the network can have link types that specify the type of resource that will be found at the end of the link. Link types define the set of information and the links that can be contained by a resource of that type. Each resource type can have multiple encodings in which a resource of that type can be represented.
This architecture can make for a robust and extensible system. URLs can change and new links can be added without breaking the system.  The links in a resource provide an application the set of next available steps the application can take. Or another way of thinking about it, the network of hypermedia resources represent a virtual state machine for the web application.
By specifying all URLs in links rather than defining them in a specification, the form of the URLs becomes an implementation detail, determined by the server, rather than a part of the specification. This helps remove some of the tight coupling often seen between client and server. (This does not, however, rule out the use of URL conventions, like file extensions to specify encoding forms. Some recommend support for both HTTP '''Accept''' headers and for file extensions [1].)
By thinking in terms of link (resource) types and media types
(encoding formats), specifications have a natural division into
separate components.
* The specification of a link/resource type defines the set of information and links allowed in a resources of that type.
* The specification of a media type defines how the information and links allowed in the targetted resource type are encoded.
* The specification of how the network application is bound to a specific transport protocol (like HTTP).
To take advantage of the caching built into the web
infrastructure, one should think about commonly used resources and
whether they can be made somewhat stable and quick to return. Or at
least be sure HTTP caching headers are used properly.
The bottom line is that features of the chosen transport protocol should be used by web applications to the fullest extent possible to gain the advantages of the features of that transport protocol. A relevant quote from Wikipedia's REST page [2]:
<blockquote>
RESTful applications maximize the use of the pre-existing,
well-defined interface and other built-in capabilities provided by
the chosen network protocol, and minimize the addition of new
application-specific features on top of it.
</blockquote>
and another one specific to HTTP [2]:
<blockquote>
HTTP, for example, has a very rich vocabulary in terms of verbs
(or "methods"), URIs, Internet media types, request and response
codes, etc. REST uses these existing features of the HTTP
protocol, and thus allows existing layered proxy and gateway
components to perform additional functions on the network such as
HTTP caching and security enforcement.
</blockquote>
== Discussion ==
[[User:Jimg|Jimg]] 13:48, 10 May 2012 (PDT) I would like to see this broken up into two or more papers where one specifies the proposed HTTP implementation of the requests and responses of DAP4. Others can specify the structure and interpretation of the Capabilities response and how a web service will support multiple versions of DAP, including DAP2 and DAP4 (and variants of those protocols, should we allow that).


== Examples ==
== Examples ==


=== Example Server Capabilities Response/Request ===
=== Server Capabilities Document Request-Response ===


Client uses DAP dataset URL, '''http://server.org/dap/path/data.nc''', to GET the dataset capabilities resource as an XML document:
Client uses DAP dataset URL, '''<nowiki>http://server.org/dap/path/data.nc</nowiki>''', to GET the dataset capabilities resource as an XML document:


<pre>
<font size="2"><pre>
GET /dap/path/data.nc HTTP/1.1
GET /dap/path/data.nc HTTP/1.1
Host: server.org
Host: server.org
Accept: application/vnd.opendap.org.capabilities+xml
Accept: application/vnd.opendap.org.capabilities+xml
</pre>
</pre></font>


The response might look like like:
The response might look like like:


<pre>
<font size="2"><source lang="xml">
200 OK
200 OK
Content-Type: application/vnd.opendap.org.capabilities+xml;charset=UTF-8
Content-Type: application/vnd.opendap.org.capabilities+xml;charset=UTF-8
Line 246: Line 286:
                   href="http://server/dap/path/data.nc.ddx"/>
                   href="http://server/dap/path/data.nc.ddx"/>
       <serviceLink rel="http://opendap.org/rel/dap2/dods"
       <serviceLink rel="http://opendap.org/rel/dap2/dods"
                   type="application/vnd.opendap.org.dap2.dods,
                   type="application/vnd.opendap.org.dap2.dods"
                        application/vnd.opendap.org.dap2.dods+ascii"
                   href="http://server/dap/path/data.nc.dods"/>
                   href="http://server/dap/path/data.nc.dods"/>
      <serviceLink rel="http://opendap.org/rel/dap2/dods"
                  type="application/vnd.opendap.org.dap2.dods+ascii"
                  href="http://server/dap/path/data.nc.ascii"/>
       <constraintExp ...>...</constraintExp>
       <constraintExp ...>...</constraintExp>
       <processing ...>...</processing>
       <processing ...>...</processing>
Line 270: Line 312:
   </dataset>
   </dataset>
</capabilities>
</capabilities>
</pre>
</source></font>
 
=== Constrained Data Request-Response ===
 
Request:
<font size="2"><pre>
GET /dap/path/data.nc?x,y,temp HTTP/1.1
Host: server.org
Accept: application/vnd.opendap.org.dap4.data
</pre></font>
 
Response:
<font size="2"><source lang="xml">
200 OK
Content-Type: application/vnd.opendap.org.dap4.data
<ddx>...</ddx>...BINARY DATA...
</source></font>
 
=== Create New Dataset with Constrained Data Request-Response ===
 
Request:
<font size="2"><pre>
POST /dap/path/data.nc HTTP/1.1
Host: server.org
Content-Type: application/vnd.opendap.org.dap4.ce+xml;charset=UTF-8
<constraintExp>...</constraintExp>
</pre></font>
 
Response:
<font size="2"><source lang="xml">
201 Created
Content-Type: application/vnd.opendap.org.dap4.capabilities+xml;charset=UTF-8
<capabilities xmlns="http://opendap.org/ns/dap/capabilities">
  <server ...>...</server>
  <dataset rel="self"
          type="application/vnd.opendap.org.capabilities+xml,
                application/vnd.opendap.org.capabilities+json,
                application/vnd.opendap.org.capabilities+pbuf"
          href="http://server/dap/path/data.nc-ABCD">
    <dap:source>
      <dataset type="application/vnd.opendap.org.capabilities+xml,
                    application/vnd.opendap.org.capabilities+json,
                    application/vnd.opendap.org.capabilities+pbuf"
              href="http://server/dap/path/data.nc" />
      <constraintExp>
        ...
      </constraintExp>
    </dap:source>
    <version name="dap2">
      ...
    </version>
    <version name="dap4">
      ...
    </version>
  </dataset>
</capabilities>
</source></font>
 
== Some References ==
 
[1] http://stackoverflow.com/a/385216
* A stackoverflow answer to the question "Use 'Accept' header or extension to identify resource type?" Answer: the ideal would be to implement both.


=== Some References ===
[2] http://en.wikipedia.org/wiki/Representational_state_transfer


* http://stackoverflow.com/a/385216
[3] http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
** Response to the question: Use "Accept" header or extension to identify resource type?
** The ideal would be to implement both.

Latest revision as of 20:48, 10 May 2012

<-- back to OPULS Development

--EthanDavis 13:32, 16 April 2012 (PDT)

Background

Providing OPeNDAP clients with information on the particular capabilities of a given server and given dataset has been discussed many times. Types of capabilities include supported DAP versions, supported server-side functions. Another possible capability would be support for multiple encodings of the various OPeNDAP services. Providing capabilities documents can also make it easier to plan for extensibility and the future evolution of OPeNDAP.

Negotiating versions using HTTP headers or URL keywords is described in DAP 4.0 Design doc. Advertising DAP protocol services is described in the DAP Service Terminus document.

Problem addressed

The proposal attempts to address the following requirements:

  • Support for multiple versions of DAP protocol in a backwards compatible way
  • Support for extensibility and evolution of protocol
  • Support for multiple encoding formats (media types) for each resource type
  • Support description of server and dataset capabilities (e.g., server-side processing capabilities)

Proposed solution

Provide dataset capabilities documents from the raw DAP dataset URL. A dataset's capabilities document includes links to the various OPeNDAP services for that dataset. Each link is described with a link relationship type and a media type. The link relationship type indicates what type of resource will be returned. The media type indicates the document type in which the returned resource will be encoded.

Providing a dataset capabilities document with links to both DAP2 and DAP4 resources allows for DAP2 backward compatibility and will support protocol extensibility and future evolution of the protocol.

Raw DAP Dataset URLs Return Capabilities Documents

A GET request on a DAP dataset URL (e.g., http://server.org/some/path/data.nc) returns the capabilities document for the dataset. To take advantage of web caching, capabilities documents should try to be light weight (i.e., quick creation) and as stable as possible.

A dataset capabilities document must contain a server section and a dataset section.

The server section should include

  • a link to the full server capabilities document (if applicable)
  • a list of the DAP versions the server supports
  • the implementation version (e.g., "TDS 4.3.57" or "Hyrax 1.7.45")
  • a list of extensions the server supports

The dataset section should include

  • a link to the full dataset capabilities document (this document)
  • links to all available OPeNDAP services for the dataset
  • a list of extensions supported for this dataset (links?)
  • a link to a constraint expression document detailing the constraint supported for this dataset
  • a list of the types of processing instructions supported for this dataset

Dataset capabilities documents can be encoded in the following media types:

  • application/vnd.opendap.org.capabilities+xml
  • application/vnd.opendap.org.capabilities+json
  • application/vnd.opendap.org.capabilities+pbuf

Definitions of these media types can be found in :

  • The DAP4 Capabilities XML schema (link) which is declared in the namespace http://opendap.org/ns/dap/capabilities.
  • The DAP4 Capabilities JSON definition (link).
  • The DAP4 Capabilities protobuf definition (link).

A link to a Dataset Capabilities document can be identified by the following link relationship type:

  • http://opendap.org/rel/dap/dataset

The URL of a link with this relation type can be used to GET the dataset capabilities document for the dataset.

An example of an HTTP request for an XML encoded dataset capabilities document is given below.

DAP4 DDX

A DAP4 DDX document describes the structure of a dataset and contains other metadata for that dataset. It must contain a link to the DAP4 Data and to a CE document that describes the constraints supported for this dataset.

DAP4 DDX documents can be encoded in the following media types:

  • application/vnd.opendap.org.dap4.ddx
  • application/vnd.opendap.org.dap4.ddx+pbuf

Definitions of these media types can be found in:

  • The DAP4 DDX XML schema (link) which is declared in the namespace http://opendap.org/ns/dap4/ddx.
  • The DAP4 DDX JSON definition (link).
  • The DAP4 DDX protobuf definition (link).

A link to a DAP4 DDX document can be identified by the following link relationship type:

  • http://opendap.org/rel/dap4/ddx

The URL of a link with this relation can be used to GET a DAP4 DDX resource.

DAP4 Data

A DAP4 Data resource contains a subset of data from the dataset. They may be encoded in any of the following media types:

  • application/vnd.opendap.org.dap4.data
  • application/vnd.opendap.org.dap4.data+ascii
  • application/vnd.opendap.org.dap4.data+json
  • application/vnd.opendap.org.dap4.data+ncstream

Definitions of these media types can be found in:

  • The DAP4 Data encoding specification (link).
  • The DAP4 Data ASCII/UTF-8 encoding specification (link).
  • The DAP4 Data JSON definition (link).
  • The DAP4 Data ncstream/cdmremote definition (link).

A link to a DAP4 DDX document can be identified by the following link relationship type:

http://opendap.org/rel/dap4/data

The URL of a link with this relation can be used to GET or POST a DAP4 data resource. (POST is used if a dataset session is desired, i.e., to freeze dynamically changing datasets, or if an asynchronous response is expected.)

DAP4 Constraint Expression

A DAP4 Constraint Expression document contains a constraint that is to be applied to the dataset. They may be encoded in any of the following media types:

  • application/vnd.opendap.org.dap4.ce (for GET, URL encoding)
  • application/vnd.opendap.org.dap4.ce+xml (for POST, request body encoding)

Definitions of these media types can be found in:

  • The DAP4 CE URL encoding specification (link).
  • The DAP4 CE XML schema (link) which is declared in the namespace http://opendap.org/ns/dap4/ce.
  • The DAP4 Data JSON definition (link).
  • The DAP4 Data ncstream/cdmremote definition (link).

A link to a DAP4 CE document can be identified by the following link relationship type:

  • http://opendap.org/rel/dap4/ce

The URL of a link with this relation can be used to GET a DAP4 constraint expression document.

Generally constraint expression are only used as part of a data request. This link relation type is given in case stand-alone constraint expression resources are found useful.

NOTE: Could this document describe the types of constraints that can be applied to particular variables in a dataset? Is such a thing needed? If so, should it be a separate resource type.

DAP4 Server Capabilities Documents

Do we need/want a server-level capabilities document? Or is a dataset-level one enough?


Server capabilities documents are encoded in the same media types that are defined for the DAP4 Dataset Capabilities Documents (see above).

A link to a Server Capabilities document can be identified by the following link relationship type:

  • http://opendap.org/rel/dap/server

The URL of a link with this relation type can be used to GET the server capabilities document for the server.

DAP2 DAS

application/vnd.opendap.org.dap2.das

A representation of this media type contains a DAP2 DAS document.

DAP2 DDS

application/vnd.opendap.org.dap2.dds

A representation of this media type contains a DAP2 DDS document.

DAP2 DDX

application/vnd.opendap.org.dap2.ddx

A representation of this media type contains a DAP2 DDX document declared in the namespace http://opendap.org/nc/dap2/ddx.

DAP2 Data

  • application/vnd.opendap.org.dap2.data
  • application/vnd.opendap.org.dap2.data+ascii

A representation of one of these media types contains DAP2 data. See the DAP2 specification for details.

DAP2 Constraint Expression

  • application/vnd.opendap.org.dap2.ce

A representation of this media type contains a DAP2 constraint expression. Generally used as a DAP2 data URL query string. See the DAP2 specification for details.


Extension Resource Types

A number of extensions to the core DAP4 protocol have been discussed. Here are a few extensions and some resource types that we might consider:

DAP4 Asynchronous Request/Response

See DAP4: Asynchronous Request-Response Proposal

DAP4 Server-Side Processing Instructions

  • application/vnd.opendap.org.dap4.processing+ferret
  • application/vnd.opendap.org.dap4.processing+grads
  • application/vnd.opendap.org.dap4.processing+jython
  • application/vnd.opendap.org.dap4.processing+jvm
  • http://opendap.org/rel/dap4/processing

Rationale for the solution

In an attempt to take advantage of the architecture of the web, this proposal is based on some RESTful ideas. REST targets networks of hypermedia resources connected to each other by the hyperlinks those resources contain. The links in the network can have link types that specify the type of resource that will be found at the end of the link. Link types define the set of information and the links that can be contained by a resource of that type. Each resource type can have multiple encodings in which a resource of that type can be represented.

This architecture can make for a robust and extensible system. URLs can change and new links can be added without breaking the system. The links in a resource provide an application the set of next available steps the application can take. Or another way of thinking about it, the network of hypermedia resources represent a virtual state machine for the web application.

By specifying all URLs in links rather than defining them in a specification, the form of the URLs becomes an implementation detail, determined by the server, rather than a part of the specification. This helps remove some of the tight coupling often seen between client and server. (This does not, however, rule out the use of URL conventions, like file extensions to specify encoding forms. Some recommend support for both HTTP Accept headers and for file extensions [1].)

By thinking in terms of link (resource) types and media types (encoding formats), specifications have a natural division into separate components.

  • The specification of a link/resource type defines the set of information and links allowed in a resources of that type.
  • The specification of a media type defines how the information and links allowed in the targetted resource type are encoded.
  • The specification of how the network application is bound to a specific transport protocol (like HTTP).

To take advantage of the caching built into the web infrastructure, one should think about commonly used resources and whether they can be made somewhat stable and quick to return. Or at least be sure HTTP caching headers are used properly.

The bottom line is that features of the chosen transport protocol should be used by web applications to the fullest extent possible to gain the advantages of the features of that transport protocol. A relevant quote from Wikipedia's REST page [2]:

RESTful applications maximize the use of the pre-existing, well-defined interface and other built-in capabilities provided by the chosen network protocol, and minimize the addition of new application-specific features on top of it.

and another one specific to HTTP [2]:

HTTP, for example, has a very rich vocabulary in terms of verbs (or "methods"), URIs, Internet media types, request and response codes, etc. REST uses these existing features of the HTTP protocol, and thus allows existing layered proxy and gateway components to perform additional functions on the network such as HTTP caching and security enforcement.

Discussion

Jimg 13:48, 10 May 2012 (PDT) I would like to see this broken up into two or more papers where one specifies the proposed HTTP implementation of the requests and responses of DAP4. Others can specify the structure and interpretation of the Capabilities response and how a web service will support multiple versions of DAP, including DAP2 and DAP4 (and variants of those protocols, should we allow that).

Examples

Server Capabilities Document Request-Response

Client uses DAP dataset URL, http://server.org/dap/path/data.nc, to GET the dataset capabilities resource as an XML document:

GET /dap/path/data.nc HTTP/1.1
Host: server.org
Accept: application/vnd.opendap.org.capabilities+xml

The response might look like like:

200 OK
Content-Type: application/vnd.opendap.org.capabilities+xml;charset=UTF-8

<capabilities xmlns="http://opendap.org/ns/dap/capabilities">
  <server rel="http://opendap.org/rel/dap/server"
          type="application/vnd.opendap.org.capabilities+xml,
                application/vnd.opendap.org.capabilities+json,
                application/vnd.opendap.org.capabilities+pbuf"
          href="http://server/dap/capabilities">
    <protocolVersions>
      http://opendap.org/rel/dap2
      http://opendap.org/rel/dap3
      http://opendap.org/rel/dap4
    </protocolVersions>
    <implVersion>TDS 4.3.54</implVersion>
    <supportedExt>...</supportedExt>
  </server>
  <dataset rel="self"
           type="application/vnd.opendap.org.capabilities+xml,
                 application/vnd.opendap.org.capabilities+json,
                 application/vnd.opendap.org.capabilities+pbuf"
           href="http://server/dap/path/data.nc">
    <version name="dap2">

      <serviceLink rel="http://opendap.org/rel/dap2/das"
                   type="application/vnd.opendap.org.dap2.das"
                   href="http://server/some/path/data.nc.das"/>
      <serviceLink rel="http://opendap.org/rel/dap2/dds"
                   type="application/vnd.opendap.org.dap2.dds"
                   href="http://server/dap/path/data.nc.dds"/>
      <serviceLink rel="http://opendap.org/rel/dap2/ddx"
                   type="application/vnd.opendap.org.dap2.ddx+xml"
                   href="http://server/dap/path/data.nc.ddx"/>
      <serviceLink rel="http://opendap.org/rel/dap2/dods"
                   type="application/vnd.opendap.org.dap2.dods"
                   href="http://server/dap/path/data.nc.dods"/>
      <serviceLink rel="http://opendap.org/rel/dap2/dods"
                   type="application/vnd.opendap.org.dap2.dods+ascii"
                   href="http://server/dap/path/data.nc.ascii"/>
      <constraintExp ...>...</constraintExp>
      <processing ...>...</processing>
    </version>
    <version name="dap4">
      <serviceLink rel="http://opendap.org/rel/dap4/description"
                   type="application/vnd.opendap.org.dap4.description+xml,
                         application/vnd.opendap.org.dap4.description+json,
                         application/vnd.opendap.org.dap4.description+pbuf"
                   href="http://server/dap/path/data.nc.ddx4"/>
      <serviceLink rel="http://opendap.org/rel/dap4/data"
                   type="application/vnd.opendap.org.dap4.data,
                         application/vnd.opendap.org.dap4.data+ascii,
                         application/vnd.opendap.org.dap4.data+json,
                         application/vnd.opendap.org.dap4.data+ncstream"
                   href="http://server/dap/path/data.nc.data">
        <constraintExp...>...</constraintExp>
        <processing ...>...</processing>
      </serviceLink>
    </version>
  </dataset>
</capabilities>

Constrained Data Request-Response

Request:

GET /dap/path/data.nc?x,y,temp HTTP/1.1
Host: server.org
Accept: application/vnd.opendap.org.dap4.data

Response:

200 OK
Content-Type: application/vnd.opendap.org.dap4.data
 
<ddx>...</ddx>...BINARY DATA...

Create New Dataset with Constrained Data Request-Response

Request:

POST /dap/path/data.nc HTTP/1.1
Host: server.org
Content-Type: application/vnd.opendap.org.dap4.ce+xml;charset=UTF-8
 
<constraintExp>...</constraintExp>

Response:

201 Created
Content-Type: application/vnd.opendap.org.dap4.capabilities+xml;charset=UTF-8
 
<capabilities xmlns="http://opendap.org/ns/dap/capabilities">
  <server ...>...</server>
  <dataset rel="self"
           type="application/vnd.opendap.org.capabilities+xml,
                 application/vnd.opendap.org.capabilities+json,
                 application/vnd.opendap.org.capabilities+pbuf"
           href="http://server/dap/path/data.nc-ABCD">
    <dap:source>
      <dataset type="application/vnd.opendap.org.capabilities+xml,
                     application/vnd.opendap.org.capabilities+json,
                     application/vnd.opendap.org.capabilities+pbuf"
               href="http://server/dap/path/data.nc" />
      <constraintExp>
        ...
      </constraintExp>
    </dap:source>
    <version name="dap2">
      ...
    </version>
    <version name="dap4">
      ...
    </version>
  </dataset>
</capabilities>

Some References

[1] http://stackoverflow.com/a/385216

  • A stackoverflow answer to the question "Use 'Accept' header or extension to identify resource type?" Answer: the ideal would be to implement both.

[2] http://en.wikipedia.org/wiki/Representational_state_transfer

[3] http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm