DAP4 Web Services v3: Difference between revisions

Revision as of 19:28, 19 June 2012

<<author>>

Background

Problem addressed

Proposed solution

Overview

This page describes the various web services that a DAP4 server must provide. These are for the most part REST services and as such are stateless unless noted otherwise.

The services are all defined as a modification of the service (aka resource or base) URL. This base URL essentially becomes a prefix (and could even be seen as a namespace) for all of the services available for that data resource.

In practice these have traditionally been implemented over HTTP. However they could just as easily be pushed over a different protocol, as long as the usage of the URL components remain consistent.

Media types and HTTP Content Negotiation

The DAP4 Web Services design is intended to make easily available all of the Web Services and their various media-type representations to a wide range of clients.

Media Types

DAP4 web services provide responses in a number of different media-types. Whenever possible DAP4 responses will utilize the most widely understood media-type that can be used to correctly describe the response.

For example, the dataset service document is a DAP4 application specific XML document that provides a syntactic and semantic description of the data resource. The correct media-type for it is text/xml. Since it is truly a DAP4 specific document we considered utilizing the convention described in RFC 3023 XML Media Types:

"This document also standardizes a convention (using the suffix '+xml') for naming media types outside of these five types when those media types represent XML MIME (Multipurpose Internet Mail Extensions) entities."

Which would have yielded a media-type like this: application/vnd.opendap.org.capabilities+xml. In practice, when browsers encounter documents with media types that begin with "application" they simply download the resource to a local file. This is not a behavior that is consistent with the possible uses of XSLT references in the DAP4 XML responses, as we would like browsers to apply the XSLT (if provided) to the returned DAP4 XML documents.

HTTP GET

DAP4 servers are expected to engage in server-driven content negotiation as described in the HTTP 1.1 specification, which means that the client can request a particular media-type representation of a resource (such as application/json) using the HTTP Accept header, and the server will either comply, or it will provide the default media-type representation for the requested resource.

Agent-driven negotiation is also supported in that the DAP Service Terminus will provide links (URIs) for all of the various services and alternate media representations for a particular data resource. This is in keeping with the HTTP 1.1 specification for agent-driven content negotiation which states:

Selection is based on a list of the available representations of the response included within the header fields or entity-body of the initial response, with each representation identified by its own URI.

Where we see the initial response as being the DAP Service Terminus response.

Services

A Service is represented by:

A unique "namespace" like identifier that is used to define an xlink:role attribute.
A simple human readable name called 'title'.
An access URL and a media-type describing the response.
An optional description.

A service is accessed by dereferencing its access URL, which is typically constructed by adding some type of suffix to the dataset's referent (aka base) URL. The way in which the query string (aka constraint expression) is used is defined by each service, and there is no requirement for inter-service query string API conformity.

= Primary DAP4 Services

DAP4: Dataset Service Terminus

The Dataset Description Service provides a listing of the various web services and there associated media-type representations available for a dataset. Dereferencing the base URL for a data set accessed through a DAP4 web server will return a service description document that describes the various responses possible for that data set, including URLs that can be used to access those responses.

More information on the Service Description response can be found on the DAP Service Terminus page.

service url = dataset_url
role id = http://services.opendap.org/dap4/dataset-services#

Default/primary media type:

text/xml

Optional media type(s):

application/json

url = dataset_url.json

DAP4: Dataset Service - The metadata

The Dataset Service provides a metadata description of the dataset. The Dataset response is an XML document that contains both the 'syntactic' (structural) and 'semantic' metadata for the dataset, persisted as a DAP4: Data Model representation of the dataset held at the server. The Dataset service accepts a query string (constraint expression) that allows you to inspect the effects on the data structures when sub-setting and/or server side functions are applied. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set. Note that All dap:Attribute objects have been removed from constrained dap:Dataset objects.

More information on the Dataset response can be found on the DAP4: Responses page.

More information on the syntax of DAP4 constraint expressions can be found on DAP4: Data Model page.

service suffix = .xml
service url = dataset_url + '.xml' + [?dap_constraint]
role id = http://services.opendap.org/dap4/dataset#

Default/primary media type:

text/xml

Optional media type(s):

application/json

url = dataset_url + '.xml' + '.json' + [?dap_constraint]

DAP4: Data Service

The Data Service provides DAP4 data access to a dataset, and is the way that DAP4 returns data to a client. The Data service accepts a query string (constraint expression) which allows you to subset the data and invoke server side functions. When the service is invoked it returns over the wire as a multipart MIME document where the first MIME part contains the constrained Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR, followed by a checksum, for each variable in the dataset.

More information on the Data response can be found on the DAP4: Responses page.

suffix = .dap
service url = dataset_url + '.dap' + [?dap_constraint]
role id = http://services.opendap.org/dap4/data#

Default/primary media type:

multipart/mixed
- First Part: text/xml
- Second Part: application/octet-stream

Optional media type(s):

application/json This Needs Elaboration

url = dataset_url + '.dap' + '.json' + [?dap_constraint]

Other DAP4 Services

DAP4: HTML DATA Request Form Service

The HTML DATA Request Form Service provides browser based access to the Dataset. When invoked it returns a web-browser renderable document (in html) that provides a form (or other UI) that can be used to constrain and request data in accordance with the DAP4 specification as applied to the dataset .

suffix = .html
service url = dataset_url + .html
role id = http://services.opendap.org/dap4/data-request-form#

Default/primary media type: text/html | text/xhtml

DAP4: RDF Service

The RDF service provides an RDF representation of the Dataset document (DDX). The RDF response is an XML document containing an RDF version of the DAP4: Dataset Response.

suffix = .rdf
service url = dataset_url + .rdf
role id = http://services.opendap.org/dap4/rdf#

Default/primary media type: application/rdf+xml

DAP4: ISO 19115 Service

This service provides ISO 19115 metadata for the Dataset, if any can be found. When invoked it returns an XML document containing ISO 19115 metadata located in the DAP4: Dataset Response.

suffix = .iso
service url = dataset_url + .iso
role id = http://services.opendap.org/dap4/iso-19115-metadata#

Default/primary media type: text/xml

DAP4: ISO Conformance Score Service

This service provides a browser renderable document that describes how well the metadata held in the Dataset conforms to ISO 19115. When invoked this service returns a browser renderable document that scores how well the metadata held in the Dataset Response conforms to ISO 19115.

suffix = .rubric
service url = dataset_url + .rubric
role id = http://services.opendap.org/dap4/iso-19115-score#

Default/primary media type: text/html

DAP4: NetCDF File-out Service

This service provides data responses in NetCDF-3 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 3 file.

suffix = .nc
service url = dataset_url + '.nc' + [?dap_constraint]
role id = http://services.opendap.org/dap4/netcdf-3#

Default/primary media type: application/x-netcdf

DAP4: ASCII Data Service

This service provides data responses in ASCII format. When invoked the regular DAP data response will be repackaged as an ASCII representation of the data values.

suffix = .ascii
service url = dataset_url + '.ascii' + [?dap_constraint]
role id = http://services.opendap.org/dap4/ascii#

Default/primary media type: text/plain

DAP4: XML Data Service

This service provides data responses in XML format. When invoked the constrained Dataset response document (DDX) will be marked up with the data values of the request and returned. Large requests may be denied.

suffix = .xdap
service url = dataset_url + '.xdap' + [?dap_constraint]
role id = http://services.opendap.org/dap4/xml-data#

Default/primary media type: text/xml

DAP4: Native File Access Service

This service provides direct access to the data source file (or whatever else) that is creating the DAP dataset resource. When invoked it returns the "native" data from whatever store (filesystem, etc.) it may be in.

suffix = .file
service url = dataset_url + .file
role id = http://services.opendap.org/dap4/file#

Default/primary media type: Type varies with file type.

DAP4: Server Version Service

This service provides software versioning information. When invoked the services returns an XML file containing a description of the version of the server and it's components.

suffix = .ver
service url = dataset_url + .ver
role id = http://services.opendap.org/dap4/version#

Default/primary media type: text/xml

DAP2 Services

DAP2: Data Service

The DAP2 data service provides DAP2 data access to the data resource.

suffix = .dods
service url = dataset_url + '.dods' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dods#

Default/primary media type: application/octet-stream

DAP2: DDX Service

The DAP2 DDX service provides DAP2 access to the data resource metadata. When invoked the service returns an XML document containing both syntactic and semantic dataset metadata in DAP2 XML format.

suffix = .ddx
service url = dataset_url + .ddx
role id = http://services.opendap.org/dap2/ddx#

Default/primary media type: text/xml

DAP2: DDS Service

The DAP2 DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource. When invoked returns a DAP2 DDS response document conforming to the DDS part of the DAP2 specification.

suffix = .dds
service url = dataset_url + '.dds' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dds#

Default/primary media type: text/plain

DAP2: DAS Service

The DAP2 DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource. When invoked returns a DAP2 DAS response document conforming to the DAS part of the DAP2 specification.

suffix = .das
service url = dataset_url + .das
role id = http://services.opendap.org/dap2/das#

Default/primary media type: text/plain

DAP2: Info Service

The DAP2 INFO service provides a browser renderable page that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way. When invoked this service returns a web browser renderable document that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way.

suffix = .info
service url = dataset_url + .info
role id = http://services.opendap.org/dap2/Info#

Default/primary media type: text/html

Rationale for the solution

Discussion

Template: ServiceTemplate

@@ Line 9: / Line 9: @@
 == Proposed solution ==
-== Rationale for the solution ==
+=== Overview ===
-== Discussion ==
-== Overview ==
 This page describes the various web services that a DAP4 server must provide. These are for the most part REST services and as such are stateless unless noted otherwise.
@@ Line 24: / Line 18: @@
-== Media types and HTTP Content Negotiation ==
+=== Media types and HTTP Content Negotiation ===
 The DAP4 Web Services design is intended to make easily available all of the Web Services and their various media-type representations to a wide range of clients.
-=== Media Types ===
+==== Media Types ====
 DAP4 web services provide responses in a number of different media-types. Whenever possible DAP4 responses will utilize the most widely understood media-type that can be used to correctly describe the response.
@@ Line 36: / Line 30: @@
 Which would have yielded a media-type like this: <font size="2"><code>'''application/vnd.opendap.org.capabilities+xml'''</code></font>. In practice, when browsers encounter documents with media types that begin with "''application''" they simply download the resource to a local file. This is not a behavior that is consistent with the possible uses of XSLT references in the DAP4 XML responses, as we would like browsers to apply the XSLT (if provided) to the returned DAP4 XML documents.
-=== HTTP GET ===
+==== HTTP GET ====
 DAP4 servers are expected to engage in [http://www.w3.org/Protocols/rfc2616/rfc2616-sec12.html server-driven content negotiation as described in the HTTP 1.1 specification], which means that the client can request a particular media-type representation of a resource (such as <font size="2"><code>'''application/json'''</code></font>) using the HTTP Accept header, and the server will either comply, or it will provide the default media-type representation for the requested resource.
@@ Line 44: / Line 38: @@
 <!--
-=== HTTP POST ===
+==== HTTP POST ====
 DAP4 supplies (via the DAP Service Terminus) a URI for each DAP4 service and each of its various media-type representations. How do we leverage this into a non-HTTP usage? I think we need to define a message document that is sent to the server (say over a socket) that carries everything we need in the request. Suggestion:
@@ Line 71: / Line 65: @@
 -->
-==  Services  ==
+===  Services  ===
 A Service is represented by:
@@ Line 83: / Line 77: @@
-=== Primary DAP4 Services ===
+==== Primary DAP4 Services ===
-==== [[DAP Service Terminus  | DAP4: Dataset Service Terminus]] ====
+===== [[DAP Service Terminus  | DAP4: Dataset Service Terminus]] =====
 The Dataset Description Service provides a listing of the various web services and there associated media-type representations available for a dataset. Dereferencing the base URL for a data set accessed through a DAP4 web server will return a ''service description document'' that describes the various responses possible for that data set, including URLs that can be used to access those responses.
@@ Line 101: / Line 95: @@
 *<font size="2"><code>'''application/json'''</code></font><br/>
 :url =  <font size="2"><code>''dataset_url'''''.json'''</code></font><br/>
-====  '''[[DAP4: Dataset Service | DAP4: Dataset Service - The metadata]]'''  ====
+=====  '''[[DAP4: Dataset Service | DAP4: Dataset Service - The metadata]]'''  =====
 The Dataset Service provides a metadata description of the dataset. The Dataset response is an XML document that contains both the 'syntactic' (structural) and 'semantic' metadata for the dataset, persisted as a [[DAP4: Data Model]] representation of the dataset held at the server. The Dataset service accepts a query string (constraint expression) that allows you to inspect the effects on the data structures when sub-setting and/or server side functions are applied. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set.
 Note that [[DAP4:_Responses#Transmitting_Attributes_in_constrained_Dataset_documents | All dap:Attribute objects have been removed from constrained dap:Dataset objects.]]
@@ Line 112: / Line 104: @@
 * More information on the syntax of DAP4 constraint expressions can be found on [[DAP4:_Data_Model#Constraint_Expressions | DAP4: Data Model]] page.
@@ Line 126: / Line 117: @@
 :url =  <font size="2"><code>'''dataset_url + '.xml' + '.json' + [?dap_constraint]'''</code></font><br/>
-==== '''[[DAP4: Data Service ]]''' ====
+===== '''[[DAP4: Data Service ]]''' =====
 The Data Service provides DAP4 data access to a dataset, and is the way that DAP4 returns data to a client.  The Data service accepts a query string (constraint expression) which allows you to subset the data and invoke server side functions. When the service is invoked it returns over the wire as a multipart MIME document where the first MIME part contains the ''constrained'' Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR,  followed by a checksum, for each variable in the dataset.
@@ Line 144: / Line 135: @@
 :url =  <font size="2"><code>'''dataset_url + '.dap' + '.json' + [?dap_constraint]'''</code></font><br/>
-=== Other DAP4 Services ===
+==== Other DAP4 Services ====
-==== [[DAP4: HTML DATA Request Form Service]] ====
+===== [[DAP4: HTML DATA Request Form Service]] =====
 The HTML DATA Request Form Service provides browser based access to the Dataset. When invoked it returns a web-browser renderable document (in html) that provides a form (or other UI) that can be used to constrain and request data in accordance with the DAP4 specification as applied to the dataset .
@@ Line 157: / Line 148: @@
 Default/primary media type: <font size="2"><code>'''text/html'''</code></font> | <font size="2"><code>'''text/xhtml'''</code></font><br/>
-==== [[DAP4: RDF Service  | DAP4: RDF Service]] ====
+===== [[DAP4: RDF Service  | DAP4: RDF Service]] =====
 The RDF service provides an RDF representation of the Dataset document (DDX). The RDF response is an XML document containing an RDF version of the [[DAP4: Responses#Dataset_Response | DAP4: Dataset Response.]]
@@ Line 168: / Line 159: @@
 <br/>
-==== [[DAP4: ISO 19115 Service ]] ====
+===== [[DAP4: ISO 19115 Service ]] =====
 This service provides ISO 19115 metadata for the Dataset, if any can be found. When invoked  it returns an XML document containing ISO 19115 metadata located in the  [[DAP4: Responses#Dataset_Response | DAP4: Dataset Response.]]
@@ Line 179: / Line 170: @@
 <br/>
-==== [[DAP4: ISO Conformance Score Service ]] ====
+===== [[DAP4: ISO Conformance Score Service ]] =====
 This service provides a browser renderable document that describes how well the metadata held in the Dataset conforms to ISO 19115. When invoked this service returns a browser renderable document that scores how well the metadata held in the [[DAP4: Responses#Dataset_Response | Dataset Response]] conforms to ISO 19115.
@@ Line 190: / Line 181: @@
 <br/>
-==== [[DAP4: NetCDF File-out Service ]] ====
+===== [[DAP4: NetCDF File-out Service ]] =====
 This service provides data responses in NetCDF-3 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 3 file.
@@ Line 200: / Line 191: @@
 <br/>
-==== [[DAP4: ASCII Data Service]] ====
+===== [[DAP4: ASCII Data Service]] =====
 This service provides data responses in ASCII format. When invoked the regular DAP data response will be repackaged as an ASCII representation of the data values.
@@ Line 211: / Line 202: @@
 <br/>
-==== [[DAP4: XML Data Service]] ====
+===== [[DAP4: XML Data Service]] =====
 This service provides data responses in XML format. When invoked the constrained Dataset response document (DDX) will be marked up with the data values of the request and returned. Large requests may be denied.
@@ Line 222: / Line 213: @@
 <br/>
-==== [[DAP4: Native File Access Service ]] ====
+===== [[DAP4: Native File Access Service ]] =====
 This service provides direct access to the data source file (or whatever else) that is creating the DAP dataset resource. When invoked it returns the "native" data from whatever store (filesystem, etc.) it may be in.
@@ Line 233: / Line 224: @@
 <br/>
-==== [[DAP4: Server Version Service  ]]====
+===== [[DAP4: Server Version Service  ]]=====
 This service provides software versioning information. When invoked the services returns an XML file containing a description of the version of the server and it's components.
@@ Line 244: / Line 235: @@
 <br/>
-=== DAP2 Services ===
+==== DAP2 Services ====
-==== [[DAP2: Data Service]]====
+===== [[DAP2: Data Service]]=====
 The DAP2 data service provides DAP2 data access to the data resource.
@@ Line 257: / Line 248: @@
 <br/>
-==== [[DAP2: DDX Service ]]====
+===== [[DAP2: DDX Service ]]=====
 The DAP2 DDX service provides DAP2 access to the data resource metadata. When invoked the service returns an XML document containing both syntactic and semantic  dataset metadata in DAP2 XML format.
@@ Line 267: / Line 258: @@
 <br/>
-==== [[DAP2: DDS Service ]]====
+===== [[DAP2: DDS Service ]]=====
 The DAP2 DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource. When invoked returns a DAP2 DDS response document conforming to the DDS part of the DAP2 specification.
@@ Line 278: / Line 269: @@
 <br/>
-==== [[DAP2: DAS Service ]] ====
+===== [[DAP2: DAS Service ]] =====
 The DAP2 DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource. When invoked returns a DAP2 DAS response document conforming to the DAS part of the DAP2 specification.
@@ Line 289: / Line 280: @@
 <br/>
-==== [[DAP2: Info Service ]] ====
+===== [[DAP2: Info Service ]] =====
 The DAP2 INFO service provides a browser renderable page that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way. When invoked this service returns a web browser renderable document that combines both the DAP2 'syntactic' and  'semantic' metadata for the data resource in a human readable way.
@@ Line 299: / Line 290: @@
 Default/primary media type: <font size="2"><code>'''text/html'''</code></font><br/>
 <br/>
+== Rationale for the solution ==
+== Discussion ==
 [[Template: ServiceTemplate]]

DAP4 Web Services v3: Difference between revisions

Revision as of 19:28, 19 June 2012

Contents

Background

Problem addressed

Proposed solution

Overview

Media types and HTTP Content Negotiation

Media Types

HTTP GET

Services

= Primary DAP4 Services

DAP4: Dataset Service Terminus

DAP4: Dataset Service - The metadata

DAP4: Data Service

Other DAP4 Services

DAP4: HTML DATA Request Form Service

DAP4: RDF Service

DAP4: ISO 19115 Service

DAP4: ISO Conformance Score Service

DAP4: NetCDF File-out Service

DAP4: ASCII Data Service

DAP4: XML Data Service

DAP4: Native File Access Service

DAP4: Server Version Service

DAP2 Services

DAP2: Data Service

DAP2: DDX Service

DAP2: DDS Service

DAP2: DAS Service

DAP2: Info Service

Rationale for the solution

Discussion

Navigation menu

Page actions

Personal tools

Search

Tools