DAP4 Web Services v3

<< Back to OPULS Development

This page is out of date and will not be updated. This is an old document that captures an intermediate point of the OPULS design work. This page should be referenced only as part of a baseline for the work. It has been superseded by the DAP4 Specification Volume 2: Web Services Specification

Introduction

This document defines the DAP4 Web Service protocol which DAP4-compliant software MUST support when utilizing the HTTP protocol to transmit DAP4 requests and responses, along with additional optional services that DAP4-compliant software SHOULD support.

The DAP4 protocol uses three basic responses to represent a data resource. One response, the Dataset Metadata Response (DMR) contains metadata information describing the structure of the data resource. That is, it characterizes the variables, their datatypes, names and attributes. The second response, the Data Response (DataDMR), contains both the metadata about the requested data and the actual data that was requested. The third basic response is the Dataset Services Response (DSR) that provides a listing of services, any alternate media representations if available, and all of the associated access URI's for a particular data resource.

The DAP4 protocol uses a fourth basic resource, the Constraint/Query Expression (CE), to represent subsetting of and, possibly, transformations of the dataset requested.

The DAP4 protocol uses a fifth basic resource, the Error Response (ER), to allow servers to communicate error information with clients. When a request for any of the three basic responses cannot be completed, an Error response is returned. If an error occurs before the standard response is initiated, an error response is returned in place of the standard response. If an error occurs after a data response has been initiated, an Error Response is returned as the final chunked record as described in Section 7 "DAP4 Chunked Data Representation" of the DAP4 Data Model document [DAP4_Vol1].

The DAP4 Data Model, constraint/query language, dataset metadata XML encoding, and binary data encoding are all defined in the DAP4 Volume 1: "Data Model, Persistent Representations, and Constraints" document [DAP4_Vol1].

Each of the three responses (Dataset Metadata, Data, and Dataset Services) are complete in and of themselves so that, for example, the Data response can be used by a client without ever requesting either of the two other responses. In many cases, client programs will request the Dataset Metadata response before requesting the Data response but there is no requirement they do so and servers SHALL NOT require that behavior on the part of clients.

These three standard dataset responses can each be accessed in two different ways. First, similar to DAP2 URL construction and to support client-driven content negotiation (see the 3.? "Content Negotiation" section below), each dataset resource has a standard URL suffix that can be added to the DAP4 dataset URL to retrieve the resource in its standard (normative) encoding. This allows clients, given a base DAP4 dataset URL, to construct DAP4 URLs in a simple and standard way for each of the three standard dataset resources.

Second, to support server-driven content negotiation (see the 3.? "Content Negotiation" section below) and a more hypermedia-driven style, a DAP4 resource role is defined for each standard dataset resource and a DAP4 media type is defined for each dataset resource encoding scheme.

Any particular instance of a DAP4 server MAY have one or more additional services, alternate media representations of service responses, or unique (to the server instance) server side functions. All of these things, including the core services and their default representations MUST be included in the Dataset Services Response.

The DAP4 web service is currently limited to HTTP GET requests though it is expected that extensions (e.g., asynchronous access) will use other HTTP methods (e.g., POST). This makes, for now, the DAP4 Constraint Expression something of a pseudo-resource type given that they are encoded as part of the URL query string rather than as an independent document.

DAP4 Resource Roles and Media Types

The standard DAP4 dataset resource roles and encodings (plus a few alternate encodings) that are defined in the core DAP4 documents are:

Dataset Services Response (DSR)

Resource Role
http://services.opendap.org/dap4/dataset-service

URL Suffix	Media Type	URL Example
"" or ".dsr"	application/vnd.opendap.org.dap4.dataset-services+xml Normative DSR encoding	http://server/path/dataset.nc http://server/path/dataset.nc.dsr
".xml" or ".dsr.xml"	text/xml Normative DSR encoding with generic media type	http://server/path/dataset.nc.xml http://server/path/dataset.nc.dsr.xml
".html" or ".dsr.html"	text/html HTML DSR encoding	http://server/path/dataset.nc.html http://server/path/dataset.nc.dsr.html

Dataset Metadata Response (DMR)

Resource Role
http://services.opendap.org/dap4/dataset-metadata

URL Suffix	Media Type	URL Example
".dmr"	application/vnd.opendap.org.dap4.dataset-metadata+xml Normative DMR encoding	http://server/path/dataset.nc.dmr
".dmr.xml"	text/xml Normative DMR encoding with generic media type	http://server/path/dataset.nc.dmr.xml
".dmr.html"	text/html HTML DMR encoding	http://server/path/dataset.nc.dsr.html

Dataset Data Response (Data)

Resource Role
http://services.opendap.org/dap4/data

URL Suffix	Media Type	URL Example
".dap"	application/vnd.opendap.org.dap4.data Normative Data encoding	http://server/path/dataset.nc.dap
".dap.txt"	text/plain Text (UTF-8) Data encoding	http://server/path/dataset.nc.dap.txt
".dap.xml"	text/xml XML Data encoding	http://server/path/dataset.nc.dap.xml

Error Response (ER)

Resource Role
http://services.opendap.org/dap4/error

URL Suffix	Media Type	URL Example
N/A	application/vnd.opendap.org.dap4.error+xml Normative Error encoding	N/A
N/A	text/xml Normative Error encoding with generic media type	N/A
N/A	text/html HTML Error encoding	N/A

DAP4 Core Specification Documents

• DAP4 Volume 1: "Data Model, Persistence Representations, and Constraints"
• DAP4 Web Service (this document)
• DAP4 Dataset Services
• DAP4 Requirements for Server-side Functions

Extensions to DAP4 Core Specifications

Several types of extensions can be made to the DAP4 core including:

• New encodings for the core DAP4 response types
• New response types
• New server-side functions.

Terms and Definitions

Dataset Service Response (DSR)

The DAP4 response type that contains a list of DAP (and other) services available for the dataset including any alternate media representations and all the associated access URIs.

Dataset Metadata Response (DMR)

The DAP4 response type that contains metadata information describing the structure of the requested data. The metadata information characterizes the requested data variables including their names, data types, shapes, and attributes.

Dataset Data Response (Data)

The DAP4 response type that contains both the dataset metadata and the binary data for the requested data.

Resource role ID

A URI that identifies the role of a resource, generally provided with a link to allow clients to identify the type of resource the link references. (For instance, an "atom:link" element has an optional "atom:rel" attribute.)

Media Type

A internet media type is a two-part identifier for resource encoding schemes [1]. (E.g., "text/html", "text/plain", "application/octet-stream".)

DAP4 Web Service Responses

The core of the DAP4 Web Service protocol consists of the three standard response types: Dataset Services Response (DSR), Dataset Metadata Response (DMR), and Dataset Data Response (Data). Each dataset served by a DAP4 compliant server MUST provide these three response types.

All of the example requests described below are based on the DAP4 dataset URL:

http://server.org:8080/dap/path/data.nc

DAP4 Dataset Services Response

The DAP4 Dataset Services Response (DSR) provides clients with a listing of all available DAP4 services and all the available encodings for those services as well as all available DAP4 extensions.

Each service (or response type) has a unique resource role (defined in the appropriate specification), each link (alternate representation) for a given service MUST fulfill that same role. This is not always a clear distinction to make. For example, the DMR can be mapped into ISO 19115 metadata. However, IS0 19115 is clearly a different domain.

The Dataset Services Response MUST contain the following information:

• List of DAP versions supported by server
• The implementation version (e.g., "TDS 4.3.57" or "Hyrax 1.7.45")
• List of all available DAP4 services for the dataset
• For each DAP4 services listed, a list of all available links each with its corresponding media type
• List of supported extensions
  • Resource type extensions
  • Media type extensions
  • Server-side function extensions

If SHOULD contain the following information:

• A human readable title for the dataset
• Human readable titles for each service

To take advantage of web caching, servers should try to keep DSRs light weight (i.e., quick creation) and as stable as possible.

DSR Resource Role

DSRs are identified by the resource role:

http://services.opendap.org/dap4/dataset-services

Normative Encoding of the DSR

The normative XML representation for the Dataset Services Response is defined in the "Normative XML Encoding of the DSR" appendix. The media type for the normative XML representation is

application/vnd.opendap.org.dataset-services+xml

Service Behavior

When an HTTP GET request is made on a base DAP4 dataset URL, all DAP4 servers MUST return the normative XML encoding of the DSR given these conditions:

• the request "Accept" header contains only the normative XML encoding media type,
• the request "Accept" header equals "*/*", or
• the request "Accept" header does not indicate a preference for another media type in which the server knows how to encode the DSR.

For example, the request:

GET /dap/path/data.nc HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: application/vnd.opendap.org.dataset-services+xml
Date: ...

Downgrading the Normative XML Encoding (Required)

When an HTTP GET request is made on a base DAP dataset URL with the suffix .xml
added to it:

request url = dataset_url.xml

the response MUST be the normative representation of the DSR along with the HTTP Content-Type header set to text/xml . For example:

GET /dap/path/data.nc.xml HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/xml; charset=utf-8
Date: ...

The normative XML representation MUST also be returned when an HTTP GET request is made on a base DAP4 dataset URL (without a suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an HTML encoded DSR. For example:

GET /dap/path/data.nc HTTP/1.1
Host: server.org:8080
Accept: text/xml

Other Encodings of the DSR

HTML Encoding (Optional)

When an HTTP GET request is made on a base DAP dataset URL with the suffix .html
added to it:

request url = dataset_url.html

the server MUST reply with an HTML representation of the DSR, or return an HTTP status of 404 to indicate that an HTML representation of the DSR is not available. For example:

GET /dap/path/data.nc.html HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Date: ...

If available, the HTML representation MUST also be returned when an HTTP GET request is made on a base DAP4 dataset URL (without a suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an HTML encoded DSR. For example this request:

GET /dap/path/data.nc HTTP/1.1
Host: server.org:8080
Accept: text/html

Must return the HTML representation of the DMR, if available. If no such representation is available then the server MAY return an HTTP status of 404 or even 415.

Dataset Metadata Response

The Dataset Metadata Service returns the Dataset Metadata Response (DMR) which is a metadata description of the dataset. The normative representation of the DMR is an XML document that contains both the 'syntactic' (structural) and 'semantic' metadata for the dataset, persisted as a DAP4: Data Model representation of the dataset held at the server. The DMR service accepts a query string (constraint expression) that allows you to inspect the effects on the data structures when sub-setting and/or server side functions are applied. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set. Note that All dap:Attribute objects have been removed from constrained dap:Dataset objects. More information on the syntax of DAP4 constraint expressions can be found on DAP4: Data Model page.

DMR Resource Role

DMRs are identified by the resource role:

http://services.opendap.org/dap4/dataset-metadata

Normative Encoding of the DSR

The normative XML representation for the Dataset Metadata Response is defined in the DAP4: Responses page. The media type for the normative XML representation is:

application/vnd.org.opendap.dap4.dataset-metadata+xml

(More information on the syntax of DAP4 constraint expressions can be found on DAP4: Data Model page. )

Service Behavior

Every DAP4 compliant server MUST return the normative representation (an XML document as described here ) of the Dataset Metadata Response when a client attempts to access a dataset URL with the suffix ".dmr" appended to it. The DAP4 server MAY return alternate representations if the client indicates that it can accept them and the server can provide them.

When an HTTP GET request is made on a base DAP dataset URL with the suffix .dmr added to it:

request url = dataset_url + '.dmr'+ [?dap_constraint]

the server MUST reply with an normative representation of the DMR for the (possibly constrained) dataset given these conditions:

• the request "Accept" header contains only the normative XML encoding media type (application/vnd.org.opendap.dap4.dataset-metadata+xml),
• the request "Accept" header equals "*/*", or
• the request "Accept" header does not indicate a preference for another media type in which the server knows how to encode the DMR.

For example, the request:

GET /dap/path/data.nc.dmr HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: application/vnd.opendap.org.dataset-metadata+xml
Date: ...

Downgrading the Normative XML Encoding (Required)

While the normative representation of the the Dataset Metadata response is already an XML document, the normative media type is application/vnd.opendap.org.dataset-metadata+xml which may be unfamiliar to many generic clients (such as web browsers) and it is quite conceivable that such a client might ask for the more generic text/xml media type.

When an HTTP GET request is made on a DAP DMR URL with the suffix .xml
added to it:

request url = dataset_url.dmr.xml

the response MUST be the normative representation of the DMR along with the HTTP Content-Type header set to text/xml . For example:

GET /dap/path/data.nc.dmr.xml HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/xml; charset=utf-8
Date: ...

The normative XML representation of the DMR MUST also be returned when an HTTP GET request is made on a base DAP4 DMR URL ( and the server uses server-driven content negotiation to decide that the best response for the client would be an HTML encoded DSR. For example:

GET /dap/path/data.nc.dmr HTTP/1.1
Host: server.org:8080
Accept: text/xml

Other Encodings of the DMR

HTML Encoding (Optional)

When an HTTP GET request is made on a DAP DMR URL with the suffix .html
added to it:

request url = dataset_url.dmr.html

the server MUST reply with an HTML representation of the DMR, or return an HTTP status of 404 (or 415) to indicate that an HTML representation of the DMR is not available. For example:

GET /dap/path/data.nc.dmr.html HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Date: ...

If available, the HTML representation MUST also be returned when an HTTP GET request is made on a base DAP4 DMR URL (without an additional suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an HTML encoded DMR. For example this request:

GET /dap/path/data.nc.dmr HTTP/1.1
Host: server.org:8080
Accept: text/html

Must return the HTML representation of the DMR, if available. If no such representation is available then the server MAY return an HTTP status of 404 or even 415.

DAP4: Data Response

The Data Service provides DAP4 data access to a dataset, and is the (primary) way that DAP4 returns data to a client. The Data service accepts a query string (constraint expression) which allows you to subset the data and invoke server side functions. When the service is invoked it returns the DAP4 data object. On the wire this is a binary document with MIME type application/vnd.opendap.org.dap4.data. The payload is broken into two logical parts: A DMR-type xml document that describes the data and a BLOB that contains the actual data. For more on the internal structure of the Data response's payload, see DAP4: Encoding for the Data Response and DAP4: Chunked encoding.

More information on the Data response can be found on the DAP4: Responses page.

Data Response Resource Role

DAP4 Data Responses are identified by the resource role:

http://services.opendap.org/dap4/data

Normative Encoding of the Data Response

The normative XML representation for the Data Response is defined in the DAP4: Responses page. The media type for the normative XML representation is:

application/vnd.org.opendap.dap4.data

(More information on the syntax of DAP4 constraint expressions can be found on DAP4: Data Model page. )

Service Behavior

Every DAP4 compliant server MUST return the normative representation ( as described here ) of the Data Response when a client attempts to access a dataset URL with the suffix ".dap" appended to it. The DAP4 server MAY return alternate representations if the client indicates that it can accept them and the server can provide them.

When an HTTP GET request is made on a base DAP dataset URL with the suffix .dap added to it:

request url = dataset_url + '.dap'+ [?dap_constraint]

the server MUST reply with an normative representation of the (possibly constrained) data response for the dataset given these conditions:

• the request "Accept" header contains only the normative XML encoding media type (application/vnd.org.opendap.dap4.data),
• the request "Accept" header equals "*/*", or
• the request "Accept" header does not indicate a preference for another media type in which the server knows how to encode the Data Response.

For example, the request:

GET /dap/path/data.nc.dap HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: application/vnd.opendap.org.data
Date: ...

Other Encodings of the Data Response

ASCII Encoding (Optional)

When an HTTP GET request is made on a DAP Data Response URL with the suffix .ascii
added to it:

request url =  dataset_url.dap.ascii

the server MUST reply with the ASCII representation of the Data Response, or return an HTTP status of 404 (or 415) to indicate that an ASCII representation of the Data Response is not available. For example:

GET /dap/path/data.nc.dap.ascii HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: ...

If available, the ASCII representation MUST also be returned when an HTTP GET request is made on a base DAP4 Data Response URL (without an additional suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an ASCII encoded Data Response. For example this request:

GET /dap/path/data.nc.dap HTTP/1.1
Host: server.org:8080
Accept: text/plain

Must return the ASCII representation of the Data Response, if available. If no such representation is available then the server MAY return an HTTP status of 404 or even 415.

XML Encoding (Optional)

When an HTTP GET request is made on a DAP Data Response URL with the suffix .xml
added to it:

request url =  dataset_url.dap.xml

the server MUST reply with the XML representation of the Data Response, or return an HTTP status of 404 (or 415) to indicate that an XML representation of the Data Response is not available. For example:

GET /dap/path/data.nc.dap.xml HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: text/xml; charset=utf-8
Date: ...

If available, the XML representation MUST also be returned when an HTTP GET request is made on a base DAP4 Data Response URL (without an additional suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an XML encoded data response. For example this request:

GET /dap/path/data.nc.dap HTTP/1.1
Host: server.org:8080
Accept: text/xml

Must return the XML representation of the Data Response, if available. If no such representation is available then the server MAY return an HTTP status of 404 or even 415.

NetCDF Encoding (Optional)

When an HTTP GET request is made on a DAP Data Response URL with the suffix .nc
added to it:

request url =  dataset_url.dap.nc

the server MUST reply with the XML representation of the Data Response, or return an HTTP status of 404 (or 415) to indicate that an XML representation of the Data Response is not available. For example:

GET /dap/path/data.nc.dap.nc HTTP/1.1
Host: server.org:8080
Accept: */*

Might result in the following response:

HTTP/1.1 200 OK
Content-Type: application/x-netcdf
Date: ...

If available, the NetCDF representation MUST also be returned when an HTTP GET request is made on a base DAP4 Data Response URL (without an additional suffix) and the server uses server-driven content negotiation to decide that the best response for the client would be an NetCDF encoded data response. For example this request:

GET /dap/path/data.nc.dap HTTP/1.1
Host: server.org:8080
Accept: application/x-netcdf

Must return the NetCDF representation of the Data Response, if available. If no such representation is available then the server MAY return an HTTP status of 404 or even 415.

DAP4 Asynchronous Response

Some servers may not be able to fulfill DAP4 service requests in a timely manner.

• A DAP4 server that is retrieving data content from a near-line tape storage subsystem might take several minutes to access a particular data holding.
• A DAP4 server that is providing access to data held in an Amazon Web Services Glacier Vault will have to wait ~4 hours before it can retrieve a particular holding.

In these circumstances the server may return the DAP4 Asynchronous Response.

Asynchronous Response Resource Role

DAP4 Asynchronous Responses are identified by the resource role:

http://services.opendap.org/dap4/async

Normative Encoding of the Asynchronous Response

The normative XML representation for the Asynchronous Response is defined in Appendix x "Normative XML Encoding of the Asynchronous Response". The media type for the normative XML representation is:

application/vnd.org.opendap.dap4.async.xml

DAP4 Error Response

The DAP4 protocol returns error information using an Error response. If a request for any of the three basic responses cannot be completed then an Error response is returned in its place.

The normative XML representation for the Error Response is defined by the following RELAX-NG schema.

<grammar xmlns="http://relaxng.org/ns/structure/1.0"
                xmlns:doc="http://www.example.com/annotation"
                datatypeLibrary="http://xml.opendap.org/datatypes/dap4"
                ns="http://xml.opendap.org/ns/DAP/4.0#"
                >
<start>
  <ref name="errorresponse"/>
</start>
<define name="errorresponse">
  <element name="Error">
    <optional>
      <attribute name="httpcode"><data type="dap4_integer"/></attribute>
    </optional>
    <optional>
      <interleave>
        <element name = "Message"><text/></Message>
        <element name = "Context"><text/></Message>
        <element name = "OtherInformation"><text/></Message>
      </interleave>
    </optional>
  </element>
</define>

The Error element has one optional attribute: the httpcode which is a standard HTTP protocol return code indicating the general class of error. When possible, this code should match the return code in the HTTP headers for the response.

The body of the <Error> element may contain any or all of the following inner elements each containing arbitrary text.

1. <Message> — A short informative message describing the error.
2. <Context> — Information describing the context in which the error occurred: position of a parse error in a constraint expression, for example.
3. <OtherInformation> — Arbitrary additional text information: a Java stack trace, for example.

Error Response Resource Role

DAP4 Error Responses are identified by the resource role:

http://services.opendap.org/dap4/error

Normative Encoding of the Error Response

The normative XML representation for the Error Response is defined in Appendix x "Normative XML Encoding of the Error Response". The media type for the normative XML representation is:

application/vnd.org.opendap.dap4.error.xml

HTTP

The DAP4 Web Services specification describes the features of HTTP that are required to be a compliant DAP4 client or server. It does not attempt to describe all aspects of HTTP that DAP4 servers might implement or that DAP4 clients may see in response to DAP4 requests. Similarly, it does not cover all issues related to the performance and scalability of HTTP.

However, the following sections include both DAP4 requirements as well as some suggestions of HTTP features that servers and clients are encouraged to use.

Content Negotiation and Media Types

Though the DAP4 core specifications only describe one encoding for each type of resource, DAP4 web servers MAY have the ability to provide a given resource in a number of different media types. All media types available for a resource MUST be listed in the DAP4 Dataset Services response document.

DAP4 responses MUST use the "Content-Type" header field to identify the media type of the DAP4 response body. For example, the normative value for the XML encoded DDX response is application/vnd.org.opendap.ddx+xml.

The DAP4 Dataset Services response describes the available services and their media types and provides DAP4 software (client and/or server) with two different mechanisms to negotiate for different kinds of media representations. The first of these is server-driven content negotiation as described in the HTTP 1.1 specification, section 12 "Content Negotiation". The second is similar to the agent-driven negotiation also described in section 12 of the HTTP 1.1 specification. The difference being that the "list of available representations ... [each with] its own URI" is provided by the DAP4 Dataset Services response rather than in response to an initial request.

Clients need not retrieve the Dataset Services response in order to access the normative representations of either the Dataset Metadata or Data responses, as these responses are required for every DAP4 server and are mapped to well known URL patterns. If clients attempt to access other representations or other services using agent-driven negotiation without first checking the Dataset Services response, they should be prepared to receive a 404 Not Found response.

When using server-driven negotiation, DAP4 clients are encouraged to, at a minimum, include "Accept" and "User-Agent" headers in their requests and to provide accurate and detailed information in the values of those headers. However, when server-driven negotiation is used, DAP4 servers are not limited to those headers for determining the media type that is returned. DAP4 clients should also be prepared to handle 415 "Unsupported Media type" response codes.

Redirects

While HTTP redirects are not directly part of the DAP4 web protocol it is strongly suggested that DAP4 client implementations be capable of processing HTTP redirects as nominally described in the HTTP specification sections on redirection status codes and response headers (It is suggested that implementers of DAP4 clients utilize an existing robust HTTP client library that will manage this for them.)

Caching of Responses and Conditional Requests

While, HTTP caching and conditional requests are not explicitly part of the DAP4 web protocol, they do provide important tools for improving the performance of both sides client server interaction. Therefore, it is strongly suggested that DAP4 servers and client implementers be aware of how HTTP caching works, and utilize it for working with DAP servers.

Authentication/Authorization

Authentication is the process by which a user agent establishes the identity of the user to a server, and the server establishes it's identity with the user agent. While, HTTP authentication is not explicitly part of the DAP4 web protocol, it does provide important tools for securing the client server interaction. Therefore, it is strongly suggested that DAP4 servers and client implementers be aware of how HTTP authentication works, and utilize it for working with DAP servers. Server implementers should pay particular attention to HTTP security considerations.

Authorization is the process through which a server determines who/what has access to its various holdings and services. The HTTP protocol does not directly address the issue of authorization (even though HTTP defines the 401 response status using the word "authorization" it does not provide separate semantics for authentication and authorization which we see as an important distinction for data access), but any DAP4 server implementer should be aware that some kind of mechanism for controlling access to holdings and services will likely be desired by the people that install and operate their software.

Headers

Request Headers

These are the HTTP request headers that DAP clients using HTTP MAY utilize. DAP4 servers MUST accept these headers and act on them in a manner consistent with their descriptions below.

General

Accept

The HTTP Accept header MAY be used by clients that wish to engage in server-dirven content negotiation by requesting a particular representation of a resource in the initial request. The server MUST utilize this header, if present, in a manner consistent with the HTTP content negotiation specification.

User-Agent

The HTTP User-Agent header MAY be used by clients to indicate their "software identity" to the server. The server MAY utilize this header, if present, to alter the Content-Type of the response to something that is more likely to be digestible by the requesting client software.

DAP Specific

There are no DAP specific headers required to make a general DAP request.

Asynchronous Transaction

X-DAP-Async-Accept

A client MAY indicate it's willingness to accept asynchronous responses by including the X-DAP-Async-Accept HTTP header. Clients MAY make conditional requests for asynchronous responses by indicating the maximum time they are willing to wait by using the X-DAP-Async-Accept HTTP header with a value given in seconds. A value of zero indicates that the client is willing to accept whatever delay the server may encounter.

Response Headers

These are the HTTP response headers that DAP servers using HTTP MUST and SHOULD (as indicated) return.

General

Date

DAP4 servers MUST return an HTTP Date header whose value is the time of the response and which MUST be in RFC 1123 format.

Last-Modified

DAP4 servers SHOULD return an HTTP Last-Modified header whose value is the last modified time of the request resource and which MUST be in RFC 1123 format.

Content-Type

DAP4 servers MUST return an HTTP Content-Type header, the value of which SHOULD be set in accordance with the Media Types discussion above.

Content-Description

DAP4 servers SHOULD return an HTTP Content-Description header.

Content-Disposition

DAP4 servers SHOULD return an HTTP Content-Disposition header when transmitting file typed responses.

Content-Encoding

DAP4 servers MUST return an HTTP Content-Encoding header when the content-coding of an entity is not "identity".

DAP Specific

X-DAP-Server

DAP4 servers SHOULD return the X-DAP-Server HTTP header. This HTTP header is used in a response to communicate the software version of the server. This may be a simple string with the server name and version number, or multiple software component versions may be represented. The value of this header is a string determined by the implementations author(s).

Example

X-DAP-Server: bes/3.10.0, libdap/3.11.2, dap-server/ascii/4.1.2, csv_handler/1.0.2, freeform_handler/3.8.4, fits_handler/1.0.7, fileout_netcdf/1.1.2, gateway_module/1.1.0, hdf4_handler/3.9.4, hdf5_handler/1.5.0, netcdf_handler/3.10.0, ncml_module/1.2.1, dap-server/usage/4.1.2, dap-server/www/4.1.2, xml_data_handler/1.0.1

Example

X-DAP-Server: TDS-4.19.3

X-DAP

DAP4 servers MUST return the X-DAP HTTP header. This HTTP header is used in a response to indicate the DAP protocol version used to encode the content of the response. This value is constrained to a format of "major_version" dot "minor version", where both major_version and minor_version are represented by an integer value.

Example

X-DAP: 4.0

Example

X-DAP: 2.17

Asynchronous Transaction

X-DAP-Async-Required

DAP4 servers MUST return the X-DAP-Async-Required header if the request requires an asynchronous response and the client has not indicated willingness to accept such a response. Rejection of the request should also be indicated by the 400 DAP Asynchronous Response Required HTTP response code.

Status Codes

DAP servers that provide an HTTP interface are expected to utilize the HTTP response codes in a manner consistent with the HTTP 1.1 specification.

The ones that are detailed here are used by the DAP in a manner consistent with the specifications definition, but in support of specific DAP server behavior.

200 OK

A server MUST return an HTTP status of 200 when the request has been successful and that the returned document contains the requested resource. However since DAP responses can be quite large and since it is also possible for the server to encounter any number of problems during the marshaling, serialization, and subsequent transmission of the response it is possible that the server may have committed/transmitted the HTTP headers (in which the status value is transmitted) before a subsequent error is encountered. These types of errors are transmitted in the DAP4 over-the-wire protocol and all DAP4 clients MUST be able to read and process these errors. See Section?

202 Accepted

A server MUST return a 202 Accepted HTTP response code when a request has been accepted and will be handled asynchronously. The response body MUST contain a DAP asynchronous response accepted document.

400 Bad Request

The HTTP specification defines this status code as:

The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.

DAP servers utilizes this code to mean one of two things.

400 Bad DAP4 Request Syntax

The 400 Bad DAP4 Request Syntax HTTP response code MUST be returned by the server when there is a problem with the syntax of the OPeNDAP URL. The problem could be in the formulation of the constraint expression, or it may be that the URL extension did not match any that are recognized by this server.

400 DAP4 Asynchronous Response Required

The 400 DAP4 Asynchronous Response Required HTTP response code MUST be returned by the server when the DAP request has been rejected because an asynchronous response is required and the client did not indicate willingness to accept an asynchronous response. The server MUST include the X-DAP-Async-Required HTTP header and the DAP4 Asynchronous Response Required document as the response body.

401 Unauthorized

The 401 Unauthorized HTTP response code MUST be returned by the server when access to the requested resource requires (not previously acquired) user authentication. See the HTTP specification for usage and behavior.

403 Forbidden

The 403 Forbidden HTTP response code MUST be returned when the server has understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. This is appropriate to return if, for example, the server software does not have read permission for the requested resource.

404 Not Found

The 404 Not Found HTTP response code MUST be returned when the server has not found anything matching the Request-URI.

409 Conflict (DAP4 Async Response Not Available)

The 409 Conflict HTTP response code SHOULD be returned by the server when the DAP request has been rejected because a previous asynchronous request has not been completed and the result is not ready for access. The response body MUST contain the DAP4 Asynchronous Response Not Yet Available document.

412 Precondition Failed

The 412 Precondition Failed HTTP response code MUST be returned by the server if the DAP request has been rejected because it did not meet the X-DAP-Async-Accept condition that was specified in the request. The response body MUST contain the DAP4 Asynchronous Request Rejected document.

415 Unsupported Media Type

The 415 Unsupported Media Type HTTP response code MUST be returned when the client requests a representation of the requested resource that the server cannot provide.

500 Internal Server Error

The 500 Internal Server Error SHOULD be returned when the DAP server encounters internal problems when attempting to fulfill a request.

Verbs (aka Methods)

GET

A DAP request may be made using the HTTP GET request method utilizing a Uniform Resource Identifier (URI) that encodes information specific to the DAP (see Section ?).

Each GET request MUST conform to the HTTP specification (which basically says that a GET request MUST contain an HTTP protocol version number followed by a MIME-like message containing various headers that further describe the request.). While there are some optional DAP HTTP request headers that may be used, DAP requests do not require specific HTTP headers beyond those necessary for HTTP. DAP servers SHOULD support the use of the HTTP Accept request header to allow clients to engage in HTTP content negotiation for specific representations of a DAP response.

Implementation Note

In practice, DAP clients typically use a third-party library implementation of HTTP/1.1 so the GET request, URI and HTTP version information are hidden from the client; it sees only the DAP Uniform Resource Locator (URL) and the request headers.

The DAP server responds to the GET request with an HTTP compliant response (one that includes a status line containing the HTTP protocol version and an error or success code, followed by HTTP response headers and then response itself). There are two DAP specific HTTP headers that are always included in a DAP response over HTTP: X-DAP-Server and X-DAP, as described above. The DAP response is the payload of the HTTP response. Unless otherwise negotiated, the data response payload is encoded in multpart-MIME format, while the other responses are simple HTTP responses This is further described in Section ?.

Examples

HTTP GET request.

GET /dap/path/data.nc.dap?u,v[0:8:1024] HTTP/1.1
Host: server.org
Accept: application/vnd.org.opendap.dap4.data

Normative References

[DAP4_Vol1] DAP4 Volume 1: "Data Model, Persistent Representations, and Constraints".

[HTTP_RFC2616] "Hypertext Transfer Protocol -- HTTP/1.1" (as text or HTML).

Non-Normative References

[REST] R. Fielding, UC Irvine Doctoral Thesis: "Architectural Styles and the Design of Network-based Software Architectures"

Extra Web Service Stuff

Other Common Services

DAPx servers typically offer a number of other services that, while not exactly DAP services, are commonly available. This section lists just some of the alternate services a DAP4 server might provide. By enumerating the available services in the Dataset Services Response servers can easily make clients (either software and monkey) aware of their ancillary capabilities.

DAP4: HTML DATA Request Form Service

The HTML DATA Request Form Service provides browser based access to the Dataset. When invoked it returns a web-browser renderable document (in html) that provides a form (or other UI) that can be used to constrain and request data in accordance with the DAP4 specification as applied to the dataset .

suffix = .html
service url = dataset_url + .html
role id = http://services.opendap.org/dap4/data-request-form#

Default/primary media type: text/html | text/xhtml

DAP4: RDF Service

The RDF service provides an RDF representation of the Dataset document (DDX). The RDF response is an XML document containing an RDF version of the DAP4: Dataset Response.

suffix = .rdf
service url = dataset_url + .rdf
role id = http://services.opendap.org/dap4/rdf#

Default/primary media type: application/rdf+xml

DAP4: ISO 19115 Service

This service provides ISO 19115 metadata for the Dataset, if any can be found. When invoked it returns an XML document containing ISO 19115 metadata located in the DAP4: Dataset Response.

suffix = .iso
service url = dataset_url + .iso
role id = http://services.opendap.org/dap4/iso-19115-metadata#

Default/primary media type: text/xml

DAP4: ISO Conformance Score Service

This service provides a browser renderable document that describes how well the metadata held in the Dataset conforms to ISO 19115. When invoked this service returns a browser renderable document that scores how well the metadata held in the Dataset Response conforms to ISO 19115.

suffix = .rubric
service url = dataset_url + .rubric
role id = http://services.opendap.org/dap4/iso-19115-score#

Default/primary media type: text/html

DAP4: NetCDF File-out Service

This service provides data responses in NetCDF-4 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 4 file.

suffix = .nc4
service url = dataset_url + '.nc4' + [?dap_constraint]
role id = http://services.opendap.org/dap4/netcdf-3#

Default/primary media type: application/x-netcdf-4

DAP4: ASCII Data Service

This service provides data responses in ASCII format. When invoked the regular DAP data response will be repackaged as an ASCII representation of the data values.

suffix = .ascii
service url = dataset_url + '.ascii' + [?dap_constraint]
role id = http://services.opendap.org/dap4/ascii#

Default/primary media type: text/plain

DAP4: XML Data Service

This service provides data responses in XML format. When invoked the constrained Dataset response document (DDX) will be marked up with the data values of the request and returned. Large requests may be denied.

suffix = .xdap
service url = dataset_url + '.xdap' + [?dap_constraint]
role id = http://services.opendap.org/dap4/xml-data#

Default/primary media type: text/xml

DAP4: Native File Access Service

This service provides direct access to the data source file (or whatever else) that is creating the DAP dataset resource. When invoked it returns the "native" data from whatever store (filesystem, etc.) it may be in. Servers MAY elect to not support this response, for example, for generated data, very large data, et cetera.

suffix = .file
service url = dataset_url + .file
role id = http://services.opendap.org/dap4/file#

Default/primary media type: Type varies with file type.

DAP4: Server Version Service

This service provides software versioning information. When invoked the service returns an XML file containing a description of the version of the server and it's components.

suffix = .ver
service url = dataset_url + .ver
role id = http://services.opendap.org/dap4/version#

Default/primary media type: text/xml

DAP2 Services

In order to support legacy client applications DAP4 server implementations MAY support the DAP2 services stack. If they do so the DAP2 services MUST be organized as described in this section.

DAP2: Data Service

The DAP2 data service provides DAP2 data access to the data resource.

suffix = .dods
service url = dataset_url + '.dods' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dods#

Default/primary media type: application/octet-stream

DAP2: DDX Service

The DAP2 DDX service provides DAP2 access to the data resource metadata. When invoked the service returns an XML document containing both syntactic and semantic dataset metadata in DAP2 XML format.

suffix = .ddx
service url = dataset_url + .ddx
role id = http://services.opendap.org/dap2/ddx#

Default/primary media type: text/xml

DAP2: DDS Service

The DAP2 DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource. When invoked returns a DAP2 DDS response document conforming to the DDS part of the DAP2 specification.

suffix = .dds
service url = dataset_url + '.dds' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dds#

Default/primary media type: text/plain

DAP2: DAS Service

The DAP2 DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource. When invoked returns a DAP2 DAS response document conforming to the DAS part of the DAP2 specification.

suffix = .das
service url = dataset_url + .das
role id = http://services.opendap.org/dap2/das#

Default/primary media type: text/plain

DAP2: Info Service

The DAP2 INFO service provides a browser renderable page that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way. When invoked this service returns a web browser renderable document that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way.

suffix = .info
service url = dataset_url + .info
role id = http://services.opendap.org/dap2/Info#

Default/primary media type: text/html

DAP2: NetCDF Service

This service provides data responses in NetCDF-3 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 3 file.

suffix = .nc
service url = dataset_url + '.nc' + [?dap_constraint]
role id = http://services.opendap.org/dap4/netcdf-3#

Default/primary media type: application/x-netcdf

Template: ServiceTemplate