DAP4: Specification Volume 1

<< Back to OPULS Development

The Data Access Protocol: DAP Version 4.0

Volume 1: Data Model and Serialized Representation

Date:	May 31, 2012
Last Revised:	22 November 2013
Status:	Draft
Authors:	John Caron (Unidata)
	Ethan Davis (Unidata)
	David Fulker (OPeNDAP)
	James Gallagher (OPeNDAP)
	Dennis Heimbigner (Unidata)
	Nathan Potter (OPeNDAP)
Copyright:	2013 University Corporation for Atmospheric Research and Opendap.org

Abstract

This document defines the Data Access Protocol (DAP) version 4.0 (referred to also as DAP4). This data transmission protocol is intended to supersede all previous versions of the DAP protocol. DAP4 is designed specifically for science data. The protocol relies on the widely used and stable standards, and is capable of representing a wide variety of scientific data types.

Distribution of this document is unlimited.

This document takes material from the DAP2 specification and the OPULS Wiki page.

Change List

2012.05.24:	Initial Draft
2012.05.27	Added specification of chunk order
2012.05.28	Added specification and interpretation of simple queries
2012.05.28	Added discussion about nested sequences.
2012.05.29	Formatting changes
2012.6.05	Removed serialized representation sections and constraint sections until James provides direction.
2012.6.24	Merge all changes from Gallagher, Potter, and Caron, except as noted.
2012.6.24	Removed all references to Sequences.
2012.6.24	Inserted James' version of serialized representation.
2012.6.25	Added DMR RELAX-NG Grammar.
2012.6.24	Added (semi-)formal description of the DAP4 serialization scheme.
2012.6.26	Added: (1) Revised Char type (2) Revised unlimited dimension rules (3) revised MAP rules. (4) Removed HTTP references
2012.7.09	Added discussion of identifier
2012.7.10	Added discussion of XML escaping
2012.7.10	Fix discrepancies between the formal definition of the on-th-wire format and the examples.
2012.7.12	Removed UByte and made Byte == UInt8
2012.8.21	Added draft constraints section
2012.8.25	Improved the discussion of named slices in constraints.
2012.9.4	Minor change to the grammar for simple constraints.
2012.9.6	Updated the Data Response section so that it no longer mentions Multipart MIME; edited the sections on FQNs and Attributes. I've added ‘nested attributes' back into the text. I also added ‘Sequence' in several places where we will need it once we've worked out how those are to be handled.
2012.11.1	Integrate Jame's changes with recent changes
2012.11.9	Rebuild the .docx because of repeated Word crashes; minor formatting info changed/lost.
2012.11.23	Add a Dataset construct to make the root group concept clear syntactically.
2013.3.8	Made unlimited into a boolean attribute because it does have a size.
2013.4.7	Inserted the new checksum description.
2013.4.15	Removed all mention of unlimited wrt Dimensions
2013.4.15	Remove the base and ns attributes from <Dataset>
2013.4.15	Introduce <Sequence> as a replacement for variable length dimensions; The term Sequence is subject to future change.
2013.10.14	Clarify the maximum number of elements as a function of the maximum number of bytes.
2013.10.14	Enforce a specific order on declarations in a Group body.
2013.11.21	Dennis made changes
2013.11.22	Added sections for DSR, Async, and Error responses and their schemas

Introduction

This specification defines the protocol referred to as the Data Access Protocol, version 4.0 ("DAP4"). In this document 'DAP' refers to DAP4 unless otherwise noted.

DAP is intended to be the successor to all previous versions of the DAP (specifically DAP version 2.0). The goal is to provide a very general data model capable of representing a wide variety of existing data sets.

The DAP builds upon a number of existing data representation schemes. Specifically, it is influenced by CDM^[1], HDF5 ^[2], DAP version 2.0^[3], and netCDF-4^[5].

The DAP is a protocol for access to data organized as variables. It is particularly suited to accesses by a client computer to data stored on remote (server) computers that are networked to the client computer. DAP was designed to hide the implementation of different collections of data. The assumption is that a wide variety of data sets using a wide variety of data schemas can be translated into the DAP protocol for transmission from the server holding that dataset to a client computer for processing.

It is important to stress the discipline neutrality of the DAP and the relationship between this and adoption of the DAP in disciplines other than the Earth sciences. Because the DAP is agnostic as relates to discipline, it can be used across the very broad range of data types encountered in oceanography - biological, chemical, physical and geological. There is nothing that constrains the use of the DAP to the Earth sciences.

Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY" and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. ^[7]

Overall Operation

The DAP is a stateless protocol that governs clients making requests from servers, and servers issuing responses to those requests. This section provides an overview of the requests and responses (i.e. the messages) that DAP-compliant software MUST support. These messages are used to request information about a server and data made accessible by that server, as well as requesting data values themselves.

For every data resource the DAP defines a number of responses that may elicited by a client. These response provide services information (i.e. capabilities), structural/semantic descriptions, data access timing and error information.

The Dataset Services Response (DSR) provides a 'Services' or 'Capabilities' response for DAP4. Dereferencing an unadorned DAP4 dataset resource URL will return a document describing the DAP services available for the dataset.

The DAP utilizes two responses to represent semantic structural description and data content of a data source. One response, the DMR returns metadata information describing the structure of a request for data. That is, it characterizes the variables, their datatypes, names and attributes. The second response, the Data Response, returns both the metadata about the request, but also the data that was requested. The DMR and the metadata part of the Data Response are represented using a specific XML [16] representation. The syntax of that representation is defined previously (Section 5.3).

The DAP Asynchronous Response is returned to a client when the requested resource (DMR, Data Response, etc.) is not immediately available but by making a specific request that it be made available the server is able to retrieve it. If the client makes the "retrieve it" request the server will inform the client through a subsequent Asynchronous Response when and where the client may access the requested resource.

The DAP returns error information using an Error response. If a request for any of the three basic responses cannot be completed then an Error response is returned in its place.

The two responses (DMR and Data Response) are complete in and of themselves so that, for example, a client can use the data response without ever requesting either of the two other responses. In many cases, client programs will request the DMR response first before requesting the Data Response but there is no requirement they do so and no server SHALL require that behavior on the part of clients.

Operationally, communication between a DAP client and a DAP server uses some underlying already existing protocol. Volume 2 discusses the appropriate choices for the underlying protocol.

In addition to these data objects, a DAP server MAY provide additional "services" which clients may find useful. For example, many DAP-compliant servers provide HTML-formatted representations or ASCII representations of a data source's structure and data. Such additional services are discussed in Volume 2 of this specification.

Characterization of a Data Source

The DAP characterizes a data source as a collection of variables, dimensions, and enumeration types. Each variable consists of a name, a type, a value, and a collection of Attributes. Dimensions have a name and a size. Enumerations list names and values of the enumeration constants. These elements may be grouped into collections using the concept of a "group" that has an identifier and defines a naming scope for the elements within it. Groups may contain other groups.

The distinction between information in a variable and in an Attribute is somewhat arbitrary. However, the intention is that Attributes hold information that aids in the interpretation of data held in a variable. Variables, on the other hand, hold the primary content of a data source.

Section 13 provides a formal syntax for DAP DMR characterizations. It is defined using the RelaxNG standard [13] for describing the context-free syntax of a class of XML documents, the DMR in this case. It should be noted that any syntax specification requires a specification of the lexical elements of the syntax. The XML specification [16] provides most of the lexical context for the syntax, but there are certain places where additional lexical elements must be used. Section 11 describes those additional lexical elements, and those elements are discussed at appropriate points in the following discussion.

Since the syntax is context-free, there are semantic limitations on what is legal in a DMR. These semantic limitations are noted at appropriate places in the following documentation. It should also be noted that if there are conflicts between what is described here and the RelaxNG syntax, then the syntax takes precedence.

DMR Declarations

DMR XML Format

Element and Attribute Names

Within the DMR XML document, it is assumed that element names are case sensitive. XML attribute names are case insensitive.

Character Escapes

Any string of characters appearing within an XML attribute in the DMR must apply the standard XML escapes. Specifically, any attribute value containing any of the following characters must replace them with the corresponding XML escape form.

Character	Escaped Form
&	&
<	&lt;
>	&gt; c
"	&quot;

So for example, given the occurrence of the attribute 'name="&<>"' it must be re-written to this form 'name="&amp;&lt;&gt;"'.

Names

A name (aka identifier) in DAP4 consists of a sequence of any legal non-control UTF-8 characters. A control character is any UTF-8 character in the inclusive range 0x00 — 0x1F.

Fully Qualified Names

Every object in a DAP4 Dataset has a Fully Qualified Name (FQN), which provides a way to unambiguously reference declarations in a dataset and which can be used in several contexts such as in the DMR in a constraint expression (see Section 8). These FQNs follow the common conventions of names for lexically scoped identifiers. In DAP4 three kinds of lexical items provide lexical scoping: Dataset, Groups, Structures, and Sequences . Just as with hierarchical file systems or variables in many programming languages, a simple grammar formally defines how the names are built using the names of the FQN's components (see Section 10). Consider the following simple dataset, which contains a Structure named "inner" within a Structure named "outer" all contained in the Dataset "D".

<Dataset name="D">
    <Structure name="places">
        <String name="name"/>
        <Structure name="weather">
            <Float64 name="temperature"/>
            <Float64 name="dew_point"/>
        </Structure>
    </Structure>
</Dataset>

The FQN for the field 'temperature' is

'/places.weather.temperature'.

Substituting the keyword Sequence for one or more occurrences of Structure in the above example will leave the FQNs unchanged.

As is the case with Structure or Sequence variables, Groups can be nested to form hierarchies, too, and this example shows that case.

<Dataset name="D">
    <Group name="environmental_data">
        <Structure name="places">
            <String name="name"/>
            <Sequence name="weather">
                <Float64 name="temperature"/>
                <Float64 name="dew_point"/>
            </Sequence>
        </Structure>
     </Group>
     <Group name="demographic_data">
         ...
     </Group>
</Dataset>

The FQN to the field 'temperature' in the dataset shown is

'/environmental_data/places.weather.temperature'.

Notes:

1. Every dataset has a single outermost <Dataset> declaration, which semantically, acts like the root group. Whatever name that dataset has is ignored for the purposes of forming the FQN and instead is treated as if it has the empty name ("").
2. There is no limit to the nesting of groups or the nesting of Structures or the nesting of Sequences.

The characters "/" and "." have special meaning in the context of a fully qualified name. This means that if a name is added to the FQN and that name contains either of those two characters, then those characters must be specially escaped so that they will not be misinterpreted. The defined escapes are as follows.

Character	Escaped Form
.	\.
/	\/
\	\\

Note that the escape character itself must be escaped. Also note that this form of escape using '\' is independent of any required XML escape (Section 5.1).

FQN References

DAP4 imposes the rule that the definition of any object (e.g. dimension, group, or enumeration) must occur before any reference to that object. This rule also applies within a group, which in turn implies that, for example, all dimensions must be declared before all variables that reference them.

Definitional Declarations versus Data-Bearing Declarations

The declarations in a DMR can be grouped into two classes. One class is definitional. That is, it defines metadata that is used in the rest of the DMR. These definitional declarations are Groups (including the outer Dataset), Dimensions, and Enumerations. Such declarations do not contain data values themselves, although they may define constants such as the dimension size. The data-bearing declarations are Variables and Attributes. These elements of the data model are used to house data values or semantic metadata read from the dataset (or, in the latter case) synthesized from the values and standards/conventions that the dataset is known to follow.

Dataset

Every DMR contains exactly one Dataset declaration. It is the outermost XML element of the DMR.

A dataset is specified using this XML form:

<Dataset name="..." dapVersion="..." dmrVersion="...";
...
</Dataset>

The name, dapVersion, and dmrVersion, attributes are required. The attributes have the following semantics:

• name – an identifier specifying the name of the dataset. Its content is determined solely by the Server and is completely uninterpreted with respect to DAP4.
• dapVersion – the string "4.0" currently.
• dmrVersion – the string "1.0" currently.

The body of the Dataset is the same as the body of a 5.7, and semantically the Dataset acts like the outermost, root, group.

Groups

A group is specified using this XML form:

<Group name="name">
...
<Group>

A group defines a name space and contains other DAP elements. Specifically, it can contain groups, variables, dimensions, and enumerations. The fact that groups can be nested means that the set of groups in a DMR form a tree data structure. For any given DMR, there exists a root group that is the root of this tree.

A nested set of groups defines a variety of name spaces and access to the contents of a group is specified using a notation of the form "/g1/g2/.../gn". This is called a "path". By convention "/" refers to the root group (the Dataset declaration). Thus the path "/g1/g2/g3" indicates that one should start in the root group, move to group g1 within that root group, then to group g2 within group g1, and finally to group g3. This is more fully described in the section on Fully Qualified names (Section 5.3).

The order of declarations within a Group is fixed and must conform to this order.

1. Dimension declarations,
2. Enumeration declarations,
3. Variable declarations,
4. and, finally, nested Group declarations,

For comparison purposes, DAP groups correspond to netCDF-4 groups and not to the more complex HDF5 Group type: i.e. the set of groups must form a tree.

Semantic Notes

1. If declared, Groups must be named.
2. A Group can contain any number of objects, including other Groups.
3. Each Group declares a new lexical scope for the objects it contains.
4. A Group cannot have dimensions and a Group cannot be defined within a Structure or Sequence.

Dimensions

A dimension declaration is specified using this XML form.

<Dimension name="name" size="size"/>

The size is a positive integer (which means that a zero length dimension is illegal). As described in the <a href="#arrays>Arrays Section</a>, the maximum size of any dimension is 2^{61 - 1. A dimension declaration will be referenced elsewhere in the DMR by specifying its name. It should also be noted that anonymous dimensions also exist. They have a size but no name. Anonymous dimensions SHOULD NOT be declared.}

Semantic Notes

1. Dimension declarations are not associated with a data type.
2. Dimension sizes that are not 'anonymous' MUST be a capable of being represented as a signed 64-bit integer.

Enumeration Types

An enumeration type defines a set of names with specific values: enumeration constants. As will be seen in Section 5.12, enumeration types may be used as the type for variables or attributes. The values that can be assigned to such typed objects must come from the set of enumeration constants.

An enumeration type specifies a set of named, integer constants. When a data source has a variable of type 'Enumeration' a DAP 4 server MUST represent that variable using a specified integer type, up to an including a 64-bit unsigned integer.

An Enumeration type is declared using this XML form.

<Enumeration name="name">
                basetype="Byte|Int8|UInt8|Int16|UInt16
                         |Int32|UInt32|Int64|UInt64"/>
    <EnumConst name="name" value="integer"/>
    ...
</Enumeration>

Semantic Notes

1. The optional "basetype" XML attribute defines the type for the value XML attribute of each enumeration constant. This basetype must be one of the integer types (see Section 5.10.1). If unspecified, then it defaults to the Atomic type "Int32".

Atomic Types

The DAP4 specification assumes the existence of certain pre-defined, declared types called atomic types. As their name suggests, atomic data types are conceptually indivisible. Atomic variables are used to store integers, real numbers, strings and URLs. There are five classes of atomic types, with each family containing one or more variations: integer, floating-point, string, enumerations, and opaque.

Integer Types

The integer types are summarized in the following table. The syntax for integer constants is defined in Section 11.3.

Type Name	Description	Range of Legal Values
Int8	Signed 8-bit integer	[-(27), (27) - 1]
UInt8	Unsigned 8-bit integer	[0, (28) - 1]
Byte	Synonym for UInt8	[0, (28) - 1]
Char	Synonym for UInt8	[0, (28) - 1]
Int16	Signed 16-bit integer	[-(215), (215) - 1]
UInt16	Unsigned 16-bit integer	[0, (216) - 1]
Int32	Signed 32-bit integer	[-(231), (231) - 1]
UInt32	Unsigned 32-bit integer	[0, (232) - 1]
Int64	Signed 64-bit integer	[-(263), (263) - 1]
UInt64	Unsigned 64-bit integer	[0, (264) - 1]

Note that for historical reasons, the Char type is defined to be a synonym of UInt8, this mean that technically, the Char type has no associated character set encoding. However, servers and clients are free to infer typical character semantics to this type. The inferred character set encoding is chosen purely at the discretion of the server or client using whatever conventions they agree to use.

Floating Point Types

The floating-point data types are summarized in Table 2. The two floating-point data types use IEEE 754 [6] to represent values. The two types correspond to ANSI C's float and double data types. The syntax for floating point constants is defined in Section 11.3.

Type Name	Description	Range of Legal Values
Float32	32-bit Floating-point number	Refer to the IEEE Floating Point Standard [6]
Float64	64-bit Floating-point number	Refer to the IEEE Floating Point Standard [6]

String Types

The string data types are summarized in Table 3. Again, the syntax for these is defined in Section 11.4

Strings are individually sized. This means that in an array of strings, for example, each instance of that string MAY be of a different size.

Type Name	Description	Range of Legal Values
String	A variable length string of UTF-8 characters	As defined in [14]
URI	A Uniform Resource Identifier	As defined in IETF RFC 2396 [8]

The Opaque Type

The XML scheme for declaring an Opaque type is as follows.

<Opaque>

The Opaque type is use to hold objects like JPEG images and other Binary Large Object (BLOB) data that have significant internal structure which might be understood by clients (e.g., an image display program) but that would be very cumbersome to describe using the DAP4 built-in types. Defining a variable of type "Opaque" does not communicate any information about its content, although an attribute could be used to do that.

Semantic Notes

1. The content of an opaque object is completely un-interpreted by the DAP4 implementation. The Opaque type is an Atomic Type, which might seem odd because instances of Opaque can be of different sizes. However, by thinking of Opaque as equivalent to a byte-string type, the analogy with strings makes it clear that it should be an Atomic type.

The Enum Type

The XML scheme for declaring an Enum type is as follows.

<Enum enum="FQN">

Semantic Notes

1. The Enum typed requires the an attribute that references a previously defined <Enumeration> declaration.

A Note Regarding Implementation of the Atomic Types

When implementing the DAP, it is important to match information in a data source or read from a DAP response to the local data type which best fits those data. In some cases an exact match may not be possible. For example Java lacks unsigned integer types [4]. Implementations faced with such limitations MUST ensure that clients will be able to retrieve the full range of values from the data source. If this is impractical, then the server or client may implement this rule by hiding the variable in question or returning an error.

Container Types

There are currently two container types: <Structure> and <Sequence>.

The Structure Type

A Structure groups a list of variables so that the collection can be manipulated as a single item. The variables in a Structure may also be referred to as "fields" to conform to conventional use of that term, but there is otherwise no distinction between fields and variables. The Structure's fields MAY be of any type, including Structure or Sequence. The order of items in the Structure is significant only in relation to the serialized representation of that Structure.

The Sequence Type

A Sequence is intended to represent a sequence of instances of objects. Suppose that we have a sequence of this form.

<Sequence name="s">
    <Float64 name="field1"/>
    <Float64 name="field2"/>
</Sequence>

The corresponding Structure object is obtained by substuting the Sequence keyword with Structure. Our above example then has this associated Structure.

<Structure name="s">
    <Float64 name="field1"/>
    <Float64 name="field2"/>
</Structure>

The semantics of a sequence are that it represents a sequence of instances of the corresponding Structure. The length of the Sequence MAY be different for every instance of a Sequence. Consider this array of Sequence.

<Sequence name="s">
    ...
    <Dim size="3">
    <Dim size="2">
</Sequence>

This represents an array of six (3 times 2) sequence instances. However, the length MAY be different for each of those six instances.

Note that the <Sequence> construct was introduced to replace the concept of variable length dimensions. It turns out that trying to treat variable length dimensions as dimensions causes significant conceptual and implementation difficulties. It is hoped thatisolating such variable length objects syntactically is a better representation.

Semantic Notes

1. Structures and Sequences MAY freely nested.

Variables

Each variable in a data source MUST have a name, a type and one or more values. Using just this information and armed with an understanding of the definition of the DAP data types, a program can read any or all of the information from a data source.

The DAP variables come in several different types. There are several atomic types, the basic indivisible types representing integers, floating point numbers and the like, and a container type – the Structure or Sequence type – that supports aggregation of other variables into a single unit. A container type may contain both atomic typed variable as well as other container typed variables, thus allowing nested type definitions.

The DAP variables describe the data when it is being transferred from the server to the client. It does not necessarily describe the format of the data inside the server or client. The DAP defines, for each data type described in this document, a serialized representation, which is the information actually communicated between DAP servers and DAP clients. The serialized representation consists of two parts: the declaration of the type and the serialized encoding of its value(s). The data representation is presented in Section 6.1".

Arrays

Most (but not all) types may be arrays. An Array is a multi-dimensional indexed data structure. An Array's member variable MUST be of some DAP data type. Array indexes MUST start at zero. Arrays MUST be stored in row-major order (as is the case with ANSI C), which means that the order of declaration of dimensions is significant. The size of each Array's dimensions MUST be given. The number of elements in an Array is fixed as that given by product of the size(s) of its dimension(s). Note that a dimension size of zero is illegal.

For practical reasons having to do with current hardware limitations, the total number of bytes allocated to an array must fit in an unsigned 64-bit integer. The largest atomic types currently defined in this document are the floating point double and the (U)Int64 integer types. This means that the practical limit on the total number of elements is 2⁶⁴ / 8 = 2⁶¹. Thus the dimension indices will run from 0 to a maximum of 2⁶¹ - 1. Of course this limit on the maximum number of elements also applies to the maximum dimension size since the total number of elements is the product of all the dimensions sizes of the array.

There is a prescribed limit of 64 on the number of of dimensions for a variable (i.e. its arity). This is actually larger than will occur in practice. Assuming a dimension must be at least 1 bit in size, this effectively limits the number of dimensions to 61.

Semantic Notes

1. Simple variables (see below) MAY be arrays.
2. Structures and Sequences MAY be arrays.

Simple Variables

A simple, dimensioned variable is declared using this XML form.

<Int32 name="name">
  <Dim name="{fqn};"/>
  ...
  <Dim size="{integer}"/>
</Int32>

Note the use of two types of dimensions.

1. name="{fqn}" – specify the fully qualified name of a dimensions declared previously,
2. size="{integer}" – specify an anonymous dimension of a given size,

A simple variable is one whose type is one of the Atomic Types (see Section 5.10). The name of the Atomic Type (Int32 in this example) is used as the XML element name. Within the body of that element, it is possible to specify zero or more dimension references. A dimension reference (<Dim.../>) MAY refer to a previously defined dimension declaration. It MAY also define an anonymous dimension with no name, but with a size specified as an integer constant.

Semantic Notes

1. N.A.

Dimension Ordering

Consider this example.

<Int32  name="i">
    <Dim name="/d1"/>
    <Dim name="/d2"/>
    ...
    <Dim name="/dn"/>
</Int32>

The dimensions are considered ordered from top to bottom. From this, a corresponding left-to-right order [d1][d2]...[dn] can be inferred where the top dimension is the left-most and the bottom dimension is the right-most. The assumption of row-major order means that in enumerating all possible combinations of these dimensions, the right-most is considered to vary the fastest. The terms "right(most)" or "left(most") refer to this left-to-right ordering of dimensions.

Structure Variables

As with simple variables, a structure variable specifies a type as well as any dimension for that variable. The type, however, is a Structure.

Structures

The XML scheme for a Structure typed variable is as follows.

<Structure name="name">
  {variable definition}
  {variable definition}
  ...
  {variable definition}
  <Dim ... />
  ...
  <Dim ... />
</Structure>

The Structure contains within it a list of variable definitions (Section 5.12). For discussion convenience, each such variable may be referred to as a "field" of the Structure. The list of fields may optionally be followed with a list of dimension references indicating the dimensions of the Structure typed variable.

Semantic Notes

1. Structures MAY be dimensioned.

Sequence Variables

As with simple variables, a sequence variable specifies a type as well as any dimension for that variable. The type, however, is a Sequence.

Sequences

The XML scheme for a Sequence typed variable is as follows.

<Sequence name="name">
  {variable definition}
  {variable definition}
  ...
  {variable definition}
  <Dim ... />
  ...
  <Dim ... />
</Sequence>

The Sequence contains within it a list of variable definitions (Section 5.12). For discussion convenience, each such variable may be referred to as a "field" of the Sequence. The list of fields may optionally be followed with a list of dimension references indicating the dimensions of the Sequence typed variable.

Semantic Notes

1. Sequences MAY be dimensioned.

Coverage Variables and Maps

A "Discrete Coverage" is a concept commonly found in many disciplines, where the term refers to a sampled function with both its domain and range explicitly enumerated by variables. DAP2 uses the name 'Grid' to denote what the OGC calls a 'rectangular grid' [12]. DAP4 expands on this so that other types of discrete coverages (hereafter 'coverage(s)') can be explicitly represented.

In DAP4, the range for a coverage is the values of a (simple or container) variable that includes a specific set of 'maps' or 'coordinate variables' that define the domain for the sampled function. Taken as whole, this type of variable is called a "grid" for convenience sake.

Using OGC coverage terminology, we have this.

1. The maps specify the "Domain"
2. The array specifies the "Range"
3. The Grid itself is a "Coverage" per OGC.
4. The Domain and Range are sampled functions

A map is defined using the following XML scheme.

<Map name="{FQN for some variable defined in the DMR}"/>

An example might look like this.

<Float32 name="A">
  <Dim name="/lat"/>
  <Dim name="/lon"/>
  <Map name="/lat"/>
  <Map name="/lon"/>
</Float32>

Where the map variables are defined elsewhere like this.

<Float32 name="lat">
  <Dim name="/lat"/>
</Float32>

<Float32 name="/lon">
  <Dim name="/lon"/>
</Float32>

The containing variable, A in the example, will be referred to as the "array variable".

Semantic Notes

1. Each map variable MUST have a rank no more than that of the array.
2. An array variable can have as many maps as desired.
3. The dimensions of the array variable may not contain duplicates so A[x,x] is disallowed.
4. Any map duplicates are ignored and the order of declaration of the maps is irrelevant.
5. The fully qualified name of a map must either be in the same lexical scope as the array variable, or the map must be in some enclosing scope.
6. The set of named "associated dimensions for a map must be a subset of the set of named "associated dimensions" for the array variable.

The term "associated dimensions" is computed as follows.

1. The set of associated dimensions is initialized to empty.
2. For each element mentioned in the fully qualified name (FQN) of the map or the array variable, add any named dimensions associated with FQN element to the set of associated dimensions (removing duplicates, of course).

In practice, the means that an array variable or map variable must take into account any dimensions associated with any enclosing dimensioned Structure or Sequence.

Attributes and Arbitrary XML

Attributes

Attributes are defined using the following XML scheme.

<Attribute name="name" type="{atomic type name}">
  <Namespace href="http://netcdf.ucar.edu/cf"/>
  <Value value="value"/>
  ...
  <Value value="value"/>
</Attribute>

<Attribute name="name" type="{container name}">
  <Namespace href="http://netcdf.ucar.edu/cf"/>

  <Attribute name="name" type="...">
    ...
  </Attribute>

  ...

  <Attribute name="name" type="...">
    ...
  </Attribute>

</Attribute>

In DAP4, Attributes (not to be confused with XML attributes) are tuples with four components:

• Name
• Type (one of the defined atomic types such as Int16, String, etc.), or a child attribute container
• Vector of values
• One or more Namespaces (optional)

This differs slightly from DAP2 Attributes because the namespace feature has been added, although clients can choose to ignore it. For more about namespaces, refer to Section 5.14. The intent of including the namespace information is to simplify interactions with semantic web applications where certain schemas or standards have formal definitions of attributes.

Attributes are typically used to associate semantic metadata with the variables in a data source. Attributes are similar to variables in their range of types and values, except that they are somewhat limited when compared to those for variables: they cannot use Structure or Sequence types

Attributes defined at the top-level within a group are also referred to as "group attributes". Attributes defined at the root group (i.e. Dataset) are "global attributes," which many file formats such as HDF4 or netCDF formally recognize.

While the DAP does not require any particular Attributes, some may be required by various metadata conventions. The semantic metadata for a data source comprises the Attributes associated with that data source and its variables. Thus, Attributes provide a mechanism by which semantic metadata may be represented without prescribing that a data source use a particular semantic metadata convention or standard.

Semantic Notes

1. DAP4 explicitly treats an attribute with one value as an attribute whose value is a one-element vector.
2. All of the Atomic types as well as containers are allowed as the type for an attribute
3. If the attribute has type Enum, it must also have an attribute that references a previously defined <Enumeration> declaration.
4. Attribute value constants MUST conform to the appropriate constant format for the given attribute type and as defined in Section 11.
5. Attributes may themselves have attributes: effectively leading to nested attributes. Such attributes are called container attributes. However container attributes may not have values; only lowest level (leaf) attributes may have values.

Arbitrary XML content

By supporting an explicit type to hold "arbitrary XML" markup, DAP4 provides a way for the protocol to transport information encoded in XML along with the attributes read from the dataset itself. This has proved very useful in work with semantic web software.

The form on an otherXML declaration is as follows.

<otherXML name="name">
{arbitrary xml}
</otherXML>

There are no <value/> elements because the value of otherXML is the xml inside the <otherXML>...</otherXML>. The text content of the otherXML element must be valid XML and must be distinct from the XML markup used to encode elements of the DAP4 data model (i.e., in a practical sense, the content of an <OtherXML> attribute will be in a namespace other than DAP4). XML content may appear anywhere that an attribute may appear.

Attribute and OtherXML Specification and Placement

Attribute and OtherXML declarations MAY occur within the body of the following XML elements: Group, Dataset, Dimension, Variable, Structure, Sequence, and Attribute.

Namespaces

All elements of the DMR – Dataset, Groups, Dimensions, Variables, and Attributes – can contain an associated Namespace element. The namespace's value is defined in the form of an XML style URI string defining the context for interpreting the element containing the namespace. Suppose, hypothetically, that we wanted to specify that an Attribute is to be interpreted as a CF convention [15]. One might specify this as follows.

<Attribute name="latitude">
  <Namespace href="http://cf.netcdf.unidata.ucar.edu"/>
  ...
</Attribute>

Note that this is not to claim that this is how to specify a CF convention [15].; this is purely illustrative.

Data Representation

Data can be an elusive concept. Data may exist in some storage format on some disk somewhere, on paper somewhere else, in active memory on some server, or transmitted along some wire between two computers. All these can still represent the same data. That is, there is an important distinction to be made between the data and its representation. The data can consist of numbers: abstract entities that usually represent measurements of something, somewhere. Data also consist of the relationships between those numbers, as when one number defines a time at which some quantity was measured.

The abstract existence of data is in contrast to its concrete representation, which is how we manipulate and store it. Data can be stored as ASCII strings in a file on a disk, or as twos-complement integers in the memory of some computer, or as numbers printed on a page. It can be stored in HDF5 [2], netCDF [5], GRIB[17], a relational database, or any number of other digital storage forms.

The DAP specifies a particular representation of data, to be used in transmitting that data from one computer to another. This representation of some data is sometimes referred to as the serialized representation of that data, as distinguished from the representations used in some computer's memory. The DAP standard outlined in this document has nothing at all to say about how data is stored or represented on either the sending or the receiving computer. The DAP transmission format is completely independent of these details.

Response Format

There are two response formats that a server MUST provide to the client.

1. DMR-only response
2. (DMR +) Data response

DMR-Only Response If the client requests only the DMR, then it is returned as a standard XML encoded document. If constraints were specified, then the returned DMR may differ from the full DMR in that, for example, meta-data about only variables specified in the constraint will be returned.

Data Response The DAP4 data response uses a format very similar to that used for DAP2; the data payload is broken into two pieces. The first part holds metadata describing the names and types of the variables in the response while the second part holds the values of those variables.

The metadata information, sent as part 1 of the Data Response, is the DMR limited to just those variables included in the response. DAP attributes may be included, but MAY be ignored by the receiving client.

Part 2 of the response consists of the binary data for each variable in the order they are listed in the DMR given as the response preface. DAP4 uses a receiver makes it right encoding, so the servers MAY simply write out binary data as they store it with the exceptions that floating-point data must be encoded according to IEEE 754[6] and Integer data must use twos-complement notation for signed types. Clients are responsible for performing byte-swapping operations needed to compute using the values retrieved.

The Data Response is encoded using chunking scheme (see Section 6.1.3). that breaks it into N parts where each part is prefixed with a chunk type and chunk byte count header. Chunk types include data and error types, making it simple for servers to indicate to clients that an error occurred during the transmission of the Data Response and (relatively) simple for clients to detect that error.

As with DAP2, the response describe here is a document that can be stored on disk or sent as the payload using a number of network transport protocols, HTTP being the primary transport in practice. However, any protocol that can transmit a document can be used to transmit these responses. As such, all critical information needed to decode the response is completely self-contained.

In the rest of this section we will describe the Data Response in the context of DAP4 using HTTP as its transport protocol.

Format of the DMR Part

The first part (part is not to be confused with chunk) of the Data Response always contains the DMR. The Data Response, when DAP is using HTTP as a transport protocol, is the payload for an HTTP response, is separated from the last of the HTTP response's MIME headers by a single blank line, which MIME defines as a carriage return (ASCII value 13) followed by a line feed (ASCII value 10). This combination can be abbreviated as CRLF.

Format of the Data Part The second part of the Data response consists of the serialized variables as specified by the data DMR. The variable serializations are concatenated to form a single binary dataset. If requested, each variable's serialization is followed by a CRC32 checksum.

Relationship to the Chunking format The data response format is technically independent of the chunking format

(see 6.1.3).

The assumption is that the DMR will be in a chunk of its own, the first chunk, and the serialized binary data will be in one or more additional chunks. This produces a format like this

CRLF
{DMR Length in binary form}
{DMR}
CRLF
{Chunk 1 containing some portion of the serialized data}
...
{Chunk n containing the last portion of the serialized data}

In the above and in the following, the form '{xxx}' is intended to represent any instance of the xxx.

How the Chunked Encoding Affects the Data Response Format

In a sense, the chunked encoding does not affect the format of the Data Response at all. Conceptually, the entire binary Data Response is built and then passed through a 'chunking encoder' transforming it into one that is broken up into a series of chunks. That 'chunked document' is the sent as the payload of some transport protocol, e.g., HTTP. In practice, that would be a wasteful implementation because a server would need to hold the entire response in memory. A better implementation would, for HTTP, write the initial parts of the HTTP response (its response code and headers) and then use a pipeline of filters to perform the encoding operations. The intent of the chunking scheme is to make it possible for servers to build responses in small chunks, and once they know those parts have been built without error, send them to the client. Thus a server should choose the chunk size to be small enough to fit comfortably in memory but large enough to limit the amount of overhead spent by the software that encodes and decodes those chunks. When an error is detected, the normal flow of building chunks and sending the data along is broken and an error chunk should be sent (See Section 12).

The DAP4 Serialized Representation (DSR)

Given a DMR and the corresponding data, the serialized representation is formally described in this section.

A Note on Dimension Ordering

Consider this example.

<Int32  name="i">
  <Dimension name="d1"/>
  <Dimension name="d2"/>
  ...
  <Dimension name="dn"/>
</Int32>

The dimensions are considered ordered the top-to-bottom lexically. This order is linearized into a corresponding left-to-right order [d1][d2]...[dn]. The assumption of row-major order means that in enumerating all possible combinations of these dimensions, the rightmost is considered to vary the fastest. The terms "right(most)" or "left(most") refer to this ordering of dimensions.

Order of Serialization

The data appearing in a serialized representation is the concatenation of the variables specified in the tree of Groups within a DMR, where the variables in a group are taken in depth-first, top-to-bottom order. The term "top-to-bottom" refers to the textual ordering of the variables in an XML document specifying a given DMR.

If a variable is a Structure variable, then its data representation will be the concatenation of the variables it contains, which will appear in top-to-bottom order.

If a variable is a Sequence variable, then its data representation will have two parts.

1. A 64-bit signed count of the number of elements in the sequence
2. Count instances of the 5.11.2 for the Sequence.

If a variable has dimensions, then the contents of each dimensioned data item will appear concatenated and taken in row-major order.

Variable Representation

Given a dimensioned variable, it is represented as the N scalar values concatenated in row-major order.

If the variable is scalar, then it is represented as a single scalar value.

Numeric Scalar Atomic Types

For the numeric atomic types, scalar instances are represented as follows. In all cases a consistent byte ordering is assumed, but the choice of byte order is at the discretion of the program that generates the serial representation, typically a server program.

Type Name	Description	Representation
Int8	Signed 8-bit integer	8 bits
UInt8	Unsigned 8-bit integer	8 bits
Byte	Unsigned 8-bit integer	Same as UInt8
Char	Unsigned 8-bit integer	Same as UInt8
Int16	Signed 16-bit integer	16 bits
UInt16	Unsigned 16-bit integer	16 bits
Int32	Signed 32-bit integer	32-bits
UInt32	Unsigned 32-bit integer	32-bits
Int64	Signed 64-bit integer	64-bits
UInt64	Unsigned 64-bit integer	64-bits
Float32	32-bit IEEE floating point	32-bits
Float64	64-bit IEEE floating point	64-bits

In narrative form: all numeric quantities are used as a raw, unsigned vector of N bytes, where N is 1 for Char, Int8, and UInt8; it is 2 for Int16 and UInt16; it is 4 for Int32, UInt32, and Float32; and it is 8 for Int64, UInt64, and Float64.

Byte Swapping Rules

If the server chooses to byte swap transmitted values, then the following swapping rules are used.

Size (bytes)	Byte Swapping Rules
1	Not Applicable.
2	Byte 0 -> Byte 1 Byte 1 ->Byte 0
4	Byte 0 -> Byte 3 Byte 1 ->Byte 2 Byte 2 -> Byte 1 Byte 3 ->Byte 0
8	Byte 0 -> Byte 7 Byte 1 ->Byte 6 Byte 2 -> Byte 5 Byte 3 ->Byte 4	Byte 4 -> Byte 3 Byte 5 ->Byte 2 Byte 6 -> Byte 1 Byte 7 ->Byte 0

Variable-Length Scalar Atomic Types

Type Name	Description	Representation
String	Vector of 8-bit bytes representing a UTF-8 String	The number of bytes in the string (in Int64 format) followed by the bytes.
URL	Vector of 8-bit bytes representing a URL	Same as String
Opaque	Vector of un-interpreted 8-bit bytes	The number of bytes in the vector (in Int64 format) followed by the bytes.

In narrative form, instances of String, Opaque, and URL types are represented as a 64 bit length (treated as Int64) of the instance followed by the vector of bytes comprising the value.

Structure Variable Representation

A Structure typed variable is represented as the concatenation of the representations of the variables contained in the Structure taken in textual top-to-bottom order. This representation may be nested if one of the variables itself is a Structure variable. Dimensioned structures are represented in a form analogous to dimensioned variables of atomic type. The Structure array is represented by the concatenation of the instances of the dimensioned Structure, where the instances are listed in row-major order.

Sequence Variable Representation

A Sequence typed variable is represented as a count specifying the number of objects (not bytes) of the sequence followed by count instances of the corresponding Structure using the Structure representation rules. This representation may be nested if one of the variables itself is a Sequence variable. Dimensioned sequences are represented in a form analogous to dimensioned variables of atomic type. The Sequence array is represented by the concatenation of the instances of the dimensioned Sequence, where the instances are listed in row-major order.

Each Sequence variable, then, consists of a length, L say, in Int64 form and giving the number of elements for a specific occurrence of the variable-length dimension. The count, L, is then followed by L instances of the serialized form of the sequence's corresponding structure.

Checksums

As an option, checksums will be computed for the values of all the "top-level" variables present in the DMR of a returned response from a server. The term "top-level" means that the variable is not a field of a Structure typed variable.

The purpose of the checksum is to detect changes in data over time. That is, if a client requests the same variable and the returned checksums are the same, then the client may infer that the data has not changed. The checksum is not intended for transmission error detection, although the client MAY use it for that purpose if it chooses.

The checksum is made visible to the client by adding an attribute to each top-level variable in the DMR. This attribute is named "DAP4_Checksum_CRC32".

In all cases, the checksum is computed over the serialized representation of each top-level variable. The checksum is computed before any chunking Section 7) is applied.

If the request to the server is a dmr-only request, then the server will compute the checksum for each variable mentioned in the DMR and will insert the "DAP4_Checksum_CRC32" attribute in the DMR. Note that this can have significant performance consequences since the server is required to read and serialize all of the data for all of the variables mentioned in the DMR even though that data is not transmitted to the client.

If the request to the server is a data request, then the checksum value will follow the value of the variable in the data part of the response. The computed checksum is appended to the serialized representation for transmission to the client. Note that in this case, the client is expected to add the "DAP4_Checksum_CRC32" attribute to the DMR.

The default checksum algorithm is CRC32. So the size of each checksum inserted in the serialization will be a 32 bit integer. The checksum integer will use the same endian representation as for the all other data. Note that CRC32 is not a cryptographically strong checksum, so it is not suitable for detecting man-in-the-middle attacks.

Historical Note

The encoding described in Section 6.1 is similar to the serialization form of the DAP2 protocol [3], but has been extended to support arrays with a varying dimension and stripped of redundant information added by various XDR implementations.

The DAP4 Serialization rules are derived from, but not the same as, XDR [10]. The differences are as follows.

1. Values are encoded using the byte order of the server. This is the so-called "receiver makes it right" rule.
2. No padding is used.
3. Floating point values always use the IEEE 754 standard.
4. One and two-byte values are not converted to four byte values.

Example responses

In these examples, spaces and newlines have been added to make them easier to read. The real responses are more compact. Since this proposal is just about the form of the response - and it really focuses on the BLOB part - there is no mention of 'chunking.' For information on how this BLOB will/could be chunked. see Section 7. NB: Some poetic license used in the following and the checksums for single integer values seems silly, but these are really simple examples.

A single scalar

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
<Int32 name="x"/>
</Dataset>
CRLF
{chunk count+tag}
x
{checksum}

A single array

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
<Int32 name="x">
<Dim size="2">
<Dim size="4">
</Int32>
</Dataset>
CRLF
{chunk count+tag}
x00 x01 x02 x03 x10 x11 x12 x13
{checksum}

A single structure

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
  <Structure name="S">
    <Int32 name="x">
      <Dim size="2">
      <Dim size="4">
    </Int32>
    <Float64 name="y"/>
  </Structure>
</Dataset>
CRLF
{chunk count+tag}
x00 x01 x02 x03 x10 x11 x12 x13
y
{checksum}

Note that in this example, there is a single variable at the top-level of the root Group, and that is S; so it is S for which we compute the checksum.

An array of structures

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
  <Structure name="s">
    <Int32 name="x">
      <Dim size="2"/>
      <Dim size="4"/>
    </Int32>
    <Float64 name="y"/>
    <Dim size="3"/>
  </Structure>
</Dataset>
CRLF
{chunk count+tag}
x00 x01 x02 x03 x10 x11 x12 x13 y x00 x01 x02 x03 x10 x11 x12 x13 y x00 x01 x02 x03 x10 x11 x12 x13 y
{checksum}

Single array with sequence

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
  <String name="s"/>
  <Sequence name="a-star">
      <Int32 name="a"/>
  </Sequence>
  <Sequence name="x-star">
      <Int32 name="x"/>
      <Dim size="2"/>
  </Sequence>
</Dataset>
CRLF
{chunk count+tag}
16 This is a string
{checksum}
5 a0 a1 a2 a3 a4
{checksum}
3 x00 x01 x02 6 x00 x01 x02 x03 x04 x05
{checksum}

Notes:

1. The checksum calculation includes only the values of the variable, not the prefix chunk length bytes.
2. The Sequence objects are treated 'like strings' and prefixed with a length count. In the last of the three variables, the dimensioned sequence x-star has two sequence instances where the first sequence has 3 elements and the second has 6.

Nested Sequences

The sequence 'x-start' has a field that is itself a sequence. In the example, at the time of serialization 'x-star' has three elements the inner sequence (of which there are three instances) have three, six and one element, respectively.

...
Content-Type: application/vnd.opendap.org.dap4.data
CRLF
{chunk count+tag}
<Dataset name="foo">
  <Sequence name="x-star">
      <Sequence name="y-star">
          <Int32 name="z"/>
      </Sequence">
  </Sequence">
</Dataset>
CRLF
{chunk count+tag}
3 3 x00 x01 x02 6 x10 x11 x12 x3 x14 x15 1 x20
{checksum}

DAP4 Chunked Data Representation

An important capability for DAP4 is supporting client in determining when a data transmission fails. This is especially difficult when sending binary data (Section 6.1). In order to support such a capability, the DAP4 protocol uses a simplified variation on the HTTP/1.1 chunked transmission format [9] to serialize the data part of the response document so that errors are simple to detect. Furthermore, this format is independent of the form or content of that part of the response, so the same format can be used with different response forms or dropped when/if DAP is used with protocols that support out-of-band error signaling, simplifying our ongoing refinement of the protocol.

The data part of a response document is "chunked" in a fashion similar to that outlined in HTTP/1.1. However, in addition to a prefix indicating the size of the chunk, DAP4 includes a chunk-type code. This provides a way for the receiver to know if the next chunk is part of the data response or if it contains an error response (Section 12). In the latter case, the client should assume that the data response has ended, even though the correct closing information was not provided.

Each chunk is prefixed by a chunk header consisting of a chunk type and byte count, all contained in a single four-byte word. The encoding of this word is always network byte order (i.e. Big-Endian) The chunk type will be encoded in the high-order byte of the four-byte word and chunk size will be given by the three remaining bytes of that word. The maximum chunk size possible is 2²⁴ bytes. Immediately following the four-byte chunk header will be chunk-count bytes followed by another chunk header. More precisely the initial four bytes of the chunk are decoded using the following steps.

1. Treat the 32 bit header a single, big-endian, unsigned integer.
2. Convert the integer to the local machine byte order by swapping bytes as necessary (Section 6.2.3.2). Let the resulting integer be called H.
3. Compute the chunk type by the following expression: type = (H >> 24) & 0xff (Using C-language operators).
4. Compute the chunk length by the following expression: length = (H & 0x00ffffff) (Using C-language operators).

The chunk type is determined as a set of one or more flags. Currently, the possible flags are as follows:

• Data (= 0) – Indicates a data containing chunk.
• End (= 1) – Indicates the current chunk is the last data chunk
• Error (= 2) – Indicates the current chunk contains an error message. The Error flag also implies the End flag.
• Little-Endian (= 4) – Indicates that the data in this response is encoded using Little-Endian byte order. If not specified, then Big-Endian is assumed.

It is possible for a chunk type to have more than one of the flags. So, for example, if the data fits into a single chunk, then its chunk type would be Data+End. Error implies End. Note that in order for this to work, the chunk flags values must be powers of two.

The Little-Endian flag must be set only in the first Data chunk. It applies to the whole response. If set in any subsequent chunk type, it will be ignored.

Chunked Format Grammar

chunked_response: chunklist ;
chunklist: chunk | chunklist chunk ;
chunk: CHUNKTYPE SIZE CHUNKDATA ;

Note that there is semantic limitation in the definition of 'chunk': the number of bytes in the CHUNKDATA must be equal to SIZE.

Lexical Structure

/* A single 8-bit byte,
   with the encoding 0 = data, 1 = end, 2 = error, 4 = Little-Endian */
CHUNKTYPE = '\x00'|'\x01'|'\x02'|'\x4'|'\x06'
/* A sequence of three 8-bit bytes,
  interpreted as an integer on network byte order */
SIZE = [\0x00-\0xFF][\0x00-\0xFF][\0x00-\0xFF]
CHUNKDATA = [\0x00-\0xFF]*

Constraints

A request to a DAP4 server for either metadata (the DMR) or data may include a constraint expression. This constraint expression specifies which variables are to be returned and what subset of the data for each variable is to be returned.

It is important to define a minimal request language – a constraint language – to select information from a dataset on a server and obtaining in response a DMR and data corresponding to that request.

This section defines the syntax and semantics of the minimal request language that MUST be supported by all implementations. The method by which a server is provided with a constraint is specified in Volume 2. But as a typical example, if such a constraint were to be embedded in a URL, then it is presumed that it is prefixed with a "?CE={constraint}" and is appended to the end of the URL.

Syntax

The syntax of the minimal constraint language, also referred to as the "simple constraint" language, is as follows.

simpleconstraint: /*empty*/ | constraintlist ;

constraintlist: constraint | constraintlist ',' constraint ;

constraint: variablesubset | namedslice ;

variablesubset: PATH structpath ;

structpath: ID dimset | structpath NAME dimset ;

dimset: /*empty*/ | slicelist ;

slicelist: slice | slicelist slice ;

slice:    '[' start ']'
        | '[' start ':' last ']'
        | '[' start ':' stride ':' last ']'
        | '[' slicename ']' ;

namedslice: slicename '=' slice ;

slicename: ID ;

start: INTEGER ;
last: INTEGER ;
stride: INTEGER ;

The variablesubset rule specifies a subset of values for a variable as specified by the slices. The PATH lexical element is the same as the FQN path as defined in Section 10.

The structpath is almost the same as the FQN prefix as defined in that same Section. The difference is that each component (between '.' separators) of the structpath can have an optional dimset indicating the set of dimension slices to apply.

A dimset is either empty or is a slicelist.

A slicelist is a non-empty list of slices, where a slice indicates a subset of dimension indices. The first case of a slice (e.g. '[5]') indicates a single dimension value, 5 in this case. The second case (e.g. '[5:9]' indicates the range of dimension values 5,6,7,8,9. The third case (e.g. '[5:2:11]') indicates a range of dimension values separated by the stride (the middle values. Thus the example would be the dimension values 5,7,9,11. The fourth case (e.g. '[time]', shows the use of a named slice.

Note that unlike a suffix, intermediate structures in the structlist can have associated dimsets Thus we might have something like this.

/g/S1[5][5:9].v[5:2:11].

A 'namedslice' provides a way to define a slice and give it a slice name. The slice name has lexical type ID. The name, when enclosed in "[]" can be used anywhere a slice is legal. The goal of the 'namedslice' is to ensure that the same slice is used consistently across multiple 'variablesubsets' as a way to impose shared dimension semantics.

There are certain context sensitive constraints on 'structpaths' and 'slicelists'.

1. The terminal variable in the 'structpath' must be an atomic-typed variable.
2. The number of slices associated with a component in the 'structpath' must correspond to the arity of that structure or the last, atomic-typed variable.
3. A slice name must be defined before it is used.

Interpretation

Consider the following Array.

<Int32 name="A">
  <Dim size="d1"/>
  <Dim size="d2"/>
  ...
  <Dim size="dn"/>
</Int32>

where all of the dimension sizes, di, are integers.

Consider the following array subset constraint, where for the purposes of interpretation, all named slices are assumed to have been replaced with their defined slice.

A[start1:stride1:end1]...[startn:striden:endn]

Where

for i=1 .. n, starti < di & endi < di & starti < endi & starti >= 0 & stridei >= 1 & endi >= 0.

The constraint selects the elements A[i1][i2]...[in] from A where ii is in the set {starti+stridei*j} and where j=0..k such that starti+stridei*k <= endi and starti+stridei*(k+1) > endi.

Now consider the same array embedded in a dimensioned Structure.

<Structure name="S">
  <Int32 name="A">
    <Dim size="d3"/>
    ...
    <Dim size="dn"/>
  </Int32>
  <Dim size="d1"/>
  <Dim size="d2"/>
</Structure>

where all of the dimension sizes, di , are again integers.

Consider the following subset constraint.

S[start1:stride1:end1][start2:stride2:end2].A[start3:stride3:end3]...[startn:striden:endn]

with conditions as before.

This constraint selects the Structure instances

S[i1][i2]

where ii is in the set {starti+stridei*j} and where j=0..k such that starti+stridei*k <= endi and starti+stridei*(k+1) > endi.

Then for each selected structure, the elements A[i3]...[in] are selected from that instance of A where ii is in the set {starti+stridei*j} and where j=0..k such that starti+stridei*k <= endi and starti+stridei*(k+1) > endi.

The results of all of the selections of the instances of A are concatenated as the value of the whole constraint.

Dataset Services Response

The Dataset Services Response provides a 'Services' or 'Capabilities' response for DAP4. Dereferencing an unadorned DAP4 dataset resource URL will return a document describing the DAP services available for the dataset. This is a REQUIRED response and the only valid response for an 'unadorned' dataset resource URL. We will refer to this response as the 'Dataset Services Response' or 'DSR' in the 'DAP4 Web Services' specification and the remainder of this document.

At minimum the DSR document MUST provide, for each service available for the dataset:

• A human readable title of that service (e.g., The Dataset Metadata Response service for the dataset)
• One or more links that can be dereferenced to get the various representations of the response for the dataset.
• An unambiguous, unique to the service type, resource role ID that serves as a way to clearly identify what each service does. (Note: Resource roles for all DAP4 services are defined, as well as for several other legacy DAP2 related services in the 'DAP4 Specification, Volume 2 Web Services'. Defining a resource role for a new service will be the responsibility of the service creator.)
• A brief description of the service

The response should also include:

• A link to a complete, human readable, description of the service.
• A reference to an XSLT that a browser would use to render the description into HTML for a presentation view.
• Descriptions and syntax of server side functions available for the dataset

Service Access

A service is accessed by dereferencing one of the access URLs held in the href attribute of a <dsr:link> elements, which is typically constructed by adding some type of suffix to the dataset's referent (aka base) URL. The way in which the query string (aka constraint expression) is used is defined by each service, and there is no requirement for inter-service query string API conformity.

A more extensive discussion of the various services that might appear in a DatasetServices document can be found in DAP4 Volume 2, Web Services.

Note: If a client understands how to modify/augment the dataset resource URL (as described in the DAP4 Volume 2, Web Services ) such that it becomes a DAP response URL then it need never obtain the Dataset Services response.

Dataset Services Response Encoding

c.f. DAP4 Volume 2, Web Services

In this section the namespace prefix dsr is associated with the namespace http://xml.opendap.org/ns/DAP/4.0/dataset-services#.

dsr:DatasetServices element

The Dataset Services Response MUST contain a top level <dsr:DatasetServices> element. This <dsr:DatasetServices> element MUST contain:

• An xml:base attribute whose value is the resource URL that was dereferenced to request the DSR response.
• A list of DAP versions supported by server

  These appear as one or more child <dsr:DapVersion> elements of the <dsr:DatasetServices> element.

• The implementation version of the server that produced the DSR.

  This appears as a single <dsr:ServerSoftwareVersion> child element of the <dsr:DatasetServices> element.

• A list of all available DAP4 services for the dataset

  This is represented as 3 or more <dsr:Service> elements. (There are 3 required services for a DAP4 server.)

• A list of supported extensions
  • Resource type extensions
  • Media type extensions
  • Server-side function extensions

The <dsr:DatasetServices> element SHOULD contain:

• A title attribute whose value is a human readable title for the dataset.

The <dsr:DatasetServices> element MAY contain:

• A <dsr:Description> element whose value is a human readable description of the dataset.

dsr:DapVersion Element

The <dsr:DapVersion> element contains a single DAP version value as a text string. The DAP version is represented as two integer values separated by a period, MM.mm where the first value (MM) is the major version of the DAP protocol, and the second value (mm) is the minor version. Multiple <dsr:DapVersion> elements are used to represent that server can support multiple versions of the DAP protocol, one instance for each supported version.

Example:

<DapVersion>4.0</DapVersion>
<DapVersion>3.2</DapVersion>
<DapVersion>2.0</DapVersion>

dsr:ServerSoftwareVersion element

The <dsr:ServerSoftwareVersion> element has no attributes and may contain as little as a simple text string (e.g., "TDS 4.3.57" or "Hyrax 1.7.45"), or it may contain any well-formed XML content not in the dsr namespace that the server implementer sees fit to use to describe the software version of their server implementation.

dsr:Service element

Each DAP4 web service is described by a <dsr:Service> element with:

• An optional title attribute whose value is a simple human readable title for the service.
• An required role attribute whose value is a unique to the service type, resource role ID (a URI) that serves as a way to clearly identify what each service does. This is intended to allow people and software to unambiguously identify specific services, irrespective of their human readable title. The idea is that that the role attribute tells you what is going to happen when you dereference the URL held in the <dsr:link> element. Each service has a single role and all of the links (alternate representations) must fulfill the same role.
• An optional <dsr:Description> element .
• One or more <dsr:link> elements.

Regarding the role attribute

With regards to multiple representations of a service response and the role attribute: While there may be many alternate representations of a response, not all will fufill the same role. Representations fulfilling the same role should be bundled together in their own Service with the appropriate role value. For example the DMR can be mapped to the ISO19115 metadata space, but IS0-19115 responses are clearly outside of the role of the regular DMR service, which returns a DAP4 metadata response. One could view it as the two roles described different domains.

dsr:Description element

Each <dsr:Description> element MUST contain:

• An optional href attribute whose value is a URL string that points to a human readable document describing the service.
• The required text content of the dsr:Description element MUST be a human readable description of the service.

dsr:link element

Each <dsr:link> element MUST contain:

• A required type attribute whose value is the media-type for the normative representation returned by that URL held in the href attribute.
• A required attribute called href whose value is a URL that when dereferenced will return the service response in the media type described by the value of the type attribute.
• An optional attribute called description whose value is a human readable string that provides a brief description of the <dsr:link> elements semantics.
• Zero or more <dsr:alt> elements, one for each and every alternate representation of the response that can be requested using HTTP server-driven content negotiation in conjunction with the href URL. Each <dsr:alt> element MUST contain:
  • A required type attribute whose value is the MIME type that would be used in the HTTP Accept header to request that representation from the server.

Extensions

Server extensions are simple descriptions of server behaviors that fall outside of the services descriptions provided so far. There are three types of extensions: function, functionGroup, and extension.

A linear regression, a re-gridding operation, and a coordinate re-projection are all examples of a function. A collection of functions that allows the user to perform geocentric manipulations of the data (such as re-gridding operations, reprojection operations) taken together could define a functionGroup. An extension is a server behavior that does not operate directly on the data values, but may provide other types of services and functionality for the client.

It is beyond the scope of this specification to attempt to describe the syntax of usage of a function in some general way. Rather we have chosen to provide a simple mechanism (the role attribute) to alert client software implementations that the server supports specific additional behaviors they can utilize, along with more descriptive elements through which humans can be alerted to the presence of server features and where to find out more about them.

dsr:function

A function extension always describes an operation that applies to the Data Response and the Dataset Metadata Response (because some functions might alter the syntacic structure of the data the change would be previewed by applying the function to the DMR).

Each <dsr:function> MUST contain:

• A name attribute whose value is the name of the function as it would be used in a DR or DMR request.
• A role attribute whose value is a URI that universally defines the function.
• A <dsr:Description> element whose value is a human readable description of the function and a link to detailed documentation of the function and its usage.

A common example of a function would be the DAP2 (and DAP4) geogrid function that allows users to be sub-sampled gridded data using georeferenced values.

dsr:functionGroup

A function group identifies that the server supports a group of functions. Rather than enumerating a large list of related functions, the functionGroup provides the user with the information that the server supports a collection of related functions.

Each <dsr:functionGroup> MUST contain:

• A name attribute whose value is the human readable name of the collection of functions.
• A role attribute whose value is a URI that universally defines the function group.
• A <dsr:Description> element whose value is a human readable description of the function group and a link to detailed documentation regarding the functions in the function group and their usage.

An example of a function group would be a server that implemented a functional syntax for all of the functions found in the Ferret application.

dsr:extension

An extension is a mechanism for indicating that the server supports a certain behavior. This behavior MUST NOT be an operation/computation on the data (other wise it would be a dsr:functionor a dsr:functionGroup), but rather some other behavior that the server can undertake.

Each <dsr:extension> MUST contain:

• A name attribute whose value is the human readable name of the extension.
• A role attribute whose value is a URI that universally defines the extension.
• A <dsr:Description> element whose value is a human readable description of the extension and a link to detailed documentation regarding the extension and it's behavior and usage.

An example of an extension would be the DAP4 asynchronous response support. Some servers will support this functionality, and the functionality is not defined as an operation on the data itself, but in terms of the transaction procedures for accessing the data.

Example: A Minimal DSR

This example contains only the required components of the DSR document from a minimal DAP4 server implementation.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<DatasetServices xmlns="http://xml.opendap.org/ns/DAP/4.0/dataset-services#"
    xml:base="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml">

    <DapVersion>4.0</DapVersion>
    
    <ServerSoftwareVersion>Hyrax-2.7.9</ServerSoftwareVersion>
        
    <Service title="DAP4 Dataset Services" role="http://services.opendap.org/dap4/dataset-services">
        <link type="application/vnd.opendap.org.dataset-services+xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml">
              <alt type="text/xml"/>
        </link>
        <link type="text/xml"  href="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml.xml"/>
    </Service>

    <Service title="DAP4 Dataset Metadata"  role="http://services.opendap.org/dap4/dataset-metadata">
        <link type="application/vnd.org.opendap.dap4.dataset-metadata+xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml.dmr">
              <alt type="text/xml"/>
        </link>
        <link type="text/xml" href="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml.dmr.xml"/>
    </Service>

    <Service title="DAP4 Data" role="http://services.opendap.org/dap4/data">
        <link type="application/vnd.org.opendap.dap4.data" href="http://test.opendap.org:8090/opendap/hyrax/ECMWF_ERA-40_subset.ncml.dap" />
    </Service>

    
</DatasetServices>

Asynchronous Service Behavior and Responses

Asynchronous responses are responses that will take the server some time to build. When a client is told that a response 'is asynchronous,' it must know to come back at a later time to retrieve the response. The concept is a very simple one, and the existing network infrastructure is very good at supporting these kinds of interactions. A major factor in the success of the proposed solution will be the level of uniform support for the design. Secondly, as is often the case, the details will be more complex than the underlying concept. In particular, the request mechanism must be extended so that synchronous (regular) requests are not affected by the addition of asynchronous requests and, at the same time, clients do not inadvertently make asynchronous requests. another detail is that the (asynchronous) responses are ephemeral because they typically only persist for a period of time and then be purged.

A typical 'workflow' for an asynchronous request is:

1. A client makes a data request that indicates that it will accept either an asynchronous or synchronous response. Optionally, the client can place a time constraint on the response, indicating that if the response will not be ready in a given period of time, it does not want the response.
2. The server returns an initial response (without delay) that indicates the request has indeed resulted in an asynchronous response and provides the client with a URL and time estimate.
3. The client reads the time estimate and waits...
4. The client dereferences the URL and gets the response.

The remainder of this section will expand on this basic workflow using examples that focus on the HTTP protocol but that also allow for the use of other transport protocols.

Client willingness to accept asynchronous responses

A client can indicate willingness to accept asynchronous responses in one of two ways:

• By including the X-DAP-Async-Accept HTTP header.
• By adding the async keyword to the DAP constraint expression.

If the client indicates that it must have access to the asynchronous response content within a certain time (utilizing either the X-DAP-Async-Accept HTTP header and/or the async keyword in the constraint expression) and the response will not be available in that time frame, the server MUST reject the request and return an HTTP status of 412 and the DAP Asynchronous Request Rejected XML document.

If both the X-DAP-Async-Accept HTTP header and the async keyword are used, the keyword takes precedence.

Servers must reject requests that require an asynchronous response if the client has not indicated willingness to accept such a response. Rejection of such requests is indicated by all three of the following:

1. HTTP status of 400
2. Inclusion of the X-DAP-Async-Required HTTP response header
3. The response body must contain the DAP Asynchronous Response Required XML document.

This safety check (requiring clients to explicitly indicate their willingness to accept asynchronous responses) is required because otherwise very simple clients might inadvertently make requests that will result in an asynchronous responses, and these kinds of responses are likely to use disproportionately (relative to synchronous responses) more server resources. We want to make DAP4 so that simple clients work well and don't encounter unexpected 'hiccups.'

Initial processing by the server

When a request is accepted by the server and it will result in an asynchronous response, the server MUST the server MUST return a 202 (Accepted) HTTP status code and the DAP Asynchronous Request Accepted XML document. This document contains a URL to the pending result of the request.

Of course, this discussion is about the mechanism that enables a client to make a request and the server to provide information about an asynchronous response to that request. It does not cover any of the nearly infinite ways a server might actually make the content of that response. It is likely that servers will write the responses to files and the URL returned to the client will be used to retrieve that file, but there's no requirement that servers do that. The only requirements on server are that:

1. The URL returned asserts, using the constraint expression syntax for async that the client accepts async responses.
2. The URL returned can be dereferenced and that operation will return the response requested by the client.

Response retrieval by the client

When a client requests an asynchronous result that is ready, the server MUST return a 200 (OK) HTTP status code and the resulting data response. If the client attempts to access the asynchronous result prior to it's availability, the server SHOULD return an HTTP response status of 409 (DAP Response Not Ready) along with the DAP Asynchronous Response Not Available XML document. If the server does not return the 409 response status then it MUST return a 404 (Not Found) response along with whatever document it deems fit as the response body.

If the client attempts to access the asynchronous result after it is no longer available, the server SHOULD return an HTTP response status of 410 (Gone) along with the DAP4 Asynchronous Response Gone document. If the server does not return the 410 response status then the server MUST return a 404 (Not Found) response along with whatever document it deems fit as the response body.

In each case above where the server SHOULD return a specific error code, but may return a 404 code instead, the intent is for servers to provide the most appropriate use of HTTP/1.1's error codes while also providing servers with an 'out' when that is hard for them to do. For example, knowing that a response, which is essentially ephemeral, is gone would, in theory, require to server to keep a record of every URL ever issued for an asynchronous response and that is not practical. At the same time, it is easy to see that a client would really like to know that the response has not yet been finished (i.e., it has not waited long enough) or that it is gone (i.e., it waited too long).

Detail: Client requests

DAP4 Constraint Expression extension for Async

By adding a keyword/value pair to the DAP4 constraint expression we can allow a client to encode it's willingness to accept an asynchronous response, along with the a maximum amount of time the client can wait before it can access the response.

async

A value of zero indicates the client is willing to unconditionally accept an asynchronous response. A positive integer value will be interpreted as the number of seconds that the client will wait for access to the response. If the value is negative the serve MUST return an error.

Examples

Client is willing to unconditionally accept an asynchronous response

?async=0

Client is willing to wait for 60 seconds for access to the asynchronous response

?async=60

X-DAP-Async-Accept

A client may indicate willingness to accept asynchronous responses by including the X-DAP-Async-Accept HTTP header. Clients can make conditional requests for asynchronous responses by indicating the maximum time they are willing to wait by using the X-DAP-Async-Accept HTTP header with a value given in seconds. A value of zero indicates that the client is willing to accept whatever delay the server may encounter.

Detail: Server responses

Several 'experimental' HTTP headers are used by this design. They convey information either in the request (like the X-DAP-Async-Accept described above) or they encode information for a response. While only clients that intend to support asynchronous responses need to understand all of these, every client SHOULD understand the X-DAP-Async-Required header. Because we need to support clients like web browsers, knowledge of that header is not required, but DAP4-specific clients will provide the most information to users if they know to look for at least that response header.

X-DAP-Async-Required

The X-DAP-Async-Required HTTP response header is included in the response if the request requires an asynchronous response and the client has not indicated willingness to accept such a response. Rejection of the request should also be indicated by the 400 DAP Asynchronous Response Required HTTP response code.

X-DAP-Async-Accepted

The X-DAP-Async-Accepted HTTP response header is included in the response if the server has accepted an asynchronous request. Acceptance of the request should also be indicated by the 202 Asynchronous Request Accepted HTTP response code.

HTTP Response Codes

HTTP provides a number of response codes beyond the simple 200 (OK), 404 (Not Found) and 500 (Internal Server Error). In this design we describe how those standard codes SHOULD be used by DAP4 servers. We don't enumerate all of the possible codes, instead opting for a description of those that most relevant.

202 Accepted

A server indicates that a request has been accepted and will be handled asynchronously by returning a '202 Accepted' HTTP response code. The response body must contain a document in one of the asynchronous information media types listed below. A server MUST return this response, and only do so, when a client has indicated a willingness to process an asynchronous response and the response will actually be returned using the asynchronous mechanism.

400 DAP4 Asynchronous Response Required

The '400 DAP Asynchronous Response Required' HTTP response code is used to indicate that the DAP4 request has been rejected because an asynchronous response is required and the client did not indicate willingness to accept an asynchronous response.

The response code text is used to indicate the reason for the rejection. However, since the '400' HTTP response code is not specific to asynchronous DAP (the standard text for the '400' code is "Bad Request"), the X-DAP-Async-Required HTTP response header is also included in the response (see above).

Note that a standard 400 HTTP response code is returned. In this way, a client that does not understand asynchronous DAP can fail gracefully. The response code text message has been changed to be more informative of the reason for the failure. For clients that are aware of asynchronous DAP, the "DAP-Async-Required" header is set to "true". The body of the response also returns some information the client can use to decide on how it will continue.

409 Conflict - DAP4 Response Not Ready

The '409 Conflict' HTTP response code MAY be returned by a server to indicate that the DA4P request has been rejected because a previous asynchronous request has not been completed and the result is not ready for access. If a server utilizes the '409 Conflict' HTTP response code it must also return a DAP4 Asynchronous Response Not Yet Available document in the response body.

410 Gone - DAP4 Response No Longer Available

The '410 Gone' HTTP response code MAY be used by a server to indicate that the result of an asynchronous request is no longer available. If a server utilizes the '410 Gone' HTTP response code it must also return a DAP4 Asynchronous Response Gone document in the response body.

412 Precondition Failed

The '412 Precondition Failed' HTTP response code is used to indicate that the DAP request has been rejected because it did not meet the X-DAP-Async-Accept condition (see above) that was specified in the request.

500 Internal Error

The '500 Internal Error' HTTP response code is used to indicate that the DAP request has caused an error on the server. The request body and other headers must be compliant with the DAP4 Error Response and Status Codes sections of the web services specification. The request should not be repeated.

Asynchronous Response Documents

The uses of these documents are:

• to inform clients that a request will result in an asynchronous response;
• to provide clients with the status of an an accepted asynchronous request; and
• to inform clients that a request for and asynchronous response has been rejected.

These response documents are the payloads to various responses, including errors. By using the HTTP 400-series error response codes, the design ensures that generic web clients will understand that their request was in error (even if they don't really understand why). The text provided with the response code will be sufficient that person could understand the gist of the problem, if not more. The response documents described here, along with the X-DAP describe above, are a way of providing additional information to a savvy client so that it can take full advantage of the synchronous response system.

These documents are XML that follows the DAP Asynchronous XML schema and are declared in the namespace http://opendap.org/ns/dap/asynchronous.

DAP4 Asynchronous Response Required

This document informs clients that a request will result in an asynchronous response, and that the client has not yet indicated it's willingness to accept an asynchronous response. It might seem superfluous to include a document that clearly only a client knowledgable about the asynchronous response features could parse, but many such clients may not, as a matter of course, indicate they will accept these responses. For example, a user-configurable parameter might be turn off support for the feature. The expectedDelay and responseLifetime elements convey information about conditions the clients can expect if it submits an asynchronous request for the response. As noted below, these are estimates made by the server since a number of things that the server cannot predict can affect them in the interleaving time between the client's requests. Additionally, a server MAY return values of zero for either of the values, indicating that it cannot make an accurate estimate.

<AsynchronousResponse status="required">
  <expectedDelay seconds="600" />
  <responseLifetime seconds="3600"/>
</AsynchronousResponse>

This response MUST be associated with the 400 HTTP response code and the X-DAP-Async-Required response header.

DAP4 Asynchronous Request Accepted

This response informs clients that a request resulting in an asynchronous response has been accepted, along with operational information about retrieving the asynchronous response result. Note that the expectedDelay and responseLifetime elements are an estimate by the server. A server SHOULD ensure that the response will remain available for the time period given by expectedDelay and responseLifetime. We say SHOULD and not MUST because we cannot predict all possible operational situations where these kinds of responses might be used. For example, a server might be providing access for several types of users who might have different access priorities, especially to limited resources like those typically involved with asynchronous access, and thus some responses might be further delayed, or removed early, to enable processing of requests from users with higher priority. It should be kept in mind, however, that the usefulness of the asynchronous responses will depend, in part, on servers providing a facility on which clients can depend.

While the expectedDelay and responseLifetime elements are required, a server MAY set their seconds attribute to 0 to indicate that it cannot provide a reliable value. In this case, clients SHOULD poll every 300 seconds and servers SHOULD expect this behavior. This is the default TCP user timeout period (see http://tools.ietf.org/html/rfc5482).

<AsynchronousResponse status="accepted">
  <expectedDelay seconds="600" />
  <responseLifetime seconds="3600"/>
  <link href="http://server.org/async/path/result" />
</AsynchronousResponse>

This response document MUST be associated with the 202 HTTP status code and the X-DAP-Async-Accepted response header.

DAP4 Asynchronous Response Not (Yet) Available

This document informs clients that a while a previous request for an asynchronous response has been accepted, the result is not available.

<AsynchronousResponse status="pending"/>

This response document MUST be associated with the 409 HTTP response code.

Servers SHOULD return this response document and it's associated HTTP status of 409, but servers MAY return any document in the response body along with either a a 404 (Not Found) or a 400 (Bad Request) HTTP status.

DAP4 Asynchronous Response Gone

This document informs clients that a while a previous request for an asynchronous response has been accepted, the result is no longer available.

<AsynchronousResponse status="gone"/>

This response document MUST be associated with the 410 HTTP status code.

Servers SHOULD return this response document and it's associated HTTP status of 410, but servers MAY return any document in the response body along with either a a 404 (Not Found) or a 400 (Bad Request) HTTP status.

DAP4 Asynchronous Request Rejected

This document informs clients that a request for an asynchronous response has been rejected, even though the client said it is willing to process an asynchronous response. There are at least as many reasons a server might reject the request for an asynchronous response as there are systems that might return such responses. However, this design provides suggested response codes for cases that seem likely so that clients can make educated decisions about the reason for the rejection. The reason codes supported are:

time

The client indicated that it was only willing to wait X seconds and the server thought it would take more time to build the result.

unavailable

A needed resource is not available. This might indicate that hardware, like a robot tape system, cannot be currently accessed.

privileges

The client is not allowed to make the request.

other

Self evident...

In addition to the reason codes, this response will contain a text description of the reason for rejection.

Servers SHOULD make every effort to use the correct reason codes and provide cogent descriptions.

<AsynchronousResponse status="rejected">
    <reason code="time"/>
    <description>Acceptable access delay was less than estimated delay.</description>
</AsynchronousResponse>

This response document MUST associated with the 412 HTTP status code.

Servers SHOULD return this response document along with an HTTP status of 412, but servers MAY return any document in the response body along with an HTTP status of 404 (Not Found) or of 400 (Bad Request) in its place.

DAP4 Error

If the server encounters an error it must MUST (MAY?) return an HTTP status of 500 (Internal Error) along with a request body and other headers compliant with the DAP4 Error Response and Status Codes sections of the web services specification. The request should not be repeated.

Examples

Constrained Data Request-Response using GET

Simple Request

GET /dap/path/data.nc?projection=x,y,temp HTTP/1.1
Host: server.org

If the server decides it needs to handle this request in an asynchronous manner, it will refuse the request because it did not say it would accept an asynchronous response.

Response

400 DAP Asynchronous Response Required
X-DAP-Async-Required: true
Content-Type: text/xml;charset=UTF-8
 
<AsynchronousResponse status="required">
  <expectedDelay seconds="600" />
  <responseLifetime seconds="3600"/>
</AsynchronousResponse>

Constrained Data Request-Response with DAP-Async-Accept Request Header

Request:

GET /dap/path/data.nc?projection=x,y,temp HTTP/1.1
Host: server.org
X-DAP-Async-Accept: 0

Alternately, this request would produce the same result using only the URL:

GET /dap/path/data.nc.dap?accept=0&projection=x,y,temp HTTP/1.1
Host: server.org

Response:

202 Accepted
Content-Type: text/xml;charset=UTF-8

<AsynchronousResponse status="accepted">
  <expectedDelay seconds="600" />
  <responseLifetime seconds="3600"/>
  <link href="http://server.org/async/path/result" />
</AsynchronousResponse>

NB: This example originally included an Accept header with the value of multipart/mixed. However, that is not a good example. The HTTP/1.1 specification says that when a specific media type is indicated as the only one acceptable, a server must return a 406 response code if it cannot return that media type. The meaning of Accept: */* is the same as not including the header, so I have removed the header from these examples. We need to be heads up in the ways that we suggest that header should be used by clients

Constrained Data Request-Response with conditional DAP-Async-Accept Request Headers

Request:

GET /dap/path/data.nc?projection=x,y,temp HTTP/1.1
Host: server.org
X-DAP-Async-Accept: 60

Alternately, this request would produce the same result using only the URL:

GET /dap/path/data.nc.dap?acceptAsync=60&projection=x,y,temp HTTP/1.1
Host: server.org

Response:

412 Precondition Failed
Content-Type: text/xml;charset=UTF-8
 
<AsynchronousResponse status="rejected">
    <reason code="time"/>
    <description>Acceptable access delay was less than estimated delay.</description>
</AsynchronousResponse>

Premature Request For Asynchronous Result

Request:

GET /async/path/data.nc?projection=x,y,temp HTTP/1.1
Host: server.org

Alternately, this request would produce the same result using only the URL:

GET /async/path/data.nc?projection=x,y,temp HTTP/1.1
Host: server.org

Response:

409 Conflict
Content-Type: text/xml;charset=UTF-8

<AsynchronousResponse status="pending"/>

Error Responses

DAP4 Error Response documents are XML documents that have a <dap4:Error> as their root element and that may contain any or all of the following inner elements:

<dap4:code> A numerically valued error code. This code must be associated with a protocol, such as HTTP, via the protocol attribute. It is not a requirement that the protocol over which the originating request and subsequent Error response where transmitted be the same as the protocol identified by the value of the protocol attribute.

<dap4:Message> A short informative text message describing the error.

<dap4:Context> Textual information describing the context in which the error occurred: position of a parse error in a constraint expression, for example.

<dap4:OtherInformation> Arbitrary additional text information: a Java stack trace, for example.

Error Response Resource Role

DAP4 Error Responses are identified by the resource role:

http://services.opendap.org/dap4/error

Normative Encoding of the Error Response

The normative XML representation for the Error Response is defined in Appendix x "Normative XML Encoding of the Error Response". The media type for the normative XML representation is:

application/vnd.opendap.dap4.error.xml

Examples

Not Found

<?xml version="1.0" encoding="UTF-8"?>
<dap4:Error xmlns:dap4="http://xml.opendap.org/ns/DAP/4.0#">
    <dap4:code protocol="http">404</dap4:code>
    <dap4:Message>Unable to locate requested resource</dap4:Message>
</dap4:Error>

Parse Error

<?xml version="1.0" encoding="UTF-8"?>
<dap4:Error xmlns:dap4="http://xml.opendap.org/ns/DAP/4.0#">
    <dap4:code protocol="http">400</dap4:code>
    <dap4:Message>Bad DAP4 Request Syntax</dap4:Message>
    <dap4:Context>
        The constraint expression "?u[3][lat<66]" failed to parse at the sub expression "lat<66". 
         Relational syntax expressions are not supported for array subsetting operations.
    </dap4:Context>
</dap4:Error>

References

1. Caron, J., Unidata's Common Data Model Version 4, 2012 (http://www.unidata.ucar.edu/software/netcdf-java/CDM/).

2. Folk, M. and E. Pourmal, HDF5 Data Model, File Format and Library — HDF5 1.6, Category: Recommended Standard January 2007 NASA Earth Science Data Systems Recommended Standard ESDS-RFC-007, 2007 (http://earthdata.nasa.gov/sites/default/files/esdswg/spg/rfc/ese-rfc-007/ESDS-RFC-007v1.pdf).

3. Gallagher J., N. Potter, T. Sgouros, S. Hankin, and G. Flierl, The Data Access Protocol—DAP 2.0, NASA Earth Science Data Systems Recommended Standard ESE-RFC-004.1.2 (http://opendap.org/pdf/ESE-RFC-004v1.2.pdf).

4. Gosling, J., B. Joy, G. Steele, G. Bracha, A Buckley, The Java™ Language Specification — 7th Editition Oracle Corporation, 2012, (http://docs.oracle.com/javase/specs/jls/se7/html/).

5. Hartnett, E., netCDF-4/HDF5 File Format, NASA Earth Science Data Systems Recommended Standard ESDS-RFC-022, 2011 (http://earthdata.nasa.gov/sites/default/files/field/document/ESDS-RFC-022v1.pdf).

6. IEEE, IEEE Standard for Binary Floating-Point Arithmetic, ANSI/IEEE Std 754-1985, Digital Object Identifier: 10.1109/IEEESTD.1985.82928, 1985.

7. The Internet Society, IETF RFC 2119: Key words for use in RFCs to Indicate Requirement Levels , 1997 (http://tools.ietf.org/html/rfc2119).

8. The Internet Society, IETF RFC 2396: Uniform Resource Identifiers (URI): Generic Syntax , 1998 (http://tools.ietf.org/html/rfc2396).

9. The Internet Society, IETF RFC 2616: Hypertext Transfer Protocol — HTTP/1.1 , 1999 (http://tools.ietf.org/html/rfc2616).

10. The Internet Society, IETF RFC 4506: XDR: External Data Representation Standard, 2006 (http://tools.ietf.org/html/rfc4506).

11. ISO/IEC, Information technology — Portable Operating System Interface (POSIX) — Part 2: Shell and Utilities, ISO/IEC 9945-2,1993 (http://www.iso.org/iso/catalogue_detail.htm?csnumber=17841).

12. The Open Geospatial Consortium Inc., Abstract Specifications, (http://www.opengeospatial.org/standards/as).

13. The Organization for the Advancement of Structured Information Standards, RELAX NG Specification, Committee Specification: 2001, J. Clark, M. Makoto (eds.) (http://relaxng.org/spec-20011203.html).

14. The Unicode Consortium. The Unicode Standard, Version 6.2.0, ISBN 978-1-936213-07-8, 2012.

15. Unidata, CF Metadata, (http://www.cfconventions.org/).

16. W3C, Extensible Markup Language (XML) 1.0, T. Bray, J. Paoli, C. M. Sperberg-McQueen, E. Maler, F. Yergeau (eds.), Fifth Edition. 2008 (http://www.w3.org/TR/2008/REC-xml-20081126/).

17. World Meteorological Organization, FM 92 GRIB, edition 2, version 2, 2003 (http://www.wmo.int/pages/prog/www/DPS/FM92-GRIB2-11-2003.pdf).007

Appendices

FQN Syntax

An FQN has two parts. First, there is the path, which refers to Group traversal, and second is the suffix, which refers to the traversal of Structures. An FQN is the concatenation of the path with the suffix and separated by the '/' Character. The suffix may not exist if O is a group or is not a Structure typed variable.

Fully qualified names conform to the following syntax.

FQN:   grouppath
     | grouppath '/' name
     | grouppath '/' structurepath
     | grouppath '/' structurepath '.' name

grouppath: /*empty*/ | grouppath '/' groupname

structurepath: /*empty*/ | structurepath '.' structname

To write a path for an object O, follow these steps.

1. Locate the closest enclosing group G for O. If O is a group, then O and G will be the same.
2. Create the scope prefix for O by traversing a path through the Group tree, starting with the Dataset and continuing down to and including G. Concatenate the group names on that path and separating them with '/'. The name for Dataset is ignored, hence the FQN will begin with "/".

If O is not a Structure typed variable, then we are done and the FQN for O is just the path. Otherwise, the suffix must be computed as follows.

1. Traverse the nested Structure declarations from G to O, including O, but not including G in the path. Traversal here means following the enclosing Structure typed variables until O is reached.
2. Concatenate the names on that suffix path and separating them with '.' to create a suffix.
3. Create the final FQN as the concatenation of the path, the character '/', and the suffix.

DAP4 Lexical Elements

This section describes the lexical elements that occur in the DAP4 DMR.

Within the RELAXNG DAP4 grammar (Section 13) there are markers for occurrences of primitive type such as integers, floats, or strings (ignoring case). The markers typically look like this when defining an attribute that can occur in the DAP4 DMR.

<attribute name="Principal_Investigator">
<datatype="dap4_string"/>
</attribute>

The "<data type="dap4_string"/>" specifies the lexical class for the values that this attribute can have. In this case, the "Principal_Investigator" attribute is defined to have a DAP4 string value. Similar notation is used for values occurring as text within an xml element.

The lexical specification later in this section defines the legal lexical structure for such items. Specifically, it defines the format of the following lexical items.

1. Constants, namely: string, float, integer, character, and opaque.
2. Identifiers
3. Fully qualified names (also referred to as FQNs) (Section 5.3).

The specification is written using the extended POSIX regular expression notation [11] with some additions.

1. Names are assigned to regular expressions using the notation "name = {regular expression}"
2. Named expressions can be used in subsequent regular expressions by using the notation "{name}". Such occurrences are equivalent to textually substituting the expression associated with name for the "{name}" occurrence.

Notes:

1. The definition of {UTF8} is deferred to the next section.
2. Comments are indicated using the "//" notation. Standard xml escape formats (&x#DDD; or &{name};) are assumed to be used as needed.

Basic character set definitions

CONTROLS   = [\x00-\x1F] // ASCII control characters

WHITESPACE = [ \r\n\t\f]+

HEXCHAR    = [0-9a-zA-Z]

// ASCII printable characters

ASCII = [0-9a-zA-Z !"#$%&'()*+,-./:;<=>?@[\\\]\\^_`|{}~]

Ascii characters that may appear unescaped in Identifiers

This is assumed to be basically all ASCII printable characters except these characters: '.', '/', '"', ''', and '&'. Occurrences of these characters are assumed to be representable using the standard xml &{name}; notation (e.g. &amp;). In this expression, backslash is interpreted as an escape character.

IDASCII=[0-9a-zA-Z!#$%()*+:;<=>?@\[\]\\^_`|{}~]

The Numeric Constant Classes: integer and float

INTEGER    = {INT}|{UINT}|{HEXINT}

INT        = [+-][0-9]+{INTTYPE}?

UINT       = [0-9]+{INTTYPE}?

HEXINT     = {HEXSTRING}{INTTYPE}?

INTTYPE    = ([BbSsLl]|"ll"|"LL")

HEXSTRING  = (0[xX]{HEXCHAR}+)

FLOAT      = ({MANTISSA}{EXPONENT}?)|{NANINF}

EXPONENT   = ([eE][+-]?[0-9]+)

MANTISSA   = [+-]?[0-9]*\.[0-9]*

NANINF     = (-?inf|nan|NaN)B.1.4 The String Constant Class

STRING     = ([^"&<>]|{XMLESCAPE})*

CHAR       = ([^'&<>]|{XMLESCAPE})

URL        = (http|https|[:][/][/][a-zA-Z0-9\-]+
             ([.][a-zA-Z\-]+)+([:][0-9]+)?
             ([/]([a-zA-Z0-9\-._,'\\+%)*
             ([?].+)?([#].+)?

The String/URL Constant Class

STRING = "\({SIMPLESTRING}{ESCAPEDQUOTE}?\)*"
SIMPLESTRING = [^"\\]
ESCAPEDQOTE=\\"

The Opaque Constant Class

OPAQUE = 0x([0-9A-Fa-f] [0-9A-Fa-f])+

There is a semantic constraint that if there is an odd number of hex digits in the opaque constant, a zero hex digit will be added to the end to ensure that the constant represents a set of 8-bit bytes.

The Identifier Class

ID         = {IDCHAR}+

IDCHAR     = ({IDASCII}|{XMLESCAPE}|{UTF8})

XMLESCAPE  = [&][#][0-9]+;

The Atomic Type Class

ATOMICTYPE =   Char | Byte
             | Int8 | UInt8 | Int16 | UInt16
             | Int32 | UInt32 | Int64 | UInt64
             | Float32 | Float64
             | String | URL
             | Enum
             | Opaque ;

This list should be consistent with the atomic types in the grammar.

The Fully Qualified Name Class

FQN      = ([/]{EID})+([.]{EID})*
EID      = {EIDCHAR}+
EIDCHAR  =  ({EIDASCII}|{XMLESCAPE}|{UTF8})
EIDASCII = [0-9a-zA-Z!#$%()*+:;<=>?@\[\]\\^_`|{}~]

This should be consistent with the definition in Section 5.3.

DAP4 Type Definitions

The RELAXNG [13] grammar references the following specific types. For each type, the following table give the lexical format as defined by the patterns previously given or by specific patterns as listed.

RELAXNG Data Type Name	Lexical Pattern
dap4_integer	{INTEGER}
dap4_float	{FLOAT}
dap4_char	{CHAR}
dap4_string	{STRING}
dap4_opaque	{OPAQUE}
dap4_vdim	[*]
dap4_id	{ID}
dap4_fqn	{FQN}
dap4_uri	{URL}
dap4_dim	[0-9]+

Note that the above lexical element classes are not disjoint. The type element "<datatype=.../>" should be sufficient to interpret the type within the DMR.

UTF-8

The UTF-8 specification [14] defines several ways to validate a UTF-8 string of characters.

The full (most correct) validating version of UTF8 character set is as follows.

UTF8 =   ([\xC2-\xDF][\x80-\xBF])
       | (\xE0[\xA0-\xBF][\x80-\xBF])
       | ([\xE1-\xEC][\x80-\xBF][\x80-\xBF])
       | (\xED[\x80-\x9F][\x80-\xBF])
       | ([\xEE-\xEF][\x80-\xBF][\x80-\xBF])
       | (\xF0[\x90-\xBF][\x80-\xBF][\x80-\xBF])
       | ([\xF1-\xF3][\x80-\xBF][\x80-\xBF][\x80-\xBF])
       | (\xF4[\x80-\x8F][\x80-\xBF][\x80-\xBF])

The lines of the above expression cover the UTF-8 characters as follows:

1. non-overlong 2-byte
2. excluding overlongs
3. straight 3-byte
4. excluding surrogates
5. straight 3-byte
6. planes 1-3
7. planes 4-15
8. plane 16

Note that values from 0 through 127 (ASCII and control characters) are not included in this any of these definitions.

The above reference also defines some alternative regular expressions.

There is what is termed the partially relaxed version of UTF8 defined by this regular expression.

UTF8 =    ([\xC0-\xD6][\x80-\xBF])
        | ([\xE0-\xEF][\x80-\xBF][\x80-\xBF])
        | ([\xF0-\xF7][\x80-\xBF][\x80-\xBF][\x80-\xBF])

Second, there is what is termed the most-relaxed version of UTF8 defined by this regular expression.

UTF8 = ([\xC0-\xD6]...)|([\xE0-\xEF)...)|([\xF0 \xF7]...)

Any conforming DAP4 implementation MUST use at least the most-relaxed expression for validating UTF-8 character strings, but MAY use either the partially-relaxed or the full validation expression.

DAP4 Error Response Format

The Error Response is defined to be an XML document with media type application/vnd.org.opendap.dap4.error.xml. The specific format of the error response is defined in this document: http://docs.opendap.org/index.php/DAP4_Web_Services_v3#DAP4_Error_Response

DAP4 DMR Syntax as a RELAX NG Schema

The RELAX NG grammar for the DMR currently resides at this URL. https://scm.opendap.org/svn/trunk/dap4/dap4.rng

DAP4 Dataset Services Response: Examples and XML Schema

Example: A fully featured and annotated DSR document

This example contains all of the optional components for a hypothetical DAP4 server that also supports DAp2 requests, server side functions, and asynchronous transactions.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<?xml-stylesheet type="text/xsl" href="/opendap/xsl/serviceDescription.xsl"?>
<DatasetServices xmlns="http://xml.opendap.org/ns/DAP/4.0/dataset-services#" 
                 xml:base="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc">

    <Description>Our friend fnoc1.nc</Description>

    <!-- ##################################################################################### -->
    <!--       DAP versions this server supports                                               --> 
    <!-- ..................................................................................... -->
    <DapVersion>4.0</DapVersion>
    <DapVersion>3.2</DapVersion>
    <DapVersion>2.0</DapVersion>
    
    <!-- ##################################################################################### -->
    <!--  The software version of the server that is providing the DAP4 Dataset Services       -->
    <!--   response.                                                                           --> 
    <!-- The ServerSoftwareVersion element may contain text, or any XML element content as     -->
    <!-- long as the XML is not in the document namespace                                      -->
    <!-- ..................................................................................... -->
    <ServerSoftwareVersion>Hyrax-2.7.9</ServerSoftwareVersion>
    
    
    <!-- ##################################################################################### -->
    <!--       Required DAP4 Services                                                          --> 
    <!-- ..................................................................................... -->

    <Service title="DAP4 Dataset Services" role="http://services.opendap.org/dap4/dataset-services">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_Dataset_Services_Description_Service">An index of the Services available for this data resource.</Description>

        <link description="Normative form of the DSR" 
              type="application/vnd.opendap.org.dataset-services+xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc">
              <alt type="text/html"/>
              <alt type="text/xml"/>
        </link>
        
        <link description="HTML representation of the DSR"  
              type="text/html" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.html"/>
              
        <link description="Normative DSR with generic Content-Type" 
              type="text/xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.xml"/>
              
    </Service>

    <Service title="DAP4 Dataset Metadata"  role="http://services.opendap.org/dap4/dataset-metadata">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_Dataset_Service_-_The_metadata">The DAP4 metadata content for this data resource..</Description>

        <link description="Normative form of the DMR" 
              type="application/vnd.org.opendap.dap4.dataset-metadata+xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr">
              <alt type="text/html"/>
              <alt type="text/xml"/>
              <alt type="application/rdf+xml"/>
        </link>
        
        <link description="Data Request Form" 
              type="text/html" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr.html"/>
              
        <link description="Normative DMR with generic Content-Type" 
              type="text/xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr.xml"/>
              
        <link description="RDF representation of DMR" 
              type="application/rdf+xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr.rdf"/>
    </Service>

    <Service title="DAP4 Data" role="http://services.opendap.org/dap4/data">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_Data_Service">DAP4 Data object for this data resource.</Description>
        <link description="The normative form of the DAP4 Data Response"
              type="application/vnd.org.opendap.dap4.data"  
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dap" >
              <alt type="text/plain"/>
              <alt type="text/xml"/>
              <alt type="application/x-netcdf"/>
        </link>
        
        <link description="A comma separated values (CSV) representation of the DAP4 Data Response object."
              type="text/plain"  
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dap.ascii" />
        <link description="XML representation of the DAP4 Data Response object."
              type="text/xml"   
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dap.xml" />
        <link description="NetCDF-3 representation of the DAP4 Data Response object."
              type="application/x-netcdf"   
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dap.nc" />
    </Service>

    <!-- ##################################################################################### -->
    <!--   Optional DAP4 Related Services                                                      --> 
    <!-- ..................................................................................... -->

    <Service title="ISO-19115 Metadata Service" role="http://services.opendap.org/dap4/iso-19115">
        <link description="Dataset metadata as ISO-19115" 
              type="text/xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr.iso">
              <alt type="text/html"/>
        </link>
        <link description="ISO-19115 conformance score" 
              type="text/html" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dmr.rubric">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_ISO_Conformance_Score_Service">ISO-19115 conformance score for the DMR.</Description>
    </Service>

    <Service title="File Access" role="http://services.opendap.org/dap4/file">
        <link type="text/xml ## This value depends on the file being accessed."  
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.file">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_Native_File_Access_Service">Access to dataset file.</Description>
    </Service>



    <!-- ##################################################################################### -->
    <!--       DAP2 Services                                                                   --> 
    <!-- ..................................................................................... -->
    <Service title="DAP2 Data" role="http://services.opendap.org/dap2/dods">
        <link type="application/octet-stream" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dods">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP2:_Data_Service">DAP2 Data Object.</Description>
    </Service>

    <Service title="DDX" role="http://services.opendap.org/dap2/ddx">
        <link type="text/xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.ddx">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP2:_DDX_Service">OPeNDAP Data Description and Attribute XML Document.</Description>
    </Service>

    <Service title="DDS" role="http://services.opendap.org/dap2/dds">
        <link type="text/plain" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.dds">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP2:_DDS_Service">OPeNDAP Data Description Structure.</Description>
    </Service>

    <Service title="DAS" role="http://services.opendap.org/dap2/das" >
        <link type="text/plain" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.das">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP2:_DAS_Service">OPeNDAP Dataset Attribute Structure.</Description>
    </Service>


    <Service title="INFO" role="http://services.opendap.org/dap2/info">
        <link type="text/html" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.info">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP2:_Info_Service">OPeNDAP Dataset Information Page.</Description>
    </Service>

    <Service title="Server Version" role="http://services.opendap.org/dap4/version">
        <link type="text/xml" 
              href="http://test.opendap.org:8090/opendap/hyrax/data/fnoc1.nc.ver">
        <Description href="http://docs.opendap.org/index.php/DAP4_Web_Services#DAP4:_Server_Version_Service">An XML document containing information about the software version of the server..</Description>
    </Service>


    <!-- ##################################################################################### -->
    <!--       Server Side Functions                                                           --> 
    <!-- ..................................................................................... -->


    <function name="geogrid" role="http://services.opendap.org/dap4/server-side-function/geogrid">
        <Description  href="http://docs.opendap.org/index.php/Server_Side_Processing_Functions#geogrid">Allows a DAP Grid variable to be sub-sampled using georeferenced values.</Description>
    </function>

    <function name="grid" role="http://services.opendap.org/dap4/server-side-function/grid">
        <Description  href="http://docs.opendap.org/index.php/Server_Side_Processing_Functions#grid">Allows a DAP Grid variable to be sub-sampled using the values of the coordinate axes.</Description>
    </function>

    <function name="linear_scale" role="http://services.opendap.org/dap4/server-side-function/linear-scale">
        <Description  href="http://docs.opendap.org/index.php/Server_Side_Processing_Functions#linear_scale">Applies a linear scale transform to the named variable.</Description>
    </function>

    <function name="version" role="http://services.opendap.org/dap4/server-side-function/version">
        <Description  href="http://docs.opendap.org/index.php/Server_Side_Processing_Functions#version">Returns version information for each server side function.</Description>
    </function>
   
    <!-- ##################################################################################### -->
    <!--       Server Side Function Group                                                      --> 
    <!-- ..................................................................................... -->

 
    <functionGroup name="ferret" role="http://pmel.noaa.gov/dap4extension/ferret" >
        <Description href="http://ferret.pmel.noaa.gov/Ferret/documentation">This server supports ferret functions.</Description>
    </functionGroup>

    <!-- ##################################################################################### -->
    <!--       Extensions                                                                      --> 
    <!-- ..................................................................................... -->

    <extension name="async" role="http://opendap.org/extension/async">
        <Description href="http://docs.opendap.org/index.php/DAP4:_Asynchronous_Request-Response_Proposal_v3">The server supports asynchronous transactions.</Description>
    </extension>
    
    
</DatasetServices>

DSR XML Schema

I believe that this schema is correct, but it rigidly enforces the order of the content. We may want to rewrite to be more lenient. ndp 16:02, 22 November 2013 (PST)

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema 
    targetNamespace="http://xml.opendap.org/ns/DAP/4.0/dataset-services#"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:xml="http://www.w3.org/XML/1998/namespace"
    xmlns:dsr="http://xml.opendap.org/ns/DAP/4.0/dataset-services#"
    xmlns="http://xml.opendap.org/ns/DAP/4.0/dataset-services#"
    elementFormDefault="qualified" 
    attributeFormDefault="unqualified">
    
    <xs:import  namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="http://www.w3.org/2001/03/xml.xsd"/>
    <!--
	
    -->
    <xs:element name="DatasetServices" type="DatasetServicesType"/>
    <xs:element name="DapVersion" type="DapVersionType"/>
    <xs:element name="ServerSoftwareVersion" type="AnyContentType"/>
    <xs:element name="Service" type="ServiceType"/>
    <xs:element name="Description" type="DescriptionType"/>
    <xs:element name="link" type="linkType"/>
    <xs:element name="alt" type="altType"/>
    <xs:element name="function" type="ExtensionType"/>
    <xs:element name="functionGroup" type="ExtensionType"/>
    <xs:element name="extension" type="ExtensionType"/>
    <!--
	
    -->
    <xs:complexType name="DatasetServicesType"> 
        <xs:annotation>
            <xs:documentation>DatasetServices root element type</xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:element ref="DapVersion" minOccurs="1" maxOccurs="unbounded"/>
            <xs:element ref="ServerSoftwareVersion" minOccurs="1" maxOccurs="1"/>
            <xs:element ref="Description" minOccurs="0" maxOccurs="1"/>
            <xs:element ref="Service" minOccurs="3" maxOccurs="unbounded"/>
            <xs:element ref="function" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element ref="functionGroup" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element ref="extension" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute ref="xml:base"  use="required"/>
    </xs:complexType>
    <!--
	
     -->
    <xs:complexType name="ServiceType"> 
        <xs:annotation>
            <xs:documentation>DatasetServices Service type</xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:element ref="Description" minOccurs="0" maxOccurs="1"/>
            <xs:element ref="link" minOccurs="1" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute name="title" type="xs:string" use="optional"/>
        <xs:attribute name="role" type="xs:anyURI" use="required"/>
    </xs:complexType>
    <!--
        
    --> 
    <xs:simpleType name="DapVersionType">
        <xs:restriction base="xs:string">
            <xs:pattern value="\d{1,}.\d{1,}"></xs:pattern>
        </xs:restriction>
    </xs:simpleType>
    <!--
        
    -->
    <xs:complexType name="DescriptionType">
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute name="href" type="xs:anyURI"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
    <!--
        
    -->
    <xs:complexType name="altType">
        <xs:attribute name="type" type="xs:string"/>
    </xs:complexType>
    <!--	
        
    -->
    <xs:complexType name="linkType"> 
        <xs:annotation>
            <xs:documentation>DatasetServices link type</xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:element ref="alt" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
        <xs:attribute name="href" type="xs:string" use="required"/>
        <xs:attribute name="type" type="xs:string" use="required"/>
        <xs:attribute name="description" type="xs:string" use="optional"/>         
    </xs:complexType>
    <!--
	
    -->
    <xs:complexType  name="AnyContentType" mixed="true">
        <xs:annotation>
            <xs:documentation>
                An element of this type may contain arbitrary XML or text content.
                The resulting content is may be ignored by DAP software. Other software
                might find the information useful. The XML elements must
                satisfy the requirements for 'lax' processing under schema 1.0.
                In practice, that means just about anything.
                ( <xs:any/>+ ) 
            </xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:any namespace="##any" minOccurs="0" processContents="lax"/>
        </xs:sequence>
        <xs:anyAttribute processContents="lax" namespace="##any"/>
    </xs:complexType>   
    <!--
	
    -->
    <xs:complexType name="ExtensionType"> 
        <xs:annotation>
            <xs:documentation>Server Extension Type</xs:documentation>
        </xs:annotation>
        <xs:sequence>
            <xs:element ref="Description" minOccurs="1" maxOccurs="1"/>
        </xs:sequence>
        <xs:attribute name="name" type="xs:string" use="required"/>
        <xs:attribute name="role" type="xs:anyURI" use="required"/>
    </xs:complexType>
    <!--
	
     -->
    
</xs:schema>

DAP4 Asynchronous Response Schema

DAP4 Error Response Schema

The normative XML representation for the Error Response is defined by the following RELAX-NG schema.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:doc="http://www.example.com/annotation" datatypeLibrary="http://xml.opendap.org/datatypes/dap4" ns="http://xml.opendap.org/ns/DAP/4.0#">
    <start>
        <ref name="errorresponse"/>
    </start>
    <define name="errorresponse">
        <element name="Error">
            <optional>
                <interleave>
                    <element name="code">
                        <attribute name="protocol">
                            <data type="text"/>
                        </attribute>
                        <data type="dap4_integer"/>
                    </element>
                    <element name="Message">
                        <text/>
                    </element>
                    <element name="Context">
                        <text/>
                    </element>
                    <element name="OtherInformation">
                        <text/>
                    </element>
                </interleave>
            </optional>
        </element>
    </define>
</grammar>