DataDDX: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
No edit summary
Line 33: Line 33:
</pre>
</pre>


=== Example DataDDX response ===
=== Example DataDDX response Sent via HTTP 1.1 ===


<pre>
<pre>

Revision as of 00:12, 11 December 2008

  • Use a multipart MIME document to hold the DDX and one data blob per DDX. Adopt the same use of Multipart. MIME as WCS uses.
  • consider the following design idea for embedding type information within the data stream:

In DAP2, the DDS is used as a descriptive header for data. For gridded data that works OK but for point data it doesn't work so well. When the data are read as from a stream, it's OK, but when the data are read and stored, then used, they need to be in a linked structure. Because the DDS contains the data type definition and not the value, it does not hold a structure suitable for holding values. Look at how the d_values field in Sequence works (see Sequence::print_val(), deserialize()). A better approach would be to encode the type of a variable in the data stream itself and have the reader (client most of the time) build instances as needed. For arrays, structures and simple types this is mostly a wash, but for Sequences it would be a major plus because the protocol could support sequences of complex objects more easily. It would also get rid of the odd situation where a DDS holds a type definition for a nested sequence while the top-most sequence holds the tree of objects which hold values. (I.E. the child sequences in the DDS don't hold data at all).

As an alternative, suppose we build a data response using the DDX in a multipart document and then encoded type information in the data stream as well? This would provide a way to bundle attributes with variables in the data response and locate type information with the data values (for sequences mostly).

Normative References

Multipart MIME

Xlink

Using Multipart MIME for the DataDDX response

The DataDDX response's network representation will be as a Multipart MIME document with two parts: One part that contains a DDX that contains zero or more variables; and one part that contains zero or more bytes of XDR-encoded data which corresponds to the variables declared in the DDX. The Data element in the DDX holds an xlink reference to this second part.

The DataDDX may be empty to account for cases where a dataset contains only type definitions, something that never happens now but which is an emerging feature of both HDF5 and NetCDF4.

The DataDDX will always have two parts, even if the second 'data' part is empty so that processing software can always assume that a DataDDX will occupy two parts of a multipart MIME document.

The Data element is used to link the DDX, in one part, to the data values, in another part, so that other protocols (e.g., DAP-SOAP) can package several responses in one document easily. That is, while this design does not provide for that capability, it is easily extensible to one that does.

Adding the Data element

In addition to the multipart MIME document that holds the two parts of the data response, the Data element holds a reference to the 'data part' of the response. Here's a sample Data element:

<Data xmlns:xlink="http://www.w3.org/XML/1999/xlink" 
      xlink:href="cid:6efa6ea4:98eda872192:-1ed1" xlink:type="simple"/>

Example DataDDX response Sent via HTTP 1.1

 HTTP/1.1 200 OK
 Server: Apache-Coyote/1.1
 Content-Type: multipart/related; type="text/xml"; start="<080B6DC4AC8AF0C43041C57CE8DE9646>"; boundary="--mimepart_7_9651610.1145395859678"
 Date: Tue, 18 Apr 2006 21:30:59 GMT
 XDAP: 3.2
 Connection: close
 
 --mimepart_7_9651610.1145395859678
 Content-Type: text/xml; charset=UTF-8
 Content-Transfer-Encoding: binary
 Content-Id: <080B6DC4AC8AF0C43041C57CE8DE9646>
 
 <?xml version="1.0" encoding="UTF-8"?>
 
     <Dataset name="fnoc1.nc" 
              xmlns="http://xml.opendap.org/ns/DAP/3.2#"
              xmlns:xml="http://www.w3.org/XML/1998/namespace"  
              xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:schemaLocation="http://xml.opendap.org/ns/DAP/3.2#  http://xml.opendap.org/dap/dap3.2.xsd"
              xml:base="http://test.opendap.org/dap/data/nc/fnoc1.nc.ddx">

        <Data xmlns:xlink="http://www.w3.org/XML/1999/xlink" 
              xlink:href="cid:6efa6ea4:98eda872192:-1ed1" xlink:type="simple"/>

        <Attribute name="NC_GLOBAL" type="Container">
	     <Attribute name="base_time" type="String">
	         <value>"88- 10-00:00:00"</value>
	     </Attribute>
	     <Attribute name="title" type="String">
	         <value>" FNOC UV wind components from 1988- 10 to 1988- 13."</value>
	     </Attribute>
	 </Attribute>
	 <Attribute name="DODS_EXTRA" type="Container">
	     <Attribute name="Unlimited_Dimension" type="String">
	         <value>"time_a"</value>
	     </Attribute>
	 </Attribute>
	 <Array name="v">
	     <Attribute name="units" type="String">
		 <value>"meter per second"</value>
	     </Attribute>
             <Attribute name="long_name" type="String">
		 <value>"Vector wind northward component"</value>
	     </Attribute>
	     <Attribute name="missing_value" type="String">
		 <value>"-32767"</value>
	     </Attribute>
	     <Attribute name="scale_factor" type="String">
		 <value>"0.005"</value>
	     </Attribute>
	     <Int16/>
	     <dimension name="time_a" size="16"/>
	     <dimension name="lat" size="17"/>
	     <dimension name="lon" size="21"/>
	 </Array>
     </Dataset>
 </xml>

 --mimepart_7_9651610.1145395859678
 Content-Type: application/octet-stream
 Content-Transfer-Encoding: binary
 Content-Id: 6efa6ea4:98eda872192:-1ed1
   
   Here be the XDR encoded binary stuff that is the data from the GetDATA request
   
 --mimepart_7_9651610.1145395859678--