DAP4 Web Services - StartingPoint: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
 
(44 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:Development|Development]] [[Category:DAP4|DAP4]]
<font size="+1" color="red">This is an old document that captures the starting point of the OPULS design work. It's out of date and should be referenced only as a baseline for the work.</font>
[[OPULS_Development | <-- back to OPULS Development]]
Author: [[User:Jimg|Jimg]], NDP, ?
== Protocol Independence ==
== Protocol Independence ==


Line 5: Line 12:
'''Discussion''':  
'''Discussion''':  


# Does this matter? Is it something to strive for? Is it a Must?  Is this just noise??
# Does this matter?
# Can the clients really make a request that's protocol independent?  
# Would we need to add new syntax server-side functions etc. to the protocol to enable this?
# Would we need to add new syntax server-side functions etc. to the protocol to enable this?


Line 17: Line 23:
In practice these have traditionally been implemented over HTTP. However they could just as easily be pushed over a different protocol, as long as the usage of the URL components remain consistent.  
In practice these have traditionally been implemented over HTTP. However they could just as easily be pushed over a different protocol, as long as the usage of the URL components remain consistent.  


==  Services  ==
A Service is represented by:
* A unique "namespace" like identifier that is used to define an xlink:role attribute.
* A simple human readable name called 'title'.
* An access URL.
* An optional description.
A service is accessed by dereferencing its access URL, which is typically constructed by adding some type of suffix to  the dataset's referent (aka base) URL. The way in which the query string (aka constraint expression) is used is defined by each service, and there is no requirement for inter-service query string API conformity.


=== Primary DAP4 Services ===


== Required Responses ==


=== Service Description ===
==== [[DAP Service Terminus  | DAP4: Dataset Services Description Service]] ====
The Dataset Services Service provides a listing of the various services available for a dataset. Dereferencing the base URL for a data set accessed through a DAP4 web server will return a ''service document'' that describes the various responses possible for that data set, including URLs that can be used to access those responses.


The base URL for a data set accessed through a DAP4 web server will return a ''service document'' that describes the various responses possible for that data set, including URLs that can be used to access those responses.
More information on the Service Description response can be found on the [[DAP Service Terminus]] page.


[[DAP Service Terminus]]


service url =  <font size="2"><code>'''dataset_url'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/dataset-services#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:  
====  '''[[DAP4: Dataset Service | DAP4: Dataset Service - The metadata]]'''  ====
:<font size="2"><code>baseURL</code></font>


=== Dataset: The metadata ===


The Dataset response is an XML document containing a dap:Dataset object as it's root element. It contains the persistent [[DAP4: Data Model]] representation of the dataset held at the server. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set.


Also: [[DAP4:_Responses#Transmitting_Attributes_in_constrained_Dataset_documents | All dap:Attribute objects have been removed from constrained dap:Dataset objects.]]
The Dataset Service provides a metadata description of the dataset. The Dataset response is an XML document that contains both the 'syntactic' (structural) and 'semantic' metadata for the dataset, persisted as a [[DAP4: Data Model]] representation of the dataset held at the server. The Dataset service accepts a query string (constraint expression) that allows you to inspect the effects on the data structures when sub-setting and/or server side functions are applied. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set.
Note that [[DAP4:_Responses#Transmitting_Attributes_in_constrained_Dataset_documents | All dap:Attribute objects have been removed from constrained dap:Dataset objects.]]


More information on the Dataset response can be found on the [[DAP4: Responses#Dataset Response | DAP4: Responses]] page.
* More information on the Dataset response can be found on the [[DAP4: Responses#Dataset Response | DAP4: Responses]] page.


More information on the syntax of DAP4 constraint expressions can be found on [[DAP4:_Data_Model#Constraint_Expressions | DAP4: Data Model]] page.
* More information on the syntax of DAP4 constraint expressions can be found on [[DAP4:_Data_Model#Constraint_Expressions | DAP4: Data Model]] page.




The standard URL for this response is:
:<font size="2"><code>baseURL + '.xml' + [?dap_constraint]</code></font>


=== Data: The data response for the dataset. ===
suffix = <font size="2"><code>'''.xml'''</code></font><br/>
service url = <font size="2"><code>'''dataset_url + '.xml' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/dataset#</nowiki>'''</code></font><br/><br/>


A Data response is the way DAP4 returns data to a client. Each Data response is returned over the wire as a multipart MIME document where the first MIME part contains the ''constrained'' Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR, for each variable in the dataset.
==== '''[[DAP4: Data Service ]]''' ====
The Data Service provides DAP4 data access to a dataset, and is the way that DAP4 returns data to a client. The Data service accepts a query string (constraint expression) which allows you to subset the data and invoke server side functions. When the service is invoked it returns over the wire as a multipart MIME document where the first MIME part contains the ''constrained'' Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR,  followed by a checksum, for each variable in the dataset.


More information on the Data response can be found on the [[DAP4:_Responses#Data_Response | DAP4: Responses]] page.
More information on the Data response can be found on the [[DAP4:_Responses#Data_Response | DAP4: Responses]] page.


The standard URL for this response is:
:<font size="2"><code>baseURL + '.dap' + [?dap_constraint]</code></font>


=== Data + Checksum: Pithy title referring to the use case... ===


A Data response is the way DAP4 returns data to a client. Each Data response is returned over the wire as a multipart MIME document where the first MIME part contains the ''constrained'' Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR, followed by a checksum, for each variable in the dataset.
suffix = <font size="2"><code>'''.dap'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + '.dap' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/data#</nowiki>'''</code></font><br/><br/>
 
=== Other DAP4 Services ===


More information on the Data response can be found on the [[DAP4:_Responses#Data_Response | DAP4: Responses]] page.


The standard URL for this response is:  
==== [[DAP4: HTML DATA Request Form Service]] ====
:<font size="2"><code>baseURL + '.cdap' + [?dap_constraint]</code></font>
The HTML DATA Request Form Service provides browser based access to the Dataset. When invoked it returns a web-browser renderable document (in html) that provides a form (or other UI) that can be used to constrain and request data in accordance with the DAP4 specification as applied to the dataset .
 
 
suffix = <font size="2"><code>'''.html'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .html'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/data-request-form#</nowiki>'''</code></font><br/><br/>
 
==== [[DAP4: RDF Service  | DAP4: RDF Service]] ====
The RDF service provides an RDF representation of the Dataset document (DDX). The RDF response is an XML document containing an RDF version of the [[DAP4: Responses#Dataset_Response | DAP4: Dataset Response.]]
 
 
suffix = <font size="2"><code>'''.rdf'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .rdf'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/rdf#</nowiki>'''</code></font><br/><br/>
 
==== [[DAP4: ISO 19115 Service ]] ====
This service provides ISO 19115 metadata for the Dataset, if any can be found. When invoked  it returns an XML document containing ISO 19115 metadata located in the  [[DAP4: Responses#Dataset_Response | DAP4: Dataset Response.]]
 
 
suffix = <font size="2"><code>'''.iso'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .iso'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/iso-19115-metadata#</nowiki>'''</code></font><br/><br/>
 
==== [[DAP4: ISO Conformance Score Service ]] ====
This service provides a browser renderable document that describes how well the metadata held in the Dataset conforms to ISO 19115. When invoked this service returns a browser renderable document that scores how well the metadata held in the [[DAP4: Responses#Dataset_Response | Dataset Response]] conforms to ISO 19115.
 
 
suffix = <font size="2"><code>'''.rubric'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .rubric'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/iso-19115-score#</nowiki>'''</code></font><br/><br/>
 
==== [[DAP4: NetCDF File-out Service ]] ====
This service provides data responses in NetCDF-3 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 3 file.


suffix = <font size="2"><code>'''.nc'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + '.nc' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/netcdf-3#</nowiki>'''</code></font><br/><br/>


== Suggested Responses ==
==== [[DAP4: ASCII Data Service]] ====
This service provides data responses in ASCII format. When invoked the regular DAP data response will be repackaged as an ASCII representation of the data values.


=== ISO 19115 ===


ISO 19115 Metadata Representation of the Dataset document
suffix = <font size="2"><code>'''.ascii'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + '.ascii' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/ascii#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:  
==== [[DAP4: XML Data Service]] ====
:<font size="2"><code>baseURL + '.iso'</code></font>
This service provides data responses in XML format. When invoked the constrained Dataset response document (DDX) will be marked up with the data values of the request and returned. Large requests may be denied.


=== ISO 19115 Conformance ===


ISO 19115 Metadata Representation conformance score for this dataset.
suffix = <font size="2"><code>'''.xdap'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + '.xdap' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/xml-data#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:  
==== [[DAP4: Native File Access Service ]] ====
:<font size="2"><code>baseURL + '.rubric'</code></font>
This service provides direct access to the data source file (or whatever else) that is creating the DAP dataset resource. When invoked it returns the "native" data from whatever store (filesystem, etc.) it may be in.


=== HTML ===


An HTML based Data Request Form for data constraints and access
suffix = <font size="2"><code>'''.file'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .file'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/file#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:  
==== [[DAP4: Server Version Service  ]]====
:<font size="2"><code>baseURL + '.html'</code></font>
This service provides software versioning information. When invoked the services returns an XML file containing a description of the version of the server and it's components.


=== RDF ===


An RDF representation of the Dataset document.
suffix = <font size="2"><code>'''.ver'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .ver'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap4/version#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:
=== DAP2 Services ===
:<font size="2"><code>baseURL + '.rdf'</code></font>


=== NC ===
==== [[DAP2: Data Service]]====
The DAP2 data service provides DAP2 data access to the data resource.


NetCDF file-out response.


The standard URL for this response is:
suffix = <font size="2"><code>'''.dods'''</code></font><br/>
:<font size="2"><code>baseURL + '.nc' + [?dap_constraint]</code></font>
service url =  <font size="2"><code>'''dataset_url + '.dods' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap2/dods#</nowiki>'''</code></font><br/><br/>


=== FILE ===
==== [[DAP2: DDX Service ]]====
The DAP2 DDX service provides DAP2 access to the data resource metadata. When invoked the service returns an XML document containing both syntactic and semantic  dataset metadata in DAP2 XML format.


Access to dataset file in it's native format. Typically this would be implemented as a runtime configurable option. No constraint expressions allowed.
suffix = <font size="2"><code>'''.ddx'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .ddx'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap2/ddx#</nowiki>'''</code></font><br/><br/>


The standard URL for this response is:
==== [[DAP2: DDS Service ]]====
:<font size="2"><code>baseURL + '.file'</code></font>
The DAP2 DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource. When invoked returns a DAP2 DDS response document conforming to the DDS part of the DAP2 specification.


== DAP2 Responses ==


=== DDX ===
suffix = <font size="2"><code>'''.dds'''</code></font><br/>
service url = <font size="2"><code>'''dataset_url + '.dds' + [?dap_constraint]'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap2/dds#</nowiki>'''</code></font><br/><br/>


==== [[DAP2: DAS Service ]] ====
The DAP2 DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource. When invoked returns a DAP2 DAS response document conforming to the DAS part of the DAP2 specification.


=== DDS ===


suffix = <font size="2"><code>'''.das'''</code></font><br/>
service url =  <font size="2"><code>'''dataset_url + .das'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap2/das#</nowiki>'''</code></font><br/><br/>


=== DAS ===
==== [[DAP2: Info Service ]] ====
The DAP2 INFO service provides a browser renderable page that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way. When invoked this service returns a web browser renderable document that combines both the DAP2 'syntactic' and  'semantic' metadata for the data resource in a human readable way.




=== INFO ===
suffix = <font size="2"><code>'''.info'''</code></font><br/>
service url = <font size="2"><code>'''dataset_url + .info'''</code></font><br/>
role id = <font size="2"><code>'''<nowiki>http://services.opendap.org/dap2/Info#</nowiki>'''</code></font><br/><br/>




=== HTML ===
[[Template: ServiceTemplate]]

Latest revision as of 21:12, 6 November 2013

This is an old document that captures the starting point of the OPULS design work. It's out of date and should be referenced only as a baseline for the work.

<-- back to OPULS Development

Author: Jimg, NDP, ?

Protocol Independence

Proposal: The DAP4 data model and it's persistent (over-the-wire) representations are transport protocol agnostic. All of the information required by the DAP must be present in the content of the DAP requests and responses. The DAP won't uniquely embed information required by the DAP into HTTP Headers or AMQP thingys, or other whatnot.

Discussion:

  1. Does this matter?
  2. Would we need to add new syntax server-side functions etc. to the protocol to enable this?

Overview

This page describes the various web services that a DAP4 server must provide. These are for the most part REST services and as such are stateless unless noted otherwise.

The services are all defined as a modification of the service (aka resource or base) URL. This base URL essentially becomes a prefix (and could even be seen as a namespace) for all of the services available for that data resource.

In practice these have traditionally been implemented over HTTP. However they could just as easily be pushed over a different protocol, as long as the usage of the URL components remain consistent.

Services

A Service is represented by:

  • A unique "namespace" like identifier that is used to define an xlink:role attribute.
  • A simple human readable name called 'title'.
  • An access URL.
  • An optional description.

A service is accessed by dereferencing its access URL, which is typically constructed by adding some type of suffix to the dataset's referent (aka base) URL. The way in which the query string (aka constraint expression) is used is defined by each service, and there is no requirement for inter-service query string API conformity.

Primary DAP4 Services

DAP4: Dataset Services Description Service

The Dataset Services Service provides a listing of the various services available for a dataset. Dereferencing the base URL for a data set accessed through a DAP4 web server will return a service document that describes the various responses possible for that data set, including URLs that can be used to access those responses.

More information on the Service Description response can be found on the DAP Service Terminus page.


service url = dataset_url
role id = http://services.opendap.org/dap4/dataset-services#

DAP4: Dataset Service - The metadata

The Dataset Service provides a metadata description of the dataset. The Dataset response is an XML document that contains both the 'syntactic' (structural) and 'semantic' metadata for the dataset, persisted as a DAP4: Data Model representation of the dataset held at the server. The Dataset service accepts a query string (constraint expression) that allows you to inspect the effects on the data structures when sub-setting and/or server side functions are applied. If a constraint expression has been successfully applied, the service will returned the constrained view of the dap:Dataset object. The constrained view may contain different data structures than the unconstrained view as the constraint may alter the reasonable representation of the data set. Note that All dap:Attribute objects have been removed from constrained dap:Dataset objects.

  • More information on the Dataset response can be found on the DAP4: Responses page.
  • More information on the syntax of DAP4 constraint expressions can be found on DAP4: Data Model page.


suffix = .xml
service url = dataset_url + '.xml' + [?dap_constraint]
role id = http://services.opendap.org/dap4/dataset#

DAP4: Data Service

The Data Service provides DAP4 data access to a dataset, and is the way that DAP4 returns data to a client. The Data service accepts a query string (constraint expression) which allows you to subset the data and invoke server side functions. When the service is invoked it returns over the wire as a multipart MIME document where the first MIME part contains the constrained Dataset response describing the data requested and the following MIME parts contain the data values, encoded using XDR, followed by a checksum, for each variable in the dataset.

More information on the Data response can be found on the DAP4: Responses page.


suffix = .dap
service url = dataset_url + '.dap' + [?dap_constraint]
role id = http://services.opendap.org/dap4/data#

Other DAP4 Services

DAP4: HTML DATA Request Form Service

The HTML DATA Request Form Service provides browser based access to the Dataset. When invoked it returns a web-browser renderable document (in html) that provides a form (or other UI) that can be used to constrain and request data in accordance with the DAP4 specification as applied to the dataset .


suffix = .html
service url = dataset_url + .html
role id = http://services.opendap.org/dap4/data-request-form#

DAP4: RDF Service

The RDF service provides an RDF representation of the Dataset document (DDX). The RDF response is an XML document containing an RDF version of the DAP4: Dataset Response.


suffix = .rdf
service url = dataset_url + .rdf
role id = http://services.opendap.org/dap4/rdf#

DAP4: ISO 19115 Service

This service provides ISO 19115 metadata for the Dataset, if any can be found. When invoked it returns an XML document containing ISO 19115 metadata located in the DAP4: Dataset Response.


suffix = .iso
service url = dataset_url + .iso
role id = http://services.opendap.org/dap4/iso-19115-metadata#

DAP4: ISO Conformance Score Service

This service provides a browser renderable document that describes how well the metadata held in the Dataset conforms to ISO 19115. When invoked this service returns a browser renderable document that scores how well the metadata held in the Dataset Response conforms to ISO 19115.


suffix = .rubric
service url = dataset_url + .rubric
role id = http://services.opendap.org/dap4/iso-19115-score#

DAP4: NetCDF File-out Service

This service provides data responses in NetCDF-3 file format. When invoked the regular DAP data response will be repackaged as a NetCDF 3 file.

suffix = .nc
service url = dataset_url + '.nc' + [?dap_constraint]
role id = http://services.opendap.org/dap4/netcdf-3#

DAP4: ASCII Data Service

This service provides data responses in ASCII format. When invoked the regular DAP data response will be repackaged as an ASCII representation of the data values.


suffix = .ascii
service url = dataset_url + '.ascii' + [?dap_constraint]
role id = http://services.opendap.org/dap4/ascii#

DAP4: XML Data Service

This service provides data responses in XML format. When invoked the constrained Dataset response document (DDX) will be marked up with the data values of the request and returned. Large requests may be denied.


suffix = .xdap
service url = dataset_url + '.xdap' + [?dap_constraint]
role id = http://services.opendap.org/dap4/xml-data#

DAP4: Native File Access Service

This service provides direct access to the data source file (or whatever else) that is creating the DAP dataset resource. When invoked it returns the "native" data from whatever store (filesystem, etc.) it may be in.


suffix = .file
service url = dataset_url + .file
role id = http://services.opendap.org/dap4/file#

DAP4: Server Version Service

This service provides software versioning information. When invoked the services returns an XML file containing a description of the version of the server and it's components.


suffix = .ver
service url = dataset_url + .ver
role id = http://services.opendap.org/dap4/version#

DAP2 Services

DAP2: Data Service

The DAP2 data service provides DAP2 data access to the data resource.


suffix = .dods
service url = dataset_url + '.dods' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dods#

DAP2: DDX Service

The DAP2 DDX service provides DAP2 access to the data resource metadata. When invoked the service returns an XML document containing both syntactic and semantic dataset metadata in DAP2 XML format.

suffix = .ddx
service url = dataset_url + .ddx
role id = http://services.opendap.org/dap2/ddx#

DAP2: DDS Service

The DAP2 DDS service provides access to the 'syntactic' metadata (aka use or structural metadata) for the data resource. When invoked returns a DAP2 DDS response document conforming to the DDS part of the DAP2 specification.


suffix = .dds
service url = dataset_url + '.dds' + [?dap_constraint]
role id = http://services.opendap.org/dap2/dds#

DAP2: DAS Service

The DAP2 DAS service provides access to the 'semantic' metadata (aka domain metadata) for the data resource. When invoked returns a DAP2 DAS response document conforming to the DAS part of the DAP2 specification.


suffix = .das
service url = dataset_url + .das
role id = http://services.opendap.org/dap2/das#

DAP2: Info Service

The DAP2 INFO service provides a browser renderable page that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way. When invoked this service returns a web browser renderable document that combines both the DAP2 'syntactic' and 'semantic' metadata for the data resource in a human readable way.


suffix = .info
service url = dataset_url + .info
role id = http://services.opendap.org/dap2/Info#


Template: ServiceTemplate