Wiki Testing/ff-server: Difference between revisions
TomSgouros (talk | contribs) No edit summary |
|||
(5 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
[[FreeForm|return to FreeForm]] | |||
=The OPeNDAP FreeForm ND Data Handler= | =The OPeNDAP FreeForm ND Data Handler= | ||
The OPeNDAP FreeForm ND Data Handler is a OPeNDAP server add-on that uses OPeNDAP FreeForm ND to convert and serve data | The OPeNDAP FreeForm ND Data Handler is a OPeNDAP server add-on that uses OPeNDAP FreeForm ND to convert and serve data | ||
in formats that are not directly supported by | in formats that are not directly supported by existing [[Hyrax]] data handlers. Bringing | ||
OPeNDAP FreeForm ND 's data conversion capacity into the OPeNDAP world widens data | OPeNDAP FreeForm ND 's data conversion capacity into the OPeNDAP world widens data | ||
access for DAP2 clients, since any format that can be described in | access for DAP2 clients, since any format that can be described in | ||
Line 19: | Line 21: | ||
#The OPeNDAP FreeForm ND Data Handler looks in its path for two files: a data file with the name sent by the client, and a format definition file to use with the data file. The format definition file contains a description of the data format, constructed according to the OPeNDAP FreeForm ND syntax. | #The OPeNDAP FreeForm ND Data Handler looks in its path for two files: a data file with the name sent by the client, and a format definition file to use with the data file. The format definition file contains a description of the data format, constructed according to the OPeNDAP FreeForm ND syntax. | ||
#The server uses both files in invoking the OPeNDAP FreeForm ND engine. The OPeNDAP FreeForm ND engine reads the data file and the format file, using the instructions in the latter to convert the former into data which is then passed back to the OPeNDAP FreeForm ND Data Handler . | #The server uses both files in invoking the OPeNDAP FreeForm ND engine. The OPeNDAP FreeForm ND engine reads the data file and the format file, using the instructions in the latter to convert the former into data which is then passed back to the OPeNDAP FreeForm ND Data Handler . | ||
#On receiving the converted data, the OPeNDAP FreeForm ND Data Handler converts the data into the DAP2 transmission format. The conversion may involve some adjustment of data types; these are listed | #On receiving the converted data, the OPeNDAP FreeForm ND Data Handler converts the data into the DAP2 transmission format. The conversion may involve some adjustment of data types; these are listed below. The server also applies any constraint expressions the client sent along with the URL. | ||
#The server then constructs DDS and DAS files based on the format of the converted data. If the server has access to DDS and DAS files that describe the data, it applies those definition before sending them back to the client. | #The server then constructs DDS and DAS files based on the format of the converted data. If the server has access to DDS and DAS files that describe the data, it applies those definition before sending them back to the client. | ||
#Finally, the server sends the DDS, DAS, and converted data back to the client. | #Finally, the server sends the DDS, DAS, and converted data back to the client. | ||
For information about how to write a OPeNDAP FreeForm ND data description, refer to | For information about how to write a OPeNDAP FreeForm ND data description, refer to | ||
the [[Wiki_Testing/tblfmt| | the [[Wiki_Testing/tblfmt|Table Format]] for sequence data and | ||
[[Wiki_Testing/arrayfmt| | [[Wiki_Testing/arrayfmt|Array Format]] for array data. | ||
For an introduction to DAP2 and to the OPeNDAP project, please refer | For an introduction to DAP2 and to the OPeNDAP project, please refer | ||
Line 36: | Line 38: | ||
resulting software: | resulting software: | ||
*The OPeNDAP FreeForm ND Data Handler is a OPeNDAP FreeForm ND application that converts data ''on receiving a client request for that data'', and not before. Data | *The OPeNDAP FreeForm ND Data Handler is a OPeNDAP FreeForm ND application that converts data ''on receiving a client request for that data'', and not before. Data served by the OPeNDAP FreeForm ND Data Handler remains in its original format. | ||
served by the OPeNDAP FreeForm ND Data Handler remains in its original format. | *The OPeNDAP FreeForm ND Data Handler does not produce an output file containing the converted data, but serves it directly over the network to the DAP2 client. Therefore, the OPeNDAP FreeForm ND Data Handler ignores the output section of the format definition file. | ||
*The OPeNDAP FreeForm ND Data Handler does not produce an output file containing the | |||
converted data, but serves it directly over the network to the DAP2 client. Therefore, the OPeNDAP FreeForm ND Data Handler ignores the output section of the | |||
format definition file. | |||
*To sample a data file, you do not write format definitions that cause the OPeNDAP FreeForm ND engine to sample the data file. Instead, you add a DAP2 "constraint expression" to the URL that the client sends to the OPeNDAP FreeForm ND Data Handler . | *To sample a data file, you do not write format definitions that cause the OPeNDAP FreeForm ND engine to sample the data file. Instead, you add a DAP2 "constraint expression" to the URL that the client sends to the OPeNDAP FreeForm ND Data Handler . | ||
*The OPeNDAP FreeForm ND Data Handler performs data conversion on the fly. Conversion only takes place when the client sends a URL requesting data from the | *The OPeNDAP FreeForm ND Data Handler performs data conversion on the fly. Conversion only takes place when the client sends a URL requesting data from the OPeNDAP FreeForm ND Data Handler . | ||
OPeNDAP FreeForm ND Data Handler . | |||
*Unlike OPeNDAP FreeForm ND , there is no static file created by the conversion. | *Unlike OPeNDAP FreeForm ND , there is no static file created by the conversion. | ||
(If you wish to create or work with such a file, use the OPeNDAP FreeForm ND | (If you wish to create or work with such a file, use the OPeNDAP FreeForm ND | ||
utilities, such as <font color='green'>newform</font>.) | utilities, such as <font color='green'>newform</font>.) | ||
==Data Type Conversions== | ==Data Type Conversions== | ||
Line 57: | Line 54: | ||
<center> | |||
DAP2 Data Type Conversions | |||
{| border="1" | {| border="1" | ||
Line 81: | Line 78: | ||
|} | |} | ||
</center> | |||
===Conversion Examples=== | ===Conversion Examples=== | ||
The examples show how the OPeNDAP FreeForm ND Data Handler treats data received from the OPeNDAP FreeForm ND | The examples show how the OPeNDAP FreeForm ND Data Handler treats data | ||
engine. Please see the OPeNDAP FreeForm ND Data Handler distribution for more test data and | received from the OPeNDAP FreeForm ND engine. Please see the OPeNDAP | ||
FreeForm ND Data Handler distribution for more test data and format | |||
writing format definitions. | definition files, and the ([[Wiki_Testing/tblfmt|Table Format]]) for | ||
more information on writing format definitions. | |||
Line 131: | Line 129: | ||
<pre> | <pre> | ||
Sequence { | Sequence { | ||
Int32 fvar1; | Int32 fvar1; | ||
Int32 svar1; | Int32 svar1; | ||
Line 159: | Line 155: | ||
<pre> | <pre> | ||
Dataset { | Dataset { | ||
Int32 fvar1[records=101] | Int32 fvar1[records=101] | ||
Int32 fvar2[records=101] | Int32 fvar2[records=101] | ||
}; | }; | ||
</pre> | </pre> | ||
Latest revision as of 23:53, 6 October 2009
The OPeNDAP FreeForm ND Data Handler
The OPeNDAP FreeForm ND Data Handler is a OPeNDAP server add-on that uses OPeNDAP FreeForm ND to convert and serve data in formats that are not directly supported by existing Hyrax data handlers. Bringing OPeNDAP FreeForm ND 's data conversion capacity into the OPeNDAP world widens data access for DAP2 clients, since any format that can be described in OPeNDAP FreeForm ND can now be served by the OPeNDAP data server.
Like all DAP2 servers, the OPeNDAP FreeForm ND Data Handler responds to client requests for data by returning either data values or information about the data. It differs from other DAP2 servers because it invokes OPeNDAP FreeForm ND to read the data from disk before serving it to the client.
The following sequence of steps illustrates how the OPeNDAP FreeForm ND Data Handler works:
- A DAP2 client sends a request for data to a OPeNDAP FreeForm ND Data Handler . The request must include the name of the file that contains the data, and may include a constraint expression to sample the data.
- The OPeNDAP FreeForm ND Data Handler looks in its path for two files: a data file with the name sent by the client, and a format definition file to use with the data file. The format definition file contains a description of the data format, constructed according to the OPeNDAP FreeForm ND syntax.
- The server uses both files in invoking the OPeNDAP FreeForm ND engine. The OPeNDAP FreeForm ND engine reads the data file and the format file, using the instructions in the latter to convert the former into data which is then passed back to the OPeNDAP FreeForm ND Data Handler .
- On receiving the converted data, the OPeNDAP FreeForm ND Data Handler converts the data into the DAP2 transmission format. The conversion may involve some adjustment of data types; these are listed below. The server also applies any constraint expressions the client sent along with the URL.
- The server then constructs DDS and DAS files based on the format of the converted data. If the server has access to DDS and DAS files that describe the data, it applies those definition before sending them back to the client.
- Finally, the server sends the DDS, DAS, and converted data back to the client.
For information about how to write a OPeNDAP FreeForm ND data description, refer to the Table Format for sequence data and Array Format for array data.
For an introduction to DAP2 and to the OPeNDAP project, please refer to The Opendap User Guide.
Differences between OPeNDAP FreeForm ND and the OPeNDAP FreeForm ND Data Handler
The OPeNDAP FreeForm ND Data Handler is based on the same libraries used to make the OPeNDAP FreeForm ND utilities. However, there are some important differences in the resulting software:
- The OPeNDAP FreeForm ND Data Handler is a OPeNDAP FreeForm ND application that converts data on receiving a client request for that data, and not before. Data served by the OPeNDAP FreeForm ND Data Handler remains in its original format.
- The OPeNDAP FreeForm ND Data Handler does not produce an output file containing the converted data, but serves it directly over the network to the DAP2 client. Therefore, the OPeNDAP FreeForm ND Data Handler ignores the output section of the format definition file.
- To sample a data file, you do not write format definitions that cause the OPeNDAP FreeForm ND engine to sample the data file. Instead, you add a DAP2 "constraint expression" to the URL that the client sends to the OPeNDAP FreeForm ND Data Handler .
- The OPeNDAP FreeForm ND Data Handler performs data conversion on the fly. Conversion only takes place when the client sends a URL requesting data from the OPeNDAP FreeForm ND Data Handler .
- Unlike OPeNDAP FreeForm ND , there is no static file created by the conversion.
(If you wish to create or work with such a file, use the OPeNDAP FreeForm ND utilities, such as newform.)
Data Type Conversions
The OPeNDAP FreeForm ND Data Handler performs data conversions, based on the data it receives from the OPeNDAP FreeForm ND engine. Note that OPeNDAP does not recommend the use of int64 and uint64 in the format definition file.
DAP2 Data Type Conversions
OPeNDAP FreeForm ND | DAP2 |
---|---|
text | String |
int8, uint8 | Byte |
int16 | Int16 |
int32, int64 | Int32 |
uint16 | UInt16 |
uint32, uint64 | UInt32 |
float32 | Float32 |
float64, enote | Float64 |
Conversion Examples
The examples show how the OPeNDAP FreeForm ND Data Handler treats data received from the OPeNDAP FreeForm ND engine. Please see the OPeNDAP FreeForm ND Data Handler distribution for more test data and format definition files, and the (Table Format) for more information on writing format definitions.
Arrays
If you define a variable as an array in the OPeNDAP FreeForm ND format definition file, the OPeNDAP FreeForm ND Data Handler produces an array of variables with matching types.
For exmple, this entry in the format definition file:
binary_input_data "array" fvar1 1 4 ARRAY["records" 1 to 101] of int32 0
in converted by the OPeNDAP FreeForm ND Data Handler to:
Int32 fvar1[records = 101]
Collections of Variables
If you define several variables in the format definition file, the OPeNDAP FreeForm ND Data Handler produces a Sequence of variables with matching types.
For example, this entry in the format definition file:
ASCII_input_data "ASCII_data" fvar1 1 10 int32 2 svar1 13 18 int16 0 usvar1 21 26 uint16 1 lvar1 29 39 int32 0 ulvar1 42 52 uint32 4
is converted by the OPeNDAP FreeForm ND Data Handler to:
Sequence { Int32 fvar1; Int32 svar1; ... } ASCII_data;
Multiple Arrays
If you define a collection of arrays in the format definition file, as you would expect, the OPeNDAP FreeForm ND Data Handler produces a dataset containing multiple arrays.
For example, this entry in the format definition file:
binary_input_data "arrays" fvar1 1 4 ARRAY["records" 1 to 101] of int32 0 fvar2 1 4 ARRAY["records" 1 to 101] of int32 0
is converted by the OPeNDAP FreeForm ND Data Handler to:
Dataset { Int32 fvar1[records=101] Int32 fvar2[records=101] };