Wiki Testing/dintro: Difference between revisions
(New page: =Introduction= The OPeNDAP FreeForm ND Data Handler is a DAP2 -compliant server that uses OPeNDAP FreeForm ND software to serve data from files in almost any format. The FreeForm ND Data...) |
TomSgouros (talk | contribs) No edit summary |
||
(9 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
[[FreeForm|return to FreeForm]] | |||
=Introduction= | =Introduction= | ||
The OPeNDAP FreeForm ND Data Handler is | The OPeNDAP FreeForm ND Data Handler is an OPeNDAP data handler | ||
OPeNDAP FreeForm ND software to serve data from files in almost any format. The | OPeNDAP FreeForm ND software to serve data from files in almost any format. The | ||
FreeForm ND Data Access System is a flexible system for specifying | FreeForm ND Data Access System is a flexible system for specifying | ||
Line 28: | Line 30: | ||
==The FreeForm ND Solution== | ==The FreeForm ND Solution== | ||
OPeNDAP FreeForm ND uses a | OPeNDAP FreeForm ND uses a ''format descriptor'' file to describe the format | ||
of one or more data files. This descriptor file is a simple text file | of one or more data files. This descriptor file is a simple text file | ||
that can be created with a text editor, describing the structure of | that can be created with a text editor, describing the structure of | ||
your data files. | your data files. | ||
A traditional DAP2 server, illustrated | A traditional DAP2 server, illustrated [[:Image:regular.jpg|below]] | ||
receives a request for data from a DAP2 client who may be at some | receives a request for data from a DAP2 client who may be at some | ||
remote computer. The data served by this server must be stored in one of the data | remote computer[[Wiki_Testing/footnotes|(2)]]. The data served by this server must be stored in one of the data formats supported by the OPeNDAP server (such as netCDF, HDF, or JGOFS), and the server uses specialized software to read this data | ||
formats supported by the OPeNDAP server (such as netCDF, HDF, or | |||
JGOFS), and the server uses specialized software to read this data | |||
from disk. | from disk. | ||
Line 44: | Line 44: | ||
sends the data back to the client. | sends the data back to the client. | ||
<center> | |||
[[Image:regular.jpg|actual size]] | |||
A Traditional DAP2 Server | |||
</center> | |||
The OPeNDAP FreeForm ND Data Handler works in a similar fashion to a traditional DAP2 server, but | The OPeNDAP FreeForm ND Data Handler works in a similar fashion to a traditional DAP2 server, but | ||
Line 55: | Line 57: | ||
send it on its way back to the client. | send it on its way back to the client. | ||
<center> | |||
[[Image:ff1.jpg|actual size]] | |||
The OPeNDAP FreeForm ND Data Handler | |||
</center> | |||
==The FreeForm ND System== | ==The FreeForm ND System== | ||
Line 64: | Line 68: | ||
for manipulating data, and the server itself. The software was built | for manipulating data, and the server itself. The software was built | ||
using the FreeForm ND library and data objects. These are documented | using the FreeForm ND library and data objects. These are documented | ||
in | in The FreeForm ND User's Guide. | ||
The OPeNDAP FreeForm ND Data Handler includes the following programs: | The OPeNDAP FreeForm ND Data Handler includes the following programs: | ||
; <font color='green'>dap_ff_handler</font> : The OPeNDAP FreeForm ND Data Handler | ; <font color='green'>dap_ff_handler</font> : The OPeNDAP FreeForm ND Data Handler ''data handler''. This program is run by the OPeNDAP server to handle the parts of any requests that require knowledge specifically about FreeForm. This program is run by the main server 'dispatch' software. That software is described in the Server Installation Guide, available on the [http://www.opendap.org/<cite>OPeNDAP Home Page</cite>] . | ||
that require knowledge specifically about FreeForm. This program is | |||
run by the main server 'dispatch' software. That software is | |||
described in the Server Installation Guide | |||
available on the [http://www.opendap.org/<cite>OPeNDAP</cite>] . | |||
The OPeNDAP FreeForm ND Data Handler distribution also includes the following OPeNDAP FreeForm ND utilities. These are quite useful to write and debug format description files. | The OPeNDAP FreeForm ND Data Handler distribution also includes the following OPeNDAP FreeForm ND utilities. These are quite useful to write and debug format description files. | ||
; <font color='green'>newform</font> : This program reformats data according to the | ; <font color='green'>newform</font> : This program reformats data according to the input ''and output'' specifications in a format description file. | ||
input ''and output'' specifications in a format description | |||
; <font color='green'> | ; <font color='green'>chkform</font> : After writing a format description file, you can use this program to cross-check the description against a data file. | ||
; <font color='green'>readfile</font> : This program is useful to decode the format used by a binary file. It allows you to try different formats on pieces of a binary file, and see what works. | |||
==Installing the OPeNDAP FreeForm ND Data Handler== | ==Installing the OPeNDAP FreeForm ND Data Handler== | ||
If you don't have the OPeNDAP FreeForm ND Data Handler , and want it, follow these directions. If | If you don't have the OPeNDAP FreeForm ND Data Handler , and want it, follow these directions. If you have a copy of the OPeNDAP FreeForm ND Data Handler , and want to know how to use it, see ([http://www <cite> ff,dquick</cite>]) for quick instructions and examples of its use, and [[Wiki_Testing/ff-server | Chapter 6]] for further information. | ||
you have a copy of the OPeNDAP FreeForm ND Data Handler , and want to know how to use it, see | |||
([http://www <cite> ff,dquick</cite>]) for quick instructions and examples of its use, and | |||
You can get the OPeNDAP FreeForm ND Data Handler from the [http://www.opendap.org/<cite>OPeNDAP</cite>] . Follow the links to | You can get the OPeNDAP FreeForm ND Data Handler from the [http://www.opendap.org/<cite>OPeNDAP</cite>] . Follow the links to | ||
Line 106: | Line 97: | ||
To get a binary release, go to that page, click on the computer you | To get a binary release, go to that page, click on the computer you | ||
use, and click on the "FreeForm" button in the "Servers" box. | use, and click on the "FreeForm" button in the "Servers" box. | ||
Click the | Click the Download button, and follow the directions. The | ||
server will make a custom binary file for you, which you then | server will make a custom binary file for you, which you then | ||
download. | download. | ||
Install the resulting shared library in a place where [[Hyrax]] can | |||
find it, and then consult the Hyrax configuration instructions for the | |||
remaining configuration steps. | |||
====Compiling the OPeNDAP FreeForm ND Data Handler==== | ====Compiling the OPeNDAP FreeForm ND Data Handler==== | ||
If the computer and operating system combination you use is not one of | If the computer and operating system combination you use is not one of | ||
the ones we own, you will have to compile the OPeNDAP FreeForm ND Data Handler from its source. | the ones we own, you will have to compile the OPeNDAP FreeForm ND Data Handler from its source. Go to the OPeNDAP home page (www.opendap.org) and follow the menu item | ||
Go to the OPeNDAP home page (www.opendap.org) and follow the menu item | |||
to the downloads page. From there you will need the libdap, dap-server | to the downloads page. From there you will need the libdap, dap-server | ||
and FreeForm handler software source distributions. Get each of these | and FreeForm handler software source distributions. Get each of these | ||
and perform the following steps: | and perform the following steps: | ||
#Expand the distribution (e.g., | #Expand the distribution (e.g., tar -xzf libdap-3.5.3.tar.gz) | ||
libdap-3.5.3.tar.gz | |||
#Change to the newly created directory (<font color='green'>cd libdap-3.5.3</font>) | #Change to the newly created directory (<font color='green'>cd libdap-3.5.3</font>) | ||
#Run the configure script (<font color='green'>./configure</font>) | #Run the configure script (<font color='green'>./configure</font>) | ||
#Run make (<font color='green'>make</font>) | #Run make (<font color='green'>make</font>) | ||
#Install the software (<font color='green'>make install</font> or | #Install the software (<font color='green'>make install</font> or sudo make install) | ||
install | |||
Each source distribution contains more detailed build instructions; | Each source distribution contains more detailed build instructions; | ||
see the <font color='green'>README</font>, <font color='green'>INSTALL</font> and <font color='green'>NEWS</font> files for the most | see the <font color='green'>README</font>, <font color='green'>INSTALL</font> and <font color='green'>NEWS</font> files for the most | ||
up-to-date information | up-to-date information. | ||
Latest revision as of 23:37, 6 October 2009
Introduction
The OPeNDAP FreeForm ND Data Handler is an OPeNDAP data handler OPeNDAP FreeForm ND software to serve data from files in almost any format. The FreeForm ND Data Access System is a flexible system for specifying data formats to facilitate data access, management, and use. Since DAP2 allows data to be translated over the internet and read by a client regardless of the storage format of the data, the combination allows several format restrictions to be overcome.
The large variety of data formats is a primary obstacle in the way of creating flexible data management and analysis software. FreeForm ND was conceived, developed, and implemented at the National Geophysical Data Center (NGDC) to alleviate the problems that occur when you need to use data sets with varying native formats or to write format-independent applications.
DAP2 was originally conceived as a way to move large amounts of scientific data over the internet. As a consequence of establishing a flexible data transmission format, DAP2 also allows substantial independence from the storage format of the original data. Up to now, however, DAP2 servers have been limited to data in a few widely used formats. Using the OPeNDAP FreeForm ND Data Handler , many more datasets can be made available through DAP2.
The FreeForm ND Solution
OPeNDAP FreeForm ND uses a format descriptor file to describe the format of one or more data files. This descriptor file is a simple text file that can be created with a text editor, describing the structure of your data files.
A traditional DAP2 server, illustrated below receives a request for data from a DAP2 client who may be at some remote computer(2). The data served by this server must be stored in one of the data formats supported by the OPeNDAP server (such as netCDF, HDF, or JGOFS), and the server uses specialized software to read this data from disk.
When it receives a request, the server reads the requested data from its archive, reformats the data into the DAP2 transmission format and sends the data back to the client.
A Traditional DAP2 Server
The OPeNDAP FreeForm ND Data Handler works in a similar fashion to a traditional DAP2 server, but before the server reads the data from the archive, it first reads the data format descriptor to determine how it should read the data. Only after it has absorbed the details of the data storage format does it attempt to read the data, pack it into the transmission format and send it on its way back to the client.
The OPeNDAP FreeForm ND Data Handler
The FreeForm ND System
The OPeNDAP FreeForm ND Data Handler comprises a format description mechanism, a set of programs for manipulating data, and the server itself. The software was built using the FreeForm ND library and data objects. These are documented in The FreeForm ND User's Guide.
The OPeNDAP FreeForm ND Data Handler includes the following programs:
- dap_ff_handler
- The OPeNDAP FreeForm ND Data Handler data handler. This program is run by the OPeNDAP server to handle the parts of any requests that require knowledge specifically about FreeForm. This program is run by the main server 'dispatch' software. That software is described in the Server Installation Guide, available on the OPeNDAP Home Page .
The OPeNDAP FreeForm ND Data Handler distribution also includes the following OPeNDAP FreeForm ND utilities. These are quite useful to write and debug format description files.
- newform
- This program reformats data according to the input and output specifications in a format description file.
- chkform
- After writing a format description file, you can use this program to cross-check the description against a data file.
- readfile
- This program is useful to decode the format used by a binary file. It allows you to try different formats on pieces of a binary file, and see what works.
Installing the OPeNDAP FreeForm ND Data Handler
If you don't have the OPeNDAP FreeForm ND Data Handler , and want it, follow these directions. If you have a copy of the OPeNDAP FreeForm ND Data Handler , and want to know how to use it, see ( ff,dquick) for quick instructions and examples of its use, and Chapter 6 for further information.
You can get the OPeNDAP FreeForm ND Data Handler from the OPeNDAP . Follow the links to "Download Software" and then to "Current Release." If your computer is one of the platforms for which we provide a binary release, get that, otherwise get the source code.
To get a binary release, go to that page, click on the computer you use, and click on the "FreeForm" button in the "Servers" box. Click the Download button, and follow the directions. The server will make a custom binary file for you, which you then download.
Install the resulting shared library in a place where Hyrax can find it, and then consult the Hyrax configuration instructions for the remaining configuration steps.
Compiling the OPeNDAP FreeForm ND Data Handler
If the computer and operating system combination you use is not one of the ones we own, you will have to compile the OPeNDAP FreeForm ND Data Handler from its source. Go to the OPeNDAP home page (www.opendap.org) and follow the menu item to the downloads page. From there you will need the libdap, dap-server and FreeForm handler software source distributions. Get each of these and perform the following steps:
- Expand the distribution (e.g., tar -xzf libdap-3.5.3.tar.gz)
- Change to the newly created directory (cd libdap-3.5.3)
- Run the configure script (./configure)
- Run make (make)
- Install the software (make install or sudo make install)
Each source distribution contains more detailed build instructions; see the README, INSTALL and NEWS files for the most up-to-date information.