Difference between revisions of "Wiki Testing/dintro"

From OPeNDAP Documentation
(Installing the OPeNDAP FreeForm ND Data Handler)
Line 2: Line 2:
  
  
The OPeNDAP FreeForm ND Data Handler is a DAP2[[Wiki_Testing/footnotes|(1)]] -compliant server that uses
+
The OPeNDAP FreeForm ND Data Handler is an OPeNDAP data handler
 
OPeNDAP FreeForm ND software to serve data from files in almost any format. The
 
OPeNDAP FreeForm ND software to serve data from files in almost any format. The
 
FreeForm ND Data Access System is a flexible system for specifying
 
FreeForm ND Data Access System is a flexible system for specifying
Line 28: Line 28:
 
==The FreeForm ND Solution==
 
==The FreeForm ND Solution==
  
OPeNDAP FreeForm ND uses a \new{format descriptor} file to describe the format
+
OPeNDAP FreeForm ND uses a ''format descriptor'' file to describe the format
 
of one or more data files.  This descriptor file is a simple text file
 
of one or more data files.  This descriptor file is a simple text file
 
that can be created with a text editor, describing the structure of
 
that can be created with a text editor, describing the structure of
 
your data files.
 
your data files.
  
A traditional DAP2 server, illustrated in [[:Image:regular.jpg|figure 1.1]]  
+
A traditional DAP2 server, illustrated [[:Image:regular.jpg|below]]  
 
receives a request for data from a DAP2 client who may be at some
 
receives a request for data from a DAP2 client who may be at some
 
remote computer[[Wiki_Testing/footnotes|(2)]]. The data served by this server must be stored in one of the data formats supported by the OPeNDAP server (such as netCDF, HDF, or JGOFS), and the server uses specialized software to read this data
 
remote computer[[Wiki_Testing/footnotes|(2)]]. The data served by this server must be stored in one of the data formats supported by the OPeNDAP server (such as netCDF, HDF, or JGOFS), and the server uses specialized software to read this data
Line 99: Line 99:
 
download.
 
download.
  
To install the OPeNDAP FreeForm ND Data Handler , first make sure you have an http server running on the machine where you plan to serve the data.  If you are unsure
+
Install the resulting shared library in a place where [[Hyrax]] can
whether one is running, you can use a web client, like Netscape, to
+
find it, and then consult the Hyrax configuration instructions for the
send an http request to your own machine.  [http://docs.opendap.org/index.php/Wiki_Testing/OpeNDAP_User%27s_Guide<cite>The OPeNDAP User Guide</cite>] contains some
+
remaining configuration steps.
hints about installing a web server.
 
 
 
When you are sure a web server is properly installed and running,
 
unpack the archive file you downloaded from the [http://www.opendap.org/<cite>OPeNDAP</cite>] . Binaries
 
are available for Solaris, IRIX, Linux, et cetera. Follow the
 
instructions that come with each, as they all have their own
 
idiosyncrasies. After unpacking the server will, by default install
 
some executable programs in /usr/local/bin and some other files in
 
both /usr/local/share/dap-server and /usr/local/share/dap-server-cgi.
 
 
 
Follow the instructions on configuring the OPeNDAP server. This
 
generally involves configuring a web server (e.g., Apache) to run the
 
<font color='green'>nh-dods</font> CGI script and editing the dap-server.rc configuration
 
file. Both reside in /usr/local/share/dap-server-cgi (assuming that
 
you have chosen the default location for installation).
 
 
 
Now that the OPeNDAP FreeForm ND Data Handler is installed, [[Wiki_Testing/ff-server | Chapter 6]] will explain how to use it to serve your data.  It may be a good idea to familiarize yourself with the information in the intervening chapters, which will explain how to specify your data's format.
 
  
 
====Compiling the OPeNDAP FreeForm ND Data Handler====
 
====Compiling the OPeNDAP FreeForm ND Data Handler====
Line 128: Line 111:
 
and perform the following steps:
 
and perform the following steps:
  
#Expand the distribution (e.g., \lit{tar -xzf libdap-3.5.3.tar.gz})
+
#Expand the distribution (e.g., tar -xzf libdap-3.5.3.tar.gz)
 
#Change to the newly created directory (<font color='green'>cd libdap-3.5.3</font>)
 
#Change to the newly created directory (<font color='green'>cd libdap-3.5.3</font>)
 
#Run the configure script (<font color='green'>./configure</font>)
 
#Run the configure script (<font color='green'>./configure</font>)
 
#Run make (<font color='green'>make</font>)
 
#Run make (<font color='green'>make</font>)
#Install the software (<font color='green'>make install</font> or \lit{sudo make install})
+
#Install the software (<font color='green'>make install</font> or sudo make install)
  
 
Each source distribution contains more detailed build instructions;
 
Each source distribution contains more detailed build instructions;
 
see the <font color='green'>README</font>, <font color='green'>INSTALL</font> and <font color='green'>NEWS</font> files for the most
 
see the <font color='green'>README</font>, <font color='green'>INSTALL</font> and <font color='green'>NEWS</font> files for the most
up-to-date information. See [[Wiki_Testing/dintro | Section 1.3]] for
+
up-to-date information.
instructions on the final installation.
 

Revision as of 23:36, 6 October 2009

1 Introduction

The OPeNDAP FreeForm ND Data Handler is an OPeNDAP data handler OPeNDAP FreeForm ND software to serve data from files in almost any format. The FreeForm ND Data Access System is a flexible system for specifying data formats to facilitate data access, management, and use. Since DAP2 allows data to be translated over the internet and read by a client regardless of the storage format of the data, the combination allows several format restrictions to be overcome.

The large variety of data formats is a primary obstacle in the way of creating flexible data management and analysis software. FreeForm ND was conceived, developed, and implemented at the National Geophysical Data Center (NGDC) to alleviate the problems that occur when you need to use data sets with varying native formats or to write format-independent applications.

DAP2 was originally conceived as a way to move large amounts of scientific data over the internet. As a consequence of establishing a flexible data transmission format, DAP2 also allows substantial independence from the storage format of the original data. Up to now, however, DAP2 servers have been limited to data in a few widely used formats. Using the OPeNDAP FreeForm ND Data Handler , many more datasets can be made available through DAP2.


1.1 The FreeForm ND Solution

OPeNDAP FreeForm ND uses a format descriptor file to describe the format of one or more data files. This descriptor file is a simple text file that can be created with a text editor, describing the structure of your data files.

A traditional DAP2 server, illustrated below receives a request for data from a DAP2 client who may be at some remote computer(2). The data served by this server must be stored in one of the data formats supported by the OPeNDAP server (such as netCDF, HDF, or JGOFS), and the server uses specialized software to read this data from disk.

When it receives a request, the server reads the requested data from its archive, reformats the data into the DAP2 transmission format and sends the data back to the client.

actual size

A Traditional DAP2 Server

The OPeNDAP FreeForm ND Data Handler works in a similar fashion to a traditional DAP2 server, but before the server reads the data from the archive, it first reads the data format descriptor to determine how it should read the data. Only after it has absorbed the details of the data storage format does it attempt to read the data, pack it into the transmission format and send it on its way back to the client.

actual size

The OPeNDAP FreeForm ND Data Handler

1.2 The FreeForm ND System

The OPeNDAP FreeForm ND Data Handler comprises a format description mechanism, a set of programs for manipulating data, and the server itself. The software was built using the FreeForm ND library and data objects. These are documented in The FreeForm ND User's Guide.

The OPeNDAP FreeForm ND Data Handler includes the following programs:


dap_ff_handler 
The OPeNDAP FreeForm ND Data Handler data handler. This program is run by the OPeNDAP server to handle the parts of any requests that require knowledge specifically about FreeForm. This program is run by the main server 'dispatch' software. That software is described in the Server Installation Guide, available on the OPeNDAP Home Page .


The OPeNDAP FreeForm ND Data Handler distribution also includes the following OPeNDAP FreeForm ND utilities. These are quite useful to write and debug format description files.


newform 
This program reformats data according to the input and output specifications in a format description file.
chkform 
After writing a format description file, you can use this program to cross-check the description against a data file.
readfile 
This program is useful to decode the format used by a binary file. It allows you to try different formats on pieces of a binary file, and see what works.

1.3 Installing the OPeNDAP FreeForm ND Data Handler

If you don't have the OPeNDAP FreeForm ND Data Handler , and want it, follow these directions. If you have a copy of the OPeNDAP FreeForm ND Data Handler , and want to know how to use it, see ( ff,dquick) for quick instructions and examples of its use, and Chapter 6 for further information.

You can get the OPeNDAP FreeForm ND Data Handler from the OPeNDAP . Follow the links to "Download Software" and then to "Current Release." If your computer is one of the platforms for which we provide a binary release, get that, otherwise get the source code.

To get a binary release, go to that page, click on the computer you use, and click on the "FreeForm" button in the "Servers" box. Click the Download button, and follow the directions. The server will make a custom binary file for you, which you then download.

Install the resulting shared library in a place where Hyrax can find it, and then consult the Hyrax configuration instructions for the remaining configuration steps.

1.3.1 Compiling the OPeNDAP FreeForm ND Data Handler

If the computer and operating system combination you use is not one of the ones we own, you will have to compile the OPeNDAP FreeForm ND Data Handler from its source. Go to the OPeNDAP home page (www.opendap.org) and follow the menu item to the downloads page. From there you will need the libdap, dap-server and FreeForm handler software source distributions. Get each of these and perform the following steps:

  1. Expand the distribution (e.g., tar -xzf libdap-3.5.3.tar.gz)
  2. Change to the newly created directory (cd libdap-3.5.3)
  3. Run the configure script (./configure)
  4. Run make (make)
  5. Install the software (make install or sudo make install)

Each source distribution contains more detailed build instructions; see the README, INSTALL and NEWS files for the most up-to-date information.