How to build the DataDDX response in/with Hyrax: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
Line 36: Line 36:


The OLFS will need to be modified to recognize the new request, generate the correct boundary and start values, and complete the response by building the correct response headers and closing 'separator' headers.
The OLFS will need to be modified to recognize the new request, generate the correct boundary and start values, and complete the response by building the correct response headers and closing 'separator' headers.
== Example Response Document ==

Revision as of 20:30, 14 August 2009

Division of Labor

Figure 1. Building the DataDDX with the OLFS making the initial and final headers. The BES/libdap/handler will have to make the 'part' headers except for the closing segment. This provides an easy way for the OLFS to combine several DataDDX responses in a single MP MIME document (easy in the sense that the OLFS does not have to pull apart the response from the BES).
Figure 2. This design would be good form the standpoint of support for protocols that don't want data responses in multipart MIME documents, but it's far from optimal since the DDS has to be built twice for each request

The BES and OLFS work together to support DAP, however, it is the job of the OLFS to correctly implement the DAP. The BES understands certain aspects of the DAP, but it does not actually build correct responses from DAP requests. Instead the OLFS reads a request and asks the BES to build parts of the correct response that it then assembles to make the complete response document.

To build a DataDDX response, the OLFS and BES will work together as shown in Figure 1. When the OLFS receives and processes a DAP4 DataDDX request, it will first parse from that request the information the BES needs and send that along with both the Multipart MIME (MPM) start and boundary values. The BES will use a handler to build a C++ DDS object and return the MPM document that forms the payload of the DataDDX response, with a few wrinkles, such as why the start and boundary values need to be passed by the OLFS to the BES, to be described shortly.

When the OLFS begins to receive the MPM payload for the DataDDX response, it will first write out the MIME headers for the response. Following that, it will stream the response from the BES, and following that, it will append (send to the stream) the closing section of the document.

Note: In Figures 1 and 2 the OLFS, the sequence of interactions between theBES and a generic Handler are shown using a UML sequence diagram. That type of diagram is intended to be used for objects and their methods, but I thought it was a useful diagram even though the idea of the BES, OLFS or a handler as an object is a bit of a stretch.

Why this design?

This design implements the DataDDX response with as little modification to the libdap and BES software as possible (and minimal changes to the OLFS, too). However, we want it to be usable as a building block for two things besides DAP4: A SOAP interface for DAP servers, where several DataDDX responses can be held in a single SOAP envelope; and implementations of DAP4 that use transport protocols other than HTTP.

Like most designs intended for several uses, this one is a compromise. Figure 2 shows a design that would be better at addressing the issues associated with support for other transport protocols and which might offer advantages to SOAP as well. In fact, it also has a cleaner separation of duties because the OLFS takes over complete responsibility for building the headers of the MPM document. But the cost is that the BES must build two DDS objects - an expensive operation, especially for large datasets. So the compromise reached is that some effort to build MIME headers, those that are part of the payload of the response, is moved into the BES/Handler/libdap software. However, because support for a SOAP interface means that we should be able to stuff several of these MPM documents into one SOAP envelope, the trailing MPM separator information is not written by the BES/Handler/libdap. Instead, the trailing separator that closes the document needs to be written by the OLFS.

With the design presented, the BES/... software can behave as efficiently as the current DataDDS code but each response can be used as a building block in a SOAP response - one response concatenated after the other - that is closed by the OLFS so the BES/Handler software does need to know about the way the responses are being packaged. At the same time the response format is likely usable by other transport protocols because either that implementation can use MPM as its payload or take apart the response (since MIME is designed to make that operation fairly easy).

Handler/libdap

The Handler running inside the BES uses libdap's DDS class and its methods to make the DDX and XDR-encoded data blob. The DDS::print_xml() method will take the Content-Id to be used with the data blob section as a parameter. A new method will be added to build the data blob.

A new set of functions will be added to libdap's collection of MIME-header writers to build the MPM boundary sections. Two functions will be needed: One to write the boundary headers for the DDX and one to write the boundary headers for the data blob.

The libdap DODSFilter class will get a new method that will package all of these calls in a way that's similar to the existing DODSFilter::send_das(), ..., send_data() methods

BES

The BES will need to be modified to respond to a get dataddx request and will need to accept the start and boundary values for the MPM document from the OLFS. Each handler will likely need some modification to support the new response type.

OLFS

The OLFS will need to be modified to recognize the new request, generate the correct boundary and start values, and complete the response by building the correct response headers and closing 'separator' headers.

Example Response Document