Hyrax - Configuring The OLFS To Work With Multiple BES's: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
No edit summary
 
(6 intermediate revisions by one other user not shown)
Line 4: Line 4:
=Top Level (''root'') BES=
=Top Level (''root'') BES=


Every installation of Hyrax requires a top level (or ''root'' level) BES.** This BES has a ''prefix'' of "/" (the forward slash character).
Every installation of Hyrax requires a top level (or ''root'' level) BES.** This BES has a ''prefix'' of "/" (the forward slash character). The prefix is a URL token between the server address/port and catalog root used to designate a particular BES instance in the case that multiple Back-End-Servers are available to a single OLFS. The default (for a single BES) is no additional tag, designated by "/". The prefix is used to provide a mapping for each BES connected to the OLFS to URI space serviced by the OLFS.


In a single BES deployment this BES would contain all of the data resources to be made visible in Hyrax. In the THREDDS ''catalog.xml'' file each top level directory/collection would have it's own <''datasetScan''> element.
In a single BES deployment this BES would contain all of the data resources to be made visible in Hyrax. In the THREDDS ''catalog.xml'' file each top level directory/collection would have it's own <''datasetScan''> element.
Line 18: Line 18:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>localhost</host>
             <host>localhost</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 37: Line 37:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>localhost</host>
             <host>localhost</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 44: Line 44:
             <prefix>/sst</prefix>
             <prefix>/sst</prefix>
             <host>comet.test.org</host>
             <host>comet.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 52: Line 52:
The first one is running on the same system as the OLFS,  the second on ''comet. test.org''. The second BES is mapped to the prefix /sst. So the URL:
The first one is running on the same system as the OLFS,  the second on ''comet. test.org''. The second BES is mapped to the prefix /sst. So the URL:
  http://localhost:8080/opendap/
  http://localhost:8080/opendap/
Will return the directory view at the top level of the first BES, running on the same system ads the OLFS. The URL:
Will return the directory view at the top level of the first BES, running on the same system as the OLFS. The URL:
  http://localhost:8080/opendap/sst
  http://localhost:8080/opendap/sst
Will return the directory view at the top level of the second BES, running on ''comet.test.org''.
Will return the directory view at the top level of the second BES, running on ''comet.test.org''.
Line 63: Line 63:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>server0.test.org</host>
             <host>server0.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 70: Line 70:
             <prefix>/sst</prefix>
             <prefix>/sst</prefix>
             <host>server1.test.org</host>
             <host>server1.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 77: Line 77:
             <prefix>/chl-a</prefix>
             <prefix>/chl-a</prefix>
             <host>server2.test.org</host>
             <host>server2.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 84: Line 84:
             <prefix>/salinity</prefix>
             <prefix>/salinity</prefix>
             <host>server3.test.org</host>
             <host>server3.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 112: Line 112:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>server0.test.org</host>
             <host>server0.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 130: Line 130:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>server0.test.org</host>
             <host>server0.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 137: Line 137:
             <prefix>/sst</prefix>
             <prefix>/sst</prefix>
             <host>server5.test.org</host>
             <host>server5.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 143: Line 143:
     </Handler>
     </Handler>
</pre>
</pre>
'''It will not appear in the top level directory unless you create a ''mount point''.''' This simply means that on the file sytem served by the ''root'' BES you would need to create a directory called "sst" in the top of the directory tree that the ''root'' BES is exposing. In other words, simply create a directory called "sst" in the same directory that contains the  "Test" and "data" directories on server0.test.org. After you did that your top level directory would look like this:
'''It will not appear in the top level directory unless you create a ''mount point''.''' This simply means that on the file system served by the ''root'' BES you would need to create a directory called "sst" in the top of the directory tree that the ''root'' BES is exposing. In other words, simply create a directory called "sst" in the same directory that contains the  "Test" and "data" directories on server0.test.org. After you did that your top level directory would look like this:


[[Image:topDir2.png|Top Level Directory]]
[[Image:topDir2.png|Top Level Directory]]
Line 156: Line 156:
             <prefix>/</prefix>
             <prefix>/</prefix>
             <host>server0.test.org</host>
             <host>server0.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 163: Line 163:
             <prefix>/GlobalTemperature </prefix>
             <prefix>/GlobalTemperature </prefix>
             <host>server1.test.org</host>
             <host>server1.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 170: Line 170:
             <prefix>/GlobalTemperature/NorthAmerica</prefix>
             <prefix>/GlobalTemperature/NorthAmerica</prefix>
             <host>server2.test.org</host>
             <host>server2.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 177: Line 177:
             <prefix>/GlobalTemperature/NorthAmerica/Canada </prefix>
             <prefix>/GlobalTemperature/NorthAmerica/Canada </prefix>
             <host>server3.test.org</host>
             <host>server3.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 184: Line 184:
             <prefix>/GlobalTemperature/NorthAmerica/USA </prefix>
             <prefix>/GlobalTemperature/NorthAmerica/USA </prefix>
             <host>server4.test.org</host>
             <host>server4.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 191: Line 191:
             <prefix>/GlobalTemperature/Europe/France </prefix>
             <prefix>/GlobalTemperature/Europe/France </prefix>
             <host>server4.test.org</host>
             <host>server4.test.org</host>
             <port>10002</port>
             <port>10022</port>
             <ClientPool maximum="10" />
             <ClientPool maximum="10" />
         </BES>
         </BES>
Line 220: Line 220:
                     &lt;prefix&gt;/&lt;/prefix&gt;
                     &lt;prefix&gt;/&lt;/prefix&gt;
                     &lt;host&gt;server0.test.org&lt;/host&gt;
                     &lt;host&gt;server0.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 227: Line 227:
                     &lt;prefix&gt;/GlobalTemperature &lt;/prefix&gt;
                     &lt;prefix&gt;/GlobalTemperature &lt;/prefix&gt;
                     &lt;host&gt;server1.test.org&lt;/host&gt;
                     &lt;host&gt;server1.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 234: Line 234:
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica&lt;/prefix&gt;
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica&lt;/prefix&gt;
                     &lt;host&gt;server2.test.org&lt;/host&gt;
                     &lt;host&gt;server2.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 241: Line 241:
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica/Canada &lt;/prefix&gt;
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica/Canada &lt;/prefix&gt;
                     &lt;host&gt;server3.test.org&lt;/host&gt;
                     &lt;host&gt;server3.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 248: Line 248:
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica/USA &lt;/prefix&gt;
                     &lt;prefix&gt;/GlobalTemperature/NorthAmerica/USA &lt;/prefix&gt;
                     &lt;host&gt;server4.test.org&lt;/host&gt;
                     &lt;host&gt;server4.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 255: Line 255:
                     &lt;prefix&gt;/GlobalTemperature/Europe/France &lt;/prefix&gt;
                     &lt;prefix&gt;/GlobalTemperature/Europe/France &lt;/prefix&gt;
                     &lt;host&gt;server4.test.org&lt;/host&gt;
                     &lt;host&gt;server4.test.org&lt;/host&gt;
                     &lt;port&gt;10002&lt;/port&gt;
                     &lt;port&gt;10022&lt;/port&gt;
                     &lt;ClientPool maximum="10" /&gt;
                     &lt;ClientPool maximum="10" /&gt;
                 &lt;/BES&gt;
                 &lt;/BES&gt;
Line 275: Line 275:
             &lt;/Handler&gt;
             &lt;/Handler&gt;


             &lt;Handler className="opendap.bes. ThreddsDispatchHandler" /&gt;
             &lt;Handler className="opendap.bes.ThreddsDispatchHandler" /&gt;


         &lt;/HttpGetHandlers&gt;
         &lt;/HttpGetHandlers&gt;

Latest revision as of 20:45, 19 December 2013

Configuring Hyrax to use multiple BES backends is straight forward. It will require that you edit the olfs.xml file and possible the catalog.xml file.


Top Level (root) BES

Every installation of Hyrax requires a top level (or root level) BES.** This BES has a prefix of "/" (the forward slash character). The prefix is a URL token between the server address/port and catalog root used to designate a particular BES instance in the case that multiple Back-End-Servers are available to a single OLFS. The default (for a single BES) is no additional tag, designated by "/". The prefix is used to provide a mapping for each BES connected to the OLFS to URI space serviced by the OLFS.

In a single BES deployment this BES would contain all of the data resources to be made visible in Hyrax. In the THREDDS catalog.xml file each top level directory/collection would have it's own <datasetScan> element.


**Note: The word root here has absolutely nothing to do with the login account called root associated with the super user or system administrator.

Single BES Example (Default)

Here is the <Handler> element in an olfs.xml that defines the opendap.bes.BESManager file that configures the OLFS to use a single BES, the default configuration arrangement for Hyrax:

    <Handler className="opendap.bes.BESManager">
        <BES>
            <prefix>/</prefix>
            <host>localhost</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
    </Handler>

The BES is running on the same system as the OLFS, and it's prefix is correctly set to "/". This BES will handle all data requests directed at the OLFS and will expose it's top level directory/collection/catalog in the URI space of the OLFS here:

http://localhost:8080/opendap/

The THREDDS catalog.xml file for this should contain a <datasetScan> element for each of the top level directories | collections | catalogs that the BES exposes at the above URI.

Remember: There must be one (but only one) BES configured with the <prefix> set to "/" in your olf.xml file.

Multiple BES examples

Here is a BESManager <Handler> element that defines two BES's:

    <Handler className="opendap.bes.BESManager">

        <BES>
            <prefix>/</prefix>
            <host>localhost</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/sst</prefix>
            <host>comet.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>

    </Handler>

The first one is running on the same system as the OLFS, the second on comet. test.org. The second BES is mapped to the prefix /sst. So the URL:

http://localhost:8080/opendap/

Will return the directory view at the top level of the first BES, running on the same system as the OLFS. The URL:

http://localhost:8080/opendap/sst

Will return the directory view at the top level of the second BES, running on comet.test.org.

You can repeat this pattern to add more BES's to the configuration. This next example shows a configuration with 4 BES's: The root BES, and 3 others:

    <Handler className="opendap.bes.BESManager">

        <BES>
            <prefix>/</prefix>
            <host>server0.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/sst</prefix>
            <host>server1.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>

        <BES>
            <prefix>/chl-a</prefix>
            <host>server2.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>

        <BES>
            <prefix>/salinity</prefix>
            <host>server3.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>

    </Handler>

Note that in this example:

  1. The root BES is not necessarily running on the same host as the OLFS.
  2. Every BES has a different prefix.
  3. The OLFS would direct requests so that requests to:

Mount points

In a multiple BES installation each additional BES must have a mount point within the exposed hierarchy of collections for it to be visible in Hyrax.

Consider, if you have this configuration:

    <Handler className="opendap.bes.BESManager">

        <BES>
            <prefix>/</prefix>
            <host>server0.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
    </Handler>

And the top level directory for the root BES looks like this:

Top Level Directory

If you add another BES, like this:

    <Handler className="opendap.bes.BESManager">

        <BES>
            <prefix>/</prefix>
            <host>server0.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/sst</prefix>
            <host>server5.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
    </Handler>

It will not appear in the top level directory unless you create a mount point. This simply means that on the file system served by the root BES you would need to create a directory called "sst" in the top of the directory tree that the root BES is exposing. In other words, simply create a directory called "sst" in the same directory that contains the "Test" and "data" directories on server0.test.org. After you did that your top level directory would look like this:

Top Level Directory

This holds true for any arrangement of BESs that you make. The location of the mount point will depend on your configuration, and how you organize things. Here is a more complex example.

Consider this configuration:

    <Handler className="opendap.bes.BESManager">

        <BES>
            <prefix>/</prefix>
            <host>server0.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/GlobalTemperature </prefix>
            <host>server1.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/GlobalTemperature/NorthAmerica</prefix>
            <host>server2.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/GlobalTemperature/NorthAmerica/Canada </prefix>
            <host>server3.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/GlobalTemperature/NorthAmerica/USA </prefix>
            <host>server4.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
        <BES>
            <prefix>/GlobalTemperature/Europe/France </prefix>
            <host>server4.test.org</host>
            <port>10022</port>
            <ClientPool maximum="10" />
        </BES>
        
    </Handler>
  • The mount point "GlobalTemperature" must be in the top of the directory tree that the root BES on server0.test.org is exposing.
  • The mount point "NorthAmerica" must be in the top of the directory tree that the BES on server1.test.org is exposing.
  • The mount point "Canada" must be in the top of the directory tree that the BES on server2.test.org is exposing.
  • The mount point "USA" must be in the top of the directory tree that the BES on server2.test.org is exposing.
  • The mount point "France" must be located at "GlobalTemperature/Europe/France" relative to the top of the directory tree that the BES on server0.test.org is exposing.

Complete olfs.xml with multiple BES installations example

<?xml version="1.0" encoding="UTF-8"?>
<OLFSConfig>

    <DispatchHandlers>

        <HttpGetHandlers>

            <Handler className="opendap.bes.BESManager">

                <BES>
                    <prefix>/</prefix>
                    <host>server0.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>
                
                <BES>
                    <prefix>/GlobalTemperature </prefix>
                    <host>server1.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>
                
                <BES>
                    <prefix>/GlobalTemperature/NorthAmerica</prefix>
                    <host>server2.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>
                
                <BES>
                    <prefix>/GlobalTemperature/NorthAmerica/Canada </prefix>
                    <host>server3.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>
                
                <BES>
                    <prefix>/GlobalTemperature/NorthAmerica/USA </prefix>
                    <host>server4.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>
                
                <BES>
                    <prefix>/GlobalTemperature/Europe/France </prefix>
                    <host>server4.test.org</host>
                    <port>10022</port>
                    <ClientPool maximum="10" />
                </BES>

            </Handler>

            <Handler className="opendap.coreServlet.SpecialRequestDispatchHandler" />
            
            <Handler className="opendap.bes.VersionDispatchHandler" />

            <Handler className="opendap.bes.DirectoryDispatchHandler">
                <DefaultDirectoryView>OPeNDAP</DefaultDirectoryView>
            </Handler>

            <Handler className="opendap.bes.DapDispatchHandler" />

            <Handler className="opendap.bes.FileDispatchHandler" >
                <!-- <AllowDirectDataSourceAccess /> -->
            </Handler>

            <Handler className="opendap.bes.ThreddsDispatchHandler" />

        </HttpGetHandlers>

        <HttpPostHandlers>
            <Handler className="opendap.coreServlet.SOAPRequestDispatcher" >
                <OpendapSoapDispatchHandler>opendap.bes.SoapDispatchHandler</OpendapSoapDispatchHandler>
            </Handler>
        </HttpPostHandlers>

    </DispatchHandlers>

</OLFSConfig>