Hyrax - Apache Integration: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
 
(45 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Request for Alternate Solutions:''' ''If these suggestions work for you then we are all happy. If you have a better method for achieving the same or similar goals then we would love to hear about it. If you're stuck and can't figure out how to achieve your goals, get in touch and I will try to help. - ndp (ndp at opendap dot org) Or email the OPeNDAP tech support list (opendap-tech at unidata dot ucar dot edu); see [http://www.opendap.org/mailLists/index.html mail lists] for information about the list.''
== Overview ==
The problem of linking Tomcat with Apache has been greatly reduced as of Apache 2.2. In previous incarnations of Apache & Tomcat, it was fairly complex. To see how to solve the problem for older versions of Apace, see the [[old Apache Integration instructions]]. What follows are the instructions for Apache 2.2 and Tomcat 6.x.


== Prerequisites ==
* Apace 2.2 or greater
* Tomcat 6.x or greater
* mod_proxy_ajp installed in Apache (typically this is present in 2.2+)


==Overview==
== Connecting Tomcat to Apache  ==
If you installing a DAP server for the first time, or if you are not concerned about the URL's for your data changing when you install Hyrax, then just run Tomcat as a standalone server. It can even be configured to operate on port 80, just like Apache. If however you want to
=== Tomcat ===
* Keep your existing data access URL's intact
You have to create the AJP connector in the ''conf/server.xml'' file:
* Work with Apache security ( SSL authentication etc.)
* Work with the load balancing features found in APache and Tomcat
Then, read on.


Many people deploying Hyrax have been using previous versions of the OPeNDAP servers with their data. These OPeNDAP servers required the Apache web server and utilized CGI to deliver their functionality. This is no longer the case, as Hyrax is a Java servlet designed to run in conjunction with Tomcat. However many OPeNDAP administrators may wish to keep the URL hierarchies of their existing systems unchanged and simply replace their existing implementation of the OPeNDAP server with the new server. To this end Tomcat needs to be integrated with the Apache installation. The directions here offer methods for integrating Hyrax with Apache version 2.x. If you are using earlier versions of Apache (1.x or older) you might wish to consider upgrading Apache to a more current version (version 2.x really is better...), otherwise these instructions may or may not be helpful.


There are 2 basic steps to integrating Hyrax (and Tomcat) with Apache and Tomcat. The first step is to integrate Tomcat with Apache using the [http://tomcat.apache.org/connectors-doc/index.html  Apache Tomcat Connector], the second step is to map your old OPeNDAP (or even DODS) URLs to the new service using the Apache rewrite module (mod_rewrite).
<source lang="xml" >
    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector
        port="8009"
        protocol="AJP/1.3"
        redirectPort="443" <!-- Redirect to the Apache Web Server secure port -->
        scheme="https" <!-- Use TLS to connect -->
        address="127.0.0.1" <!-- Only allow connections from this host -->
        <!-- Setting tomcatAuthentication to 'false' will allow tomcat web applications
                to get user session information from Apache, such as uid and other user properties. -->
        tomcatAuthentication="true"
        />
</source>


Tomcat must be running and listening on a port (which one depends on how you set things up) for it to interact with Apache.
This line will enable AJP connections to the 8009 port of your tomcat server (localhost for example).


===mod_jk===
=== Apache ===
In the example below, pay special attention to the ''protocol'' part of the proxy URL - it uses '''ajp://''' and not <nowiki>'http://'</nowiki>.<br>
Add this to Apache's ''httpd.conf'' file:


Most discussions of integrating Tomcat and Apache start (and often end with) an Apache module called mod_jk, which allows Tomcat to work as part of Apache. However, mod_jk is generally not installed by default with Apache (while the rewrite and proxy modules are) so you will need to get (possibly compile) and install mod_jk .
<source lang="apache">
<Proxy *>
    AddDefaultCharset Off
    Order deny,allow
    Allow from all
</Proxy>
ProxyPass /opendap ajp://localhost:8009/opendap
ProxyPassReverse /opendap ajp://localhost:8009/opendap
</source>


===mod_rewrite===
NB: It's possible to embed these in a ''VirtualHost'' directive.


The simplest method for integrating Tomcat with Apache while preserving preexisting URL hierarchies is to use a mod_rewrite to map your old data URL's into the new service.
=== How It Works ===
ProxyPass and ProxyPassReverse are classic reverse proxy directives used to forward the stream to another location.
''ajp://...'' is the AJP connector location (your tomcat's server host/port)


A web client will connect through HTTP to ''<nowiki>http://localhost/</nowiki>'' (supposing your apache2 server is running on localhost), the ''mod_proxy_ajp'' will forward you request transparently using the AJP protocol to the tomcat application server on localhost:8009.


----
== Apache Compressed Responses ==
----
----


==Install Software Components==
Many OPeNDAP clients accept compressed responses. This can greatly increase the efficiency of the client/server interaction by diminishing the number of bytes actually transmitted over "the wire". Compression can reduce the number of bytes transmitted by an order of magnitude for many datasets!
===Install Hyrax===
Install Hyrax as described in the [[Hyrax - Installation Instructions | installation instructions]]. No matter how you want things to end up you have to do this first.


===Install mod_jk===
Tomcat provides native compression support for the GZIP compression mechanism, however it is NOT turned on by default. More perversely, even if you have configured Tomcat to provide compressed responses, if you are using AJP to proxy Tomcat through the Apache web server compression will not be enabled unless you configure the Apache web server to compress responses. This is because Tomcat NEVER compresses responses sent over AJP.


'''Before proceeding make sure you have installed the ''mod_jk'' library in your Apache ''modules'' directory.'''
When you configure your Apache web server to provide compressed responses you will probably want to make sure that Apache doesn't apply compression to images (In general images are already compressed and there is little to gain by attempting to compress them and a lot of CPU cycles to burn if you try)
Please read and follow the [http://tomcat.apache.org/connectors-doc/index.html  Apache instructions to install mod_jk] into your instance of Apache.


====OS-X Tip====
[http://httpd.apache.org/docs/2.0/mod/mod_deflate.html Apache compression module]
If you end up building mod_jk from source for an installation on OS-X 10.5.x (Leopard) you may find that the configure script fails to correctly detect the system architecture. This will cause Apache to complain when you try to use mod_jk:


:''httpd: Syntax error on line 512 of /private/etc/apache2/httpd.conf: Cannot load /usr/libexec/apache2/mod_jk.so into server: dlopen(/usr/libexec/apache2/mod_jk.so, 10): no suitable image found.  Did find:\n\t/usr/libexec/apache2/mod_jk.so: mach-o, but wrong architecture''
[http://www.askapache.com/htaccess/apache-speed-compression.html AskApache Speed Tips]
 
This can be cured by adding the architecture designation via a compiler flags switch on the configure script:
 
: ''./configure CFLAGS="-arch x86_64" ...''
 
(Thanks to: [http://blog.lo-fi.net/2007/10/leopard-for-web-developer-installing.html lo-fi] for the tip!)
 
===Install mod_rewrite===
 
Depending on your Apache installation you may need to rebuild/compile Apache to enable ''rewrite''. You should first look at the ''http.conf'' configuration file. Find the ''LoadModule'' directives and look for the lines that load the proxy and rewrite modules. They should look something like this:
LoadModule rewrite_module modules/mod_rewrite.so
 
If you see the module loaded then you're set to go, skip to the next section.
 
If not then you will probably need to compile Apache from source and enable the rewrite and proxy modules.
 
#Make sure you have the Apache source.
#Run the Apache configure script with the --enable-rewrite and --enable-proxy switches. Minimally your configure command should look like this: <br/><br/><code>./configure --enable-rerwrite --enable-proxy</code><br/><br/>
#(Re)compile.
#(Re)install.
----
----
----
 
== Configure Tomcat ==
Most distributions of Tomcat come with the AJP connector enabled. You should check your instance of Tomcat and confirm this.
 
Look in the Tomcat ''server.xml'' file to confirm the enablement of the AJP 1.3 connector.  Inside of ''$CATALINA_HOME/conf/server.xml'' add (or possibly uncomment) the AJP connector definition inside of the service named Catalina:
<pre>
    &lt;Service name=&quot;Catalina&quot;&gt;
    .
    .
    .
    &lt;!-- Define an AJP 1.3 Connector on port 8009 --&gt;
    &lt;Connector port=&quot;8009&quot;
    enableLookups=&quot;false&quot; redirectPort=&quot;8443&quot; protocol=&quot;AJP/1.3&quot; /&gt;
    .
    .
    .
    &lt;/Service&gt;
</pre>
----
----
----
 
==Configure mod_jk==
The Apache Tomcat Connector (aka mod_jk) has a large list of features (such as load balancing options) that are beyond the scope of this discussion. We will review a simple configuration to get things started.
 
 
===workers.properties file ===
Mod_jk uses a '''workers.properties''' file that defines the workers that are connecting to Tomcat. Here is a minimum example of a workers.properties file for Hyrax:
 
# Define 1 real worker
worker.list=hyrax
#
# Set properties for hyrax worker
#
# Define it as an AJP1.3 protocol worker.
worker.hyrax.type=ajp13
# Hostname or IP address for the tomcat instance that is running Hyrax
worker.hyrax.host=localhost
#Define the port for the AJP connector for the Tomcat instance
worker.hyrax.port=8009
 
Remember - the ''workers.properties'' file must direct your mod_jk worker to the AJP port being used by you Tomcat instance as defined in the  ''$CATALINA_HOME/conf/server.xml'' file. This is typically NOT  port 8080 but port 8009.
 
The workers.properties can be placed anywhere on the Apache host system, but is typically located in ''/etc/httpd/conf/workers.properties''


[http://betterexplained.com/articles/how-to-optimize-your-site-with-gzip-compression/ BetterExplained explains Apache Compression]


=== httpd.conf ===
=== httpd.conf ===


Once you have saved the workers.properties file you will need to edit the Apache configuration, http.conf. The httpd.conf file is typically located in one of:
You will need to add (something like) the following to your Apache web server's httpd.conf file:
* /etc/httpd/conf/
* /etc/httpd2/conf/
* /usr/local/apache/conf/
* /etc/apache2/ (on my OS-X 10.5 system)


At the bottom of '''httpd.conf''' you will need to add the following, localized to you specific system:
<source lang="apache">
#
# Compress everything except images.
#
<Location />
    # Insert filter
    SetOutputFilter DEFLATE


     # Load mod_jk module. This location was determined when you installed mod_jk
     # Netscape 4.x has some problems...
    LoadModule jk_module  ''location/of/''mod_jk.so
     BrowserMatch ^Mozilla/4 gzip-only-text/html
    # Where to find workers.properties
    JkWorkersFile /etc/httpd/conf/mod_jk/workers.properties
    # Where to put jk shared memory
     JkShmFile    /var/log/httpd/mod_jk.shm
    # Where to put jk logs
    JkLogFile    /var/log/httpd/mod_jk.log
    # Set the jk log level [debug/error/info]
    JkLogLevel    info
    # Select the timestamp log format
    JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
    # Send servlet for context /examples to worker named worker1
    JkMount  /opendap* hyrax


*You may not need to the line:
    # Netscape 4.06-4.08 have some more problems
:: <tt>LoadModule jk_module  ''location/of/''mod_jk.so''</tt>
    BrowserMatch ^Mozilla/4\.0[678] no-gzip
:If it already exists elsewhere in httpd.conf


* The last line of the ''mod_jk'' configuration:
    # MSIE masquerades as Netscape, but it is fine
     JkMount  /opendap/* hyrax
     # BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
: Maps all incoming requests whose local URL matches '''opendap/*''' to the ''hyrax'' worker defined in the '''workers.properties''' file.


    # NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
    # the above regex won't work. You can use the following
    # workaround to get the desired effect:
    BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html


This should complete the configuration of mod_jk. Restart Apache (apachectl -k restart) and see if in fact hitting you Apache server at the opendap context brings you to your Hyrax service:
    # Don't compress images
    SetEnvIfNoCase Request_URI \
    \.(?:gif|jpe?g|png)$ no-gzip dont-vary


:<nowiki> http://my.host/opendap/</nowiki>
    # Make sure proxies don't deliver the wrong content
    Header append Vary User-Agent env=!dont-vary
</Location>
</source>
</font>


If not then you will need to trouble shoot you mod_jk installation and configuration.
== Apache Authentication ==
----
----
----


== Preserving old  DAP/DODS data URL's ==
Hyrax may deployed into service stacks in which ''httpd'' is expected to handle the work of authenticating users. In order for Tomcat (and thus Hyrax) to be able to receive the users login name and attributes from ''httpd'' the following things need to be done to the Tomcat configuration.
Using the Apache rewrite module (mod_rewrite) we can map olds data URL's that were serviced by early versions of the DAP server to the new Hyrax installation.


=== mod_rewrite ===
In the ''$CATALINA_HOME/conf/server.xml'' file the default definition of the AJP connector typically looks like:
'''Advice''': ''If you are not familiar with [http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html Apache mod_rewrite go read about it now], before you do anything else.''


Once you have installed Apache with the ''rewrite'' module (mod_rewrite) enabled you will need to edit Apache's httpd.conf file. Add the following lines:
<source lang="xml" >
<pre>
     <!-- Define an AJP 1.3 Connector on port 8009 -->
     # Enable the rewrite module
     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
    RewriteEngine on
</source>
     # Target it's logging somewhere useful
    RewriteLog /var/log/httpd/rewrite.log
    # Turn on logging (Set to 0 to disable)
    RewriteLogLevel 2


    # Uses a reverse proxy to enable mapping old OPeNDAP URL's to Tomcat.
This line may be "commented out," with &lt;!-- on a line before and --&gt; on a line after. If so, remove those lines. If you cannot find the AJP connector element, simply create it from the code above. You will need to add several attributes to the Connector element.
    RewriteRule ^/cgi-bin/nph-dods(.*) /opendap/$1 [P]
</pre>


Assuming that your old OPeNDAP server was accessed via ''http://your.server/cgi-bin/nph-dods/'' this will map it to the AJP Connector (mod_jk) that is tied (via the ''hyrax'' worker defined in the '''workers.properties''' file) to the Hyrax service running in the Tomcat engine. You will probably need to rewrite rule to suit your previous server configuration. If you used ''Alias'' or ''AliasMatch'' with your old server, add more ''RewriteRule'' directives to get that same behavior.
* Set the <code>tomcatAuthentication</code> attribute to "false", this must be done in order to receive authentication information from Apache.
* Configure the connector to use SSL - If your Apache web server is using SSL/HTTPS (and it should be), you need to tell Tomcat about that fact so that it can construct internal URLs correctly.
** Set the <code>scheme</code> attribute to "https".
** Set the <code>proxyPort</code> attribute to Apache httpd's secure socket, typically "443" (This ensures that secure traffic gets routed through Apache httpd and and then through the AJP connector to Tomcat, allowing httpd's authentication/authorization stack to be invoked on the request).
* Restrict access to the AJP Connector. By disabling access to the connector from anywhere but the local system you prevent system probing from the greater world. To do this, set the <code>address</code> attribute to "127.0.0.1".


When you are finished making changes, your connector should look something like this:


----
<source lang="xml" >
----
    <!-- Define an AJP 1.3 Connector on port 8009 -->
----
    <Connector
== Apache AddEncoding directives ==
        port="8009"
'''Note''': If you have ''AddEncoding'' directives in your Apache configuration, those will likely need to be replaced with ''AddType''. If present, the ''AddEncoding'' directives will cause Apache 2.x to report that any page, such as the HTML form interface, is compressed, even though it is not. This problem can be very hard to track down.
        protocol="AJP/1.3"
        redirectPort="443"
        scheme="https"
        address="127.0.0.1"
        tomcatAuthentication="false"
        />
</source>


<pre>
Restart Tomcat to load the new configuration. Now Tomcat/Hyrax should see all of the authentication attributes from ''httpd''.  
    # AddEncoding allows you to have certain browsers uncompress
    # information on the fly. Note: Not all browsers support this.
    # Despite the name similarity, the following Add* directives have nothing
    # to do with the FancyIndexing customization directives above
    #
    # AddEncoding x-compress .Z
    # AddEncoding x-gzip .gz .tgz
    #
    # If the AddEncoding directives above are commented-out, then you
    # probably should define those extensions to indicate media types:
    #
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
</pre>


Restart Apache (assuming Tomcat is already running) and you should be on your way.
NB: You may wish review Tomcat documentation for the AJP Connector as there many attributes/options that can be used to tune performance. [http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html Here's a link to the Tomcat 7 AJP Connector docs]

Latest revision as of 13:00, 23 June 2016

Overview

The problem of linking Tomcat with Apache has been greatly reduced as of Apache 2.2. In previous incarnations of Apache & Tomcat, it was fairly complex. To see how to solve the problem for older versions of Apace, see the old Apache Integration instructions. What follows are the instructions for Apache 2.2 and Tomcat 6.x.

Prerequisites

  • Apace 2.2 or greater
  • Tomcat 6.x or greater
  • mod_proxy_ajp installed in Apache (typically this is present in 2.2+)

Connecting Tomcat to Apache

Tomcat

You have to create the AJP connector in the conf/server.xml file:


    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector 
        port="8009" 
        protocol="AJP/1.3" 
        redirectPort="443" <!-- Redirect to the Apache Web Server secure port -->
        scheme="https" <!-- Use TLS to connect -->
        address="127.0.0.1" <!-- Only allow connections from this host -->
        <!-- Setting tomcatAuthentication to 'false' will allow tomcat web applications 
                to get user session information from Apache, such as uid and other user properties. -->
        tomcatAuthentication="true" 
        />

This line will enable AJP connections to the 8009 port of your tomcat server (localhost for example).

Apache

In the example below, pay special attention to the protocol part of the proxy URL - it uses ajp:// and not 'http://'.
Add this to Apache's httpd.conf file:

<Proxy *>
    AddDefaultCharset Off
    Order deny,allow
    Allow from all
</Proxy>
 
ProxyPass /opendap ajp://localhost:8009/opendap
ProxyPassReverse /opendap ajp://localhost:8009/opendap

NB: It's possible to embed these in a VirtualHost directive.

How It Works

ProxyPass and ProxyPassReverse are classic reverse proxy directives used to forward the stream to another location. ajp://... is the AJP connector location (your tomcat's server host/port)

A web client will connect through HTTP to http://localhost/ (supposing your apache2 server is running on localhost), the mod_proxy_ajp will forward you request transparently using the AJP protocol to the tomcat application server on localhost:8009.

Apache Compressed Responses

Many OPeNDAP clients accept compressed responses. This can greatly increase the efficiency of the client/server interaction by diminishing the number of bytes actually transmitted over "the wire". Compression can reduce the number of bytes transmitted by an order of magnitude for many datasets!

Tomcat provides native compression support for the GZIP compression mechanism, however it is NOT turned on by default. More perversely, even if you have configured Tomcat to provide compressed responses, if you are using AJP to proxy Tomcat through the Apache web server compression will not be enabled unless you configure the Apache web server to compress responses. This is because Tomcat NEVER compresses responses sent over AJP.

When you configure your Apache web server to provide compressed responses you will probably want to make sure that Apache doesn't apply compression to images (In general images are already compressed and there is little to gain by attempting to compress them and a lot of CPU cycles to burn if you try)

Apache compression module

AskApache Speed Tips

BetterExplained explains Apache Compression

httpd.conf

You will need to add (something like) the following to your Apache web server's httpd.conf file:

#
# Compress everything except images.
#
<Location />
    # Insert filter
    SetOutputFilter DEFLATE

    # Netscape 4.x has some problems...
    BrowserMatch ^Mozilla/4 gzip-only-text/html

    # Netscape 4.06-4.08 have some more problems
    BrowserMatch ^Mozilla/4\.0[678] no-gzip

    # MSIE masquerades as Netscape, but it is fine
    # BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

    # NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
    # the above regex won't work. You can use the following
    # workaround to get the desired effect:
    BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

    # Don't compress images
    SetEnvIfNoCase Request_URI \
    \.(?:gif|jpe?g|png)$ no-gzip dont-vary

    # Make sure proxies don't deliver the wrong content
    Header append Vary User-Agent env=!dont-vary
</Location>

Apache Authentication

Hyrax may deployed into service stacks in which httpd is expected to handle the work of authenticating users. In order for Tomcat (and thus Hyrax) to be able to receive the users login name and attributes from httpd the following things need to be done to the Tomcat configuration.

In the $CATALINA_HOME/conf/server.xml file the default definition of the AJP connector typically looks like:

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

This line may be "commented out," with <!-- on a line before and --> on a line after. If so, remove those lines. If you cannot find the AJP connector element, simply create it from the code above. You will need to add several attributes to the Connector element.

  • Set the tomcatAuthentication attribute to "false", this must be done in order to receive authentication information from Apache.
  • Configure the connector to use SSL - If your Apache web server is using SSL/HTTPS (and it should be), you need to tell Tomcat about that fact so that it can construct internal URLs correctly.
    • Set the scheme attribute to "https".
    • Set the proxyPort attribute to Apache httpd's secure socket, typically "443" (This ensures that secure traffic gets routed through Apache httpd and and then through the AJP connector to Tomcat, allowing httpd's authentication/authorization stack to be invoked on the request).
  • Restrict access to the AJP Connector. By disabling access to the connector from anywhere but the local system you prevent system probing from the greater world. To do this, set the address attribute to "127.0.0.1".

When you are finished making changes, your connector should look something like this:

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector 
        port="8009" 
        protocol="AJP/1.3" 
        redirectPort="443" 
        scheme="https"
        address="127.0.0.1" 
        tomcatAuthentication="false" 
        />

Restart Tomcat to load the new configuration. Now Tomcat/Hyrax should see all of the authentication attributes from httpd.

NB: You may wish review Tomcat documentation for the AJP Connector as there many attributes/options that can be used to tune performance. Here's a link to the Tomcat 7 AJP Connector docs