Hyrax - User Identification (Authentication): Difference between revisions
(→IDV) |
(→URS) |
||
Line 458: | Line 458: | ||
https://54.172.97.47/opendap/data/nc/coads_climatology.nc | https://54.172.97.47/opendap/data/nc/coads_climatology.nc | ||
When I committed the edit (aka hit Enter) ToolsUI popped up a dialog box that indicated that the ''uat.urs.earthdata.nasa.gov'' server wanted my credentials | When I committed the edit (aka hit Enter) ToolsUI popped up a dialog box that indicated that the ''uat.urs.earthdata.nasa.gov'' server wanted my credentials. | ||
[[File:ToolsUIAuthDialog.png|broder|frame|left|ToolsUI Authentication Dialog]] | |||
<div style="clear: both"></div> | |||
I entered them, clicked the save password check box, and clicked the ''OK'' button. ToolsUI was then able to access the requested resource. | |||
==== LDAP ==== | ==== LDAP ==== |
Revision as of 21:47, 19 November 2014
Overview
This document is intended to help those that have been asked to deploy Hyrax into an environment where authentication of users is required. In many such cases Hyrax will be integrated into an existing instance of the Apache Web server (httpd) where authentication services are already configured and in use. In other cases people will be setting up a standalone instance of Tomcat and will be needing to configure it to use one of the supported authentication services. This document means to address both situations.
Terms
- Authentication
- This is the process of confirming the identity of the user. The end result is a User ID (uid or UID) which may be accessed by the software components via (both?) the Apache API (mod_*) and the Java ServletAPI (Tomcat servlets) used to trigger authorization policy chains or may be logged along with relevant request information.
- Identity Provider (IdP)
- Also known as an Identity Assertion Provider an Identity Provider (IdP) is a service that provides authentication and identity information services. An IdP is a kind of provider that creates, maintains, and manages identity information for principals and provides principal authentication to other service providers within a federation, such as with web browser profiles.
- Service Provider (SP)
- A Service Provider (SP) is a Web Service that utilizes an IdP service to determine the identity of it's users. Or more broadly, a role donned by a system entity where the system entity provides services to principals or other system entities.
With respect to this document Hyrax/Tomcat, and Hyrax/Tomcat/Apache each become part of an SP through the installation and configuration of software components such as mod_shib (shibboleth) .
See Service Providers, Identity Providers & Security Token Services explained for more.
Apache httpd Authentication Services Configuration
There are many authentication methods available for use with our friend httpd. Each of them may have a unique installation and configuration activity. There are some common changes that must be made to the Tomcat configuration regardless of the authentication method employed by Apache. We'll cover those first and the examine LDAP, Shibboleth, and NASA URS IdP configurations for Apache httpd.
- NB
- If you are deploying Hyrax with an existing Apache service then it is likely that all you have to do is configure Tomcat correctly and define security constrains in Apache
Configure Apache to Proxy Tomcat
In /etc/httpd/conf.d
create a file called hyrax.conf . Edit the file and add following:
<Proxy *>
AddDefaultCharset Off
Order deny,allow
Allow from all
</Proxy>
ProxyPass /opendap ajp://localhost:8009/opendap
ProxyPassReverse /opendap ajp://localhost:8009/opendap
This will expose the web application "opendap" (aka Hyrax) through Apache.
Tomcat/Hyrax
The primary result of the Apache authentication (the uid string) must be correctly transmitted to Tomcat. On the Tomcat side we have to open the way for this by configuring a AJP Connector
object. This is done by editing the file:
- $CATALINA_HOME/conf/server.xml
Edit the server.xml file, and find the AJP Connector element on port 8009. It should look something like this:
<Connector port="8009" protocol="AJP/1.3" />
This line may be "commented out," with <!-- on a line before and --> on a line after. If so, remove those lines. If you cannot find the AJP connector element, simply create it from the code above.
- In order to receive authentication information from Apache, you must disable Tomcat's native authentication. Set the tomcatAuthentication attribute to "false" - see below for an example.
- If your Apache web server is using SSL/HTTPS (and it should be), you need to tell Tomcat about that fact so that it can construct internal URLs correctly. Set the scheme attribute to "https" and the proxyPort attribute to "443" - see below for an example.
- For increased security, disable access to the connector from anywhere but the local system. Set the address attribute to "127.0.0.1" - see below for an example.
When you are finished making changes, your connector should look something like this:
<Connector
port="8009"
protocol="AJP/1.3"
redirectPort="443"
scheme="https"
address="127.0.0.1"
enableLookups="false"
tomcatAuthentication="false"
/>
- port
- The Connector will listen on port 8009.
- protocol
- The protocol is AJP/1.3.
- redirectPort
- Secure redirects to port 443 which is the nominal Apache HTTPS port.
- scheme
- Ensure that scheme is HTTPS
- address
- The loopback address (127.0.0.1) ensures that only local requests for the connection will be serviced.
- enableLookups
- A value of true enable DNS look ups so that the Servlet API call HttpServletRequest.getRemoteHost() returns a host name and not an IP address. Set this to false to improve performance.
- tomcatAuthentication
- A value of false will allow the Tomcat engine to receive authentication information (the uid and in some cases other attributes) from Apache httpd. A value of true will cause Tomcat to ignore Apache authentication results in favor of it's own.
Restart Tomcat to load the new configuration. Now your Tomcat web applications should see all of the Apache authentication attributes. To retrieve them, use request.getRemoteUser() or request.getAttribute("ATTRIBUTE NAME"). Note that request.getAttributeNames() may not list all available attributes – you must request each attribute individually by name.
Apache Security Constraints
While the detail of the Apache security constraints differ somewhat from one IdP to the next what is consistent is that you will need to define a security constraint on Hyrax inside the chain of httpd.conf files. The most simple example, that you want all users of the Hyrax instance to be authenticated, would look something like this:
<Location /opendap>
AuthType YourFavoriteAuthTypeHere
require valid-user
</Location>
Where the require valid-user
attribute makes the requirement that all accessors be authenticated. More complete examples are presented in the LDAP and Shibboleth sections below.
LDAP (mod_ldap, mod_authnz_ldap)
In order to get Apache httpd to use LDAP authentication you will have configure an Apache security constraint on the Hyrax web application. For this example we will configure Apache to utilize the Forum Systems public LDAP server
- All user passwords are password.
- Groups and Users:
- mathematicians
- riemann
- gauss
- euler
- euclid
- scientists
- einstein
- newton
- galieleo
- tesla
- mathematicians
Create and edit the file /etc/httpd/conf.d/ldap.conf
.
Add the following at the end of the file:
# You may need to uncomment these two lines...
# LoadModule ldap_module modules/mod_ldap.so
# LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
# You may want to comment out this line once you have it working.
LogLevel debug
<Location /opendap >
Order deny,allow
Deny from all
AuthType Basic
AuthName "Forum Systems Public LDAP Server- Login with user id"
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPURL ldap://ldap.forumsys.com:389/dc=example,dc=com
AuthLDAPBindDN "cn=read-only-admin,dc=example,dc=com"
AuthLDAPBindPassword password
AuthLDAPGroupAttributeIsDN off
ErrorDocument 401 "Please use your username and password to login into this Hyrax server"
Require valid-user
Satisfy any
</Location>
Restart Apache httpd and you should now need to authenticate to access anything in /opendap
Shibboleth (mod_shib)
The Shibboleth wiki provides excellent documentation on how to get Shibboleth authentication services working with Tomcat. This is primarily an Apache httpd activity.
Basically you need to follow the instructions for a Native Java Install and remember - Hyrax does not use either Spring or Grails.
Installation
The logical starting point for this is with the Native Java SP Installation:
But as far as the organization of the work is concerned it is really the last page you need to process, as it will send you off to do a platform dependent Shibboleth Native Service Provider for Apache installation which needs to be completed, working, and configured before you'll return to the Native Java SP Installation to enable the part where Tomcat and mod_shib pass authenticated user information into Tomcat.
The document path on the Natvie Java Install wiki page will send you off to do Shibboleth Native Service Provider installation which is platform dependent:
- https://wiki.shibboleth.net/confluence/display/SHIB2/Installation
- Install a Native Service Provider on your target system.
- In the initial testing section for Linux they suggest accessing the Status page https://localhost/Shibboleth.sso/Status, but you may have to use the loopback address to be able to do so: https://127.0.0.1/Shibboleth.sso/Status
Return to the Native Java SP Installation and complete the instructions there.
Configuration
Once the SP installation is completed go to the Native SP Configuration page:
Read that page and then follow the link to the instructions for Apache:
Follow those instructions.
- Do not be confused by the section Making URLs Used by mod_shib Get Properly Routed. While you must add this Location directive to "reveal" the shibboleth module to the world don't think the URL https://yourhost/Shibboleth.sso is a valid access point to the module. That URL may always return a Shibboleth error page even if mod_shib and shibd are configured and working correctly.
- Read and understand the section Enabling the Module for Authentication
The Shibboleth instructions should have had you add something like this:
<Location /opendap>
AuthType shibboleth
ShibRequestSetting requireSession 1
require valid-user
</Location>
to httpd.conf. This will require users to authenticate to access any part of Hyrax which may be exactly what you want. If you want more fine grained control you may want use multiple Location
elements with different require
attributes. For example:
<Location /opendap>
AuthType shibboleth
ShibCompatWith24 On
require shibboleth
</Location>
<Location /opendap/AVHRR>
AuthType shibboleth
ShibCompatWith24 On
ShibRequestSetting requireSession 1
require valid-user
</Location>
</apache>
In this example the first Location
establishes Shibboleth as the authentication tool for the entire /opendap application path, and enables the Shibboleth module over the entire Hyrax Server.
- Since there is no
ShibRequestSetting requireSession 1
line it does not require a user to be logged in order to access the path. - The
require shibboleth
command activates mod_shib for all of Hyrax.
The second Location
states that only valid-users may have access "/opendap/AVHRR" URL path.
- The
require valid-user
command requires user authentication. - The
AuthType
command is set toshibboleth
so mod_shib will be called upon to perform the authentication.
For more examples and better understanding see the Apache Configuration section of the Shibboleth wiki.
URS OAuth2 (mod_auth_urs)
The instructions for configuring the Apache module mod_auth_urs can be found here:
https://wiki.earthdata.nasa.gov/display/URSFOUR/Apache+URS+Authentication+Module
(NB: The instructions are clear and complete but you have to be a registered URS user with permissions to access that page in order to read it.)
Once I had it installed all that was need was to create the file /etc/httpd/conf.d/urs.conf and add the following content:
# Load the URS module
LoadModule auth_urs_module modules/mod_auth_urs.so
#
# Enable Debugging
LogLevel debug
#
# START - URS module configuration
# The directory where session data will be stored
#
UrsSessionStorePath /var/tmp/urs/session
# The address of the authentication server
#
UrsAuthServer https://uat.urs.earthdata.nasa.gov
# The authentication endpoint
#
UrsAuthPath /oauth/authorize
# The token exchange endpoint
#
UrsTokenPath /oauth/token
#
#
# END - URS module configuration
# Place a URS security constraint on the Hyrax service
<Location /opendap >
AuthType UrsOAuth2
AuthName "URS_AuthTest"
Require valid-user
UrsAuthGroup HyraxDataServer
UrsClientId 04xHKVaNdYNzCBG6KB7-Ig
UrsAuthCode MDR4SEtWYU5kWU56Q0JHNktCNy1JZzpKSEdqa2hmZzY3OA==
UrsRedirectUrl https://54.172.97.47/opendap/login
UrsIdleTimeout 600
UrsActiveTimeout 36000
UrsIPCheckOctets 2
UrsUserProfileEnv uid URS_USER
UrsUserProfileEnv email_address URS_EMAIL
UrsUserProfileEnv first_name URS_FIRST
UrsUserProfileEnv last_name URS_LAST
UrsAccessErrorUrl /urs403.html
</Location>
Assuming that the AJP proxy for Tomcat is already done simply restart Tomcat and access Hyrax with your URS credentials.
Tomcat Authentication Services Configuration
LDAP
Shibboleth
URS OAuth2
DAP Client Authentication
Many users access DAP servers using a browser as their primary software interface. However there is also a growing group of users that utilize either:
- A "smart" tool. Where "smart" means that the tool understands how to interact with a DAP service and construct DAP queries for data and use that in higher level client side activities like GUI based graphs, image display, selection, navigation etc.
- A command line tool such as "wget" or "curl" that can be used to extract data from a DAP service, but the URL construction is left to other software.
In both these examples we want these client software applications to be able to manage authentication without user intervention, otherwise the automation benefits of these tools is lost.
curl (a.k.a. lib_curl)
URS
I was able to use curl to retrieve URS authentication protected resources using the following technique.
First in my home directory I created a .netrc file and set its file permissions to read only for owner:
[spooky:~] ndp% touch .netrc
[spooky:~] ndp% chmod 600 .netrc
[spooky:~] ndp% ls -l .netrc
-rw-------@ 1 ndp staff 92 Nov 13 06:08 .netrc
Then I edited the .netrc file and associated my URS credentials with the URS IdP instance utilized by my target DAP server:
machine uat.urs.earthdata.nasa.gov
login my_urs_uid
password my_urs_password
I could then access the top level directory of the URS authentication enabled Hyrax server with the following curl command:
curl -k -n -c ursCookies -b ursCookies -L --url https://54.172.97.47/opendap
What is happening here?
- -k
- This tells curl to accept self-signed certificates. This is ok for working with trusted (as in your own) "test" services but should be removed for working with production systems. Because: Security, Chain-Of-Trust, etc.
- -n
- This tells curl to use that ~/.netrc file I created.
- -c ursCookies
- This tells curl to stash cookies in the file ursCookies
- -b ursCookies
- This tells curl to read cookies from the file ursCookies
- -L
- This tells curl to follow redirects, which is a must for any OAuth2 flow.
- --url https://54.172.97.47/opendap
- The desired URL, protected by the URS OAuth2 authentication flow.
In order to retrieve multiple URLs with out reauthenticating you can use multiple instances of the --url parameter:
curl -k -n -c ursCookies -b ursCookies -L --url https://54.172.97.47/opendap --url https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds --url https://54.172.97.47/opendap/data/nc/coads_climatology.nc.dds
Or, since curl is actually pretty smart about using cookies and such you can also make multiple curl requests with the same cookies and it won't have to reauthenticate with URS once it's authenticated the first time:
curl -k -n -c ursCookies -b ursCookies -L --url https://54.172.97.47/opendap
curl -k -n -c ursCookies -b ursCookies -L --url https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds
curl -k -n -c ursCookies -b ursCookies -L --url https://54.172.97.47/opendap/data/nc/coads_climatology.nc.dds
LDAP
Shibboleth
wget
URS
The wget documentation indicates that wget understands to use the .netrc file that we created for curl, and happily it appears to work, as long as other things are in place. Consider this wget command:
wget --load-cookies cookies --save-cookies cookies --keep-session-cookie --no-check-certificate https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds
What's happening here?
- --load-cookies cookies
- Load cookies from the file "cookies"
- --save-cookies cookies
- Save cookies to the file "cookies"
- --keep-session-cookie
- Save session cookies.
- --no-check-certificate
- Do not check the authenticity of the (self signed) certificates. This is good for testing against your own servers running with self-signed certificates in that this switch will allow you to experience success when interacting with such servers. However, this switch breaks the chain of trust and may allow bad things to happen if used on the open internets. THus, for regular use, do not include this switch!
- https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds
- The URL to retrieve.
Here's the output of said wget request:
[spooky:olfs/testsuite/urs] ndp% wget --load-cookies cookies --save-cookies cookies --keep-session-cookie --no-check-certificate https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds --2014-11-14 11:22:18-- https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds Connecting to 54.172.97.47:443... connected. WARNING: cannot verify 54.172.97.47's certificate, issued by `/C=US/ST=RI/L=Narragansett/O=OPeNDAP Inc./OU=Engineering/CN=54.172.97.47/emailAddress=support@opendap.org': Self-signed certificate encountered. HTTP request sent, awaiting response... 302 Found Location: https://uat.urs.earthdata.nasa.gov/oauth/authorize?app_type=401&client_id=04xHKVaNdYNzCBG6KB7-Ig&response_type=code&redirect_uri=https%3A%2F%2F54.172.97.47%2Fopendap%2Flogin&state=aHR0cHM6Ly81NC4xNzIuOTcuNDcvb3BlbmRhcC9kYXRhL25jL2Zub2MxLm5jLmRkcw [following] --2014-11-14 11:22:19-- https://uat.urs.earthdata.nasa.gov/oauth/authorize?app_type=401&client_id=04xHKVaNdYNzCBG6KB7-Ig&response_type=code&redirect_uri=https%3A%2F%2F54.172.97.47%2Fopendap%2Flogin&state=aHR0cHM6Ly81NC4xNzIuOTcuNDcvb3BlbmRhcC9kYXRhL25jL2Zub2MxLm5jLmRkcw Resolving uat.urs.earthdata.nasa.gov... 198.118.243.34, 2001:4d0:241a:4089::91 Connecting to uat.urs.earthdata.nasa.gov|198.118.243.34|:443... connected. WARNING: certificate common name `uat.earthdata.nasa.gov' doesn't match requested host name `uat.urs.earthdata.nasa.gov'. HTTP request sent, awaiting response... 401 Unauthorized Connecting to uat.urs.earthdata.nasa.gov|198.118.243.34|:443... connected. WARNING: certificate common name `uat.earthdata.nasa.gov' doesn't match requested host name `uat.urs.earthdata.nasa.gov'. HTTP request sent, awaiting response... 302 Found Location: https://54.172.97.47/opendap/login?code=a590cfc189783e29a7b8ab3ce1e0357618cbab3f590e7268a26e7ad1f7cf899d&state=aHR0cHM6Ly81NC4xNzIuOTcuNDcvb3BlbmRhcC9kYXRhL25jL2Zub2MxLm5jLmRkcw [following] --2014-11-14 11:22:20-- https://54.172.97.47/opendap/login?code=a590cfc189783e29a7b8ab3ce1e0357618cbab3f590e7268a26e7ad1f7cf899d&state=aHR0cHM6Ly81NC4xNzIuOTcuNDcvb3BlbmRhcC9kYXRhL25jL2Zub2MxLm5jLmRkcw Connecting to 54.172.97.47:443... connected. WARNING: cannot verify 54.172.97.47's certificate, issued by `/C=US/ST=RI/L=Narragansett/O=OPeNDAP Inc./OU=Engineering/CN=54.172.97.47/emailAddress=support@opendap.org': Self-signed certificate encountered. HTTP request sent, awaiting response... 302 Found Location: https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds [following] --2014-11-14 11:22:21-- https://54.172.97.47/opendap/data/nc/fnoc1.nc.dds Connecting to 54.172.97.47:443... connected. WARNING: cannot verify 54.172.97.47's certificate, issued by `/C=US/ST=RI/L=Narragansett/O=OPeNDAP Inc./OU=Engineering/CN=54.172.97.47/emailAddress=support@opendap.org': Self-signed certificate encountered. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/plain] Saving to: `fnoc1.nc.dds' [ <=> ] 197 --.-K/s in 0s 2014-11-14 11:22:22 (7.23 MB/s) - `fnoc1.nc.dds' saved [197] [spooky:olfs/testsuite/urs] ndp% more fnoc1.nc.dds Dataset { Int16 u[time_a = 16][lat = 17][lon = 21]; Int16 v[time_a = 16][lat = 17][lon = 21]; Float32 lat[lat = 17]; Float32 lon[lon = 21]; Float32 time[time = 16]; } fnoc1.nc;
It appears that wget followed the first redirect to uat.urs.earthdata.nasa.gov
, where the URS server responded with a "401 Unauthorized" (thanks to the the app_type=401 query parameter in the redirect URL provided by mod_auth_urs). After getting the 401 wget resubmits the request with the authentication credentials and the URS server accepts them and redirects wget back to the mod_auth_urs server to complete the request.
LDAP
Shibboleth
IDV
The Integrated Data Viewer is GUI driven data client that is based around the CDM/NetCDF data model and utilizes that NetCDF-Java (and thus the Java DAP implementation) to access remote DAP datasets. Because it has a GUI it can retrieve (and cache for later) users credentials directly from the user. Since IDV utilizes the Java-NetCDF library to access DAP resources then in theory if it works for IDV then it should work for all the other clients that use the Java-NetCDF library.
URS
For testing I utilized my AWS test service, configured to require URS authentication for all access of Hyrax.
I downloaded the latest version of IDV (5.0u2 on 11/19/14) and installed it on my local system. I launched IDV and then attempted to choose a new dataset by starting with the "Data" menu: Data > Choose Data > From A Web Server
In the resulting pane I entered the AWS test service URL for our friend coads_climatology.nc:
https://54.172.97.47/opendap/data/nc/coads_climatology.nc
When I committed the edit (aka hit Enter) IDV popped up a dialog box that indicated that the uat.urs.earthdata.nasa.gov server wanted my credentials. I entered them, clicked the save password check box, and clicked the OK button. IDV was then able to access the requested resource. After the first successful access other resources at the AWS server were also available, but without an additional authentication challenge being presented to the user.
LDAP
Shibboleth
ToolsUI
The ToolsUI application is a simple is GUI driven data client that is based around the CDM/NetCDF data model and utilizes that NetCDF-Java (and thus the Java DAP implementation) to access remote DAP datasets. Because it has a GUI it can retrieve (and cache for later) users credentials directly from the user.
URS
For testing I utilized my AWS test service, configured to require URS authentication for all access of Hyrax.
I downloaded the latest version of ToolsUI (4.5 on 11/19/14) and installed it on my local system. I launched ToolsUI using the command line:
java -Xmx1g -jar toolsUI-4.5.jar
I selected the Viewer tab, and entered the AWS test service URL for our friend coads_climatology.nc:
https://54.172.97.47/opendap/data/nc/coads_climatology.nc
When I committed the edit (aka hit Enter) ToolsUI popped up a dialog box that indicated that the uat.urs.earthdata.nasa.gov server wanted my credentials.
I entered them, clicked the save password check box, and clicked the OK button. ToolsUI was then able to access the requested resource.
LDAP
Shibboleth
ncdump
ncdump utilizes the NetCDF-C library (which uses the the "oc" library, which uses lib_curl under the hood) to access DAP resources so in theory if it works for ncdump then it should work for other clients that use the NetCDF-C library.
URS
The following works with the ncdump (and oc client) code bundled with NetCDF-4.3.2 Previous versions including 4.3.1 will not work.
Edit (create as needed) the file .dodsrc in your current working directory and set its file permissions to read only for owner:
[spooky:~] ndp% touch .dodsrc
[spooky:~] ndp% chmod 600 .dodsrc
[spooky:~] ndp% ls -l . dodsrc
-rw-------@ 1 ndp staff 92 Nov 13 06:08 . dodsrc
Add your URS credentials and associate them with the URS authenticated DAP service that you wish to access. This will not work if you (correctly) associate the URS credentials with the URS server (go figure):
[https://target.dap.server.org]HTTP.CREDENTIALS.USER=your_uid
[https://target.dap.server.org]HTTP.CREDENTIALS.PASSWORD=**************
Here is a typical .dodsrc file.
# OPeNDAP client configuation file. See the OPeNDAP
# users guide for information.
USE_CACHE=0
# Cache and object size are given in megabytes (20 ==> 20Mb).
MAX_CACHE_SIZE=20
MAX_CACHED_OBJ=5
IGNORE_EXPIRES=0
CACHE_ROOT=/Users/ndp/.dods_cache/
DEFAULT_EXPIRES=1
ALWAYS_VALIDATE=1
# Request servers compress responses if possible?
# 1 (yes) or 0 (false).
DEFLATE=0
# Proxy configuration:
# PROXY_SERVER=<protocol>,<[username:password@]host[:port]>
# NO_PROXY_FOR=<protocol>,<host|domain>
# AIS_DATABASE=<file or url>
[https://target.dap.server.org]HTTP.CREDENTIALS.USER=your_uid
[https://target.dap.server.org]HTTP.CREDENTIALS.PASSWORD=**************
This version of ncdump (the oc client) appears to ignore the .dodsrc HTTP.COOKIEJAR and HTTP.COOKIEFILE parameters. This means that ncdump will authenticate each time it is invoked.