https://docs.opendap.org/api.php?action=feedcontributions&feedformat=atom&user=Kneumiller 2026-05-06T16:46:01Z User contributions MediaWiki 1.38.4 https://docs.opendap.org/index.php?title=File:OPeNDAP-Logo_Large.png&diff=13512 2023-10-18T20:04:49Z

<p>Kneumiller: Kneumiller uploaded a new version of File:OPeNDAP-Logo Large.png</p> <hr /> <div>The rally big logo with a clear background.</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13319 2022-01-20T16:54:49Z

<p>Kneumiller: /* Setup CentOS-8 */</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;Load the basic software development environment plus the additional packages of openjpeg2, jasper, and libtirpc<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside the hyrax-dependencies to build and install gdal and hdf4<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13318 2022-01-20T16:51:42Z

<p>Kneumiller: /* Setup CentOS-8 */</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;Load the basic software development environment plus the additional packages of openjpeg2, jasper, and libtirpc<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside the hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13317 2022-01-20T16:51:02Z

<p>Kneumiller: /* Setup CentOS-8 */</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;Load the basic software development environment plus the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside the hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13316 2022-01-20T16:50:10Z

<p>Kneumiller: /* Setup CentOS-8 */</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need a basic software development environment plus the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside the hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=ConfigureCentos&diff=13315 2022-01-20T16:48:13Z

<p>Kneumiller: /* CentOS versions */</p> <hr /> <div>[[HowTo_guides|&lt;&lt; back to HowTo Guides]]<br /> <br /> This describes how to set up a vanilla CentOS computer so that we can build RPM binaries for the Hyrax data server. This was taken from notes describing how I configured a machine with a fresh install of CentOS 5.3 to build Hyrax. Augmented for CentOS 6. <br /> <br /> If you're installing this on a ''virtual'' host using VMware, see the section at the end of this document for some tips on making that VM run smoothly.<br /> <br /> == CentOS versions ==<br /> <br /> These instructions have been amended to cover:<br /> <br /> * CentOS-6.9 (64-bit)<br /> * CentOS-7.? (64-bit)<br /> <br /> == Initial configuration ==<br /> <br /> Set up general build stuff that you need. The RPM packages you need are listed here. Using ''yum'' (''sudo yum install &lt;package&gt;'') is the easiest way to install these packages on CentOS/RedHat. The package names for Ubuntu are a bit different, but are easy to figure out. Our CI builds are done on Ubuntu 14 (as of Nov 2017), so it's certainly possible to build there:<br /> * java-1.7.0-openjdk<br /> * java-1.7.0-openjdk-devel<br /> * ant<br /> * ant-junit<br /> * junit<br /> * git <br /> * gcc-c++ <br /> * flex <br /> * bison <br /> * autoconf <br /> * automake <br /> * libtool <br /> * emacs <br /> * openssl-devel <br /> * libuuid-devel <br /> * readline-devel <br /> * zlib-devel <br /> * bzip2, bzip2-devel <br /> * libjpeg-devel <br /> * libxml2-devel <br /> * curl-devel <br /> * libicu-devel<br /> * vim<br /> * bc<br /> <br /> If you're feeling lucky, you can put all of the packages on a single line<br /> :''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel vim bc''<br /> <br /> '''Notes:''' <br /> *''Java 1.7, Ant and CentOS 6''. I found that ant 1.7 (the default with CentOS 6) would not work with Java 1.8. To get our code to build with Java 1.7, I used ''alternatives --config java'' to configure 1.7 and then ''export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64''. I could not compile the olfs otherwise. <br /> *''7 June 2016 - CentOS 7 RPM Build''. I had to add '''bzip2-devel''', '''libicu-devel''', and '''bc''' to the set of packages to get the RPM to build on CentOS 7. It may be the case this was true of CentOS 6 as well and it just slipped by.<br /> *''11 Nov 2017'' - The latest versions of CentOS 6 and 7 had no problems using the packages above. I did not test Java 8 or Java 9, but Java 8 has no problems in development configurations. Tomcat7, 8, and 9 from RPMs all work as well, at least on my development machines.<br /> <br /> ==== Autotools ====<br /> The autotools with CentOS 6 should be fine - the scripts will complain if otherwise. However, the newer autoconf/make support some nice features, so installing these newer version is not a waste. On a Test machine, this should not be done. If you don't install the new versions, make sure to use yum to install autoconf, automake and libtool. That is<br /> :''yum install autoconf automake libtool''<br /> If you don't want to use RPM for these, then:<br /> * download the latest versions of autoconf, automake, and libtool from GNU: http://ftp.gnu.org/gnu/<br /> * ./configure, make, make install<br /> <br /> ==== RPM Construction Tools ====<br /> * ''yum install rpm-devel rpm-build redhat-rpm-config''<br /> <br /> == Hyrax specific configuration ==<br /> Set up things that are specific to Hyrax<br /> <br /> It's best to use as much of the RPM 'ecosystem' as possible because this is what will be on other people's computers; using newer stuff, like the lastest HDF5 library is only going to cause incompatibilities. But to do that for Hyrax, you need to get beyond the stock set of repositories that ''yum'' searches. <br /> <br /> ==== Add the EPEL EL repo to yum ====<br /> * Information about EPEL/EL is here: https://fedoraproject.org/wiki/EPEL.<br /> * For CentOS 6, you likely want the rpm package here: http://mirror.pnl.gov/epel/6/i386/repoview/epel-release.html<br /> <br /> &lt;blockquote&gt;'''Note''' Use CentOS 6 if possible for Hyrax at this time (Sept 2014). We'll be moving to combined v.6 and v.7 support soon; v.5 is no longer supported. Some CentOS 5 information is included here because this was initially written back when it was the latest version.&lt;/blockquote&gt;<br /> <br /> ===== Centos 5.x =====<br /> But don't use this unless you really have too...<br /> <br /> * google EPEL/EL5 and <br /> * find &amp; download ''epel-release-5.4.noarch.rpm''<br /> * install it: ''rpm -i epel-release-5.4.noarch.rpm''<br /> <br /> ===== Centos 6.x =====<br /> Use this until we start supporting CentOS 7<br /> <br /> * google EPEL/EL6 and <br /> * find &amp; download ''epel-release-6.5.noarch.rpm''<br /> * install it: ''rpm -i epel-release-6.5.noarch.rpm''<br /> <br /> ==== Data type dependencies: NetCDF, HDF4, HDF5 ====<br /> * ''yum install netcdf-devel'' (note: On CentOS-6.2 you'll get netcdf-4 and on CentOS-5.3 you'll get netcdf-3)<br /> * ''yum install hdf-devel'' (note: 'hdf', not 'hdf4')<br /> * ''yum install hdf5-devel''<br /> <br /> ==== Other dependencies ====<br /> * ''yum install libicu-devel''<br /> * ''yum install cfitsio-devel''<br /> * ''yum install cppunit-devel''<br /> <br /> ==== ... If you're feeling lucky ====<br /> As with the 'build packages,' here are all of the 'hyrax-specific' packages so you can use just one Yum command<br /> :''netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''.<br /> Note that you'll still need to set up the EPEL repository.<br /> <br /> == VMware configuration tips ==<br /> It's best if you enable a shared directory so that you can pass stuff back and forth between the host OS and the Guest OS (i.e., the Virtual machine). That will only work if you have the ''VMware tools'' installed in/on the guest.<br /> <br /> * Install VMware-tools. This process varies, but it's pretty easy for all platforms. On VMware Fusion, look under the ''Virtual Machine'' menu for the item that says ''Install VMware Tools''. This will download lump of code and, for CentOS, mount it on '/media/VMware Tools' (yes, there's a space in the directory name). Copy the *.tar.gz file to some place like your home directory, unpack it and read the INSTALL file. For an initial installation, the typical process is to run ''vmware-install.pl''.<br /> * Under the configuration/options menu (the little wrench thing in Fusion), choose ''Sharing''. Make a folder with an obvious name (e.g., ''vmware'') and turn sharing on. Now, anything you put in there when on either the OS will be available to the other OS. On linux, this directory is located at ''/mnt/hgfs/&lt;&lt;name&gt;&gt;''.<br /> <br /> == Support for DEB packages ==<br /> It is possible to use ''fpm'' to build debian packages from RPM packages. See [https://ask.fedoraproject.org/en/question/52141/how-to-convert-rpm-package-to-deb/ how-to-convert-rpm-package-to-deb] for an explanation. In summary, make a few directories to hold parts of the RPM, load files from the RPM into them and then use ''fpm'' to build the debian package. In addition to these steps, extract the package description text to a file and pass that into ''fpm'' and look at the dependencies for the RPMs, find the corresponding Ubuntu dependencies and provide them to ''fpm''.<br /> <br /> == problems not addressed ==<br /> * There is no JUnit for the OLFS build<br /> * There is no graphviz so the ''doc'' targets for libdap and bes will fail</div>

Kneumiller https://docs.opendap.org/index.php?title=ConfigureCentos&diff=13314 2022-01-20T16:47:29Z

<p>Kneumiller: Removed the CentOS 8 stuff, it is now in the hyrax source build</p> <hr /> <div>[[HowTo_guides|&lt;&lt; back to HowTo Guides]]<br /> <br /> This describes how to set up a vanilla CentOS computer so that we can build RPM binaries for the Hyrax data server. This was taken from notes describing how I configured a machine with a fresh install of CentOS 5.3 to build Hyrax. Augmented for CentOS 6. <br /> <br /> If you're installing this on a ''virtual'' host using VMware, see the section at the end of this document for some tips on making that VM run smoothly.<br /> <br /> == CentOS versions ==<br /> <br /> These instructions have been amended to cover:<br /> <br /> * CentOS-6.9 (64-bit)<br /> * CentOS-7.? (64-bit)<br /> * CentOS-8 (64-bit)<br /> <br /> == Initial configuration ==<br /> <br /> Set up general build stuff that you need. The RPM packages you need are listed here. Using ''yum'' (''sudo yum install &lt;package&gt;'') is the easiest way to install these packages on CentOS/RedHat. The package names for Ubuntu are a bit different, but are easy to figure out. Our CI builds are done on Ubuntu 14 (as of Nov 2017), so it's certainly possible to build there:<br /> * java-1.7.0-openjdk<br /> * java-1.7.0-openjdk-devel<br /> * ant<br /> * ant-junit<br /> * junit<br /> * git <br /> * gcc-c++ <br /> * flex <br /> * bison <br /> * autoconf <br /> * automake <br /> * libtool <br /> * emacs <br /> * openssl-devel <br /> * libuuid-devel <br /> * readline-devel <br /> * zlib-devel <br /> * bzip2, bzip2-devel <br /> * libjpeg-devel <br /> * libxml2-devel <br /> * curl-devel <br /> * libicu-devel<br /> * vim<br /> * bc<br /> <br /> If you're feeling lucky, you can put all of the packages on a single line<br /> :''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel vim bc''<br /> <br /> '''Notes:''' <br /> *''Java 1.7, Ant and CentOS 6''. I found that ant 1.7 (the default with CentOS 6) would not work with Java 1.8. To get our code to build with Java 1.7, I used ''alternatives --config java'' to configure 1.7 and then ''export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64''. I could not compile the olfs otherwise. <br /> *''7 June 2016 - CentOS 7 RPM Build''. I had to add '''bzip2-devel''', '''libicu-devel''', and '''bc''' to the set of packages to get the RPM to build on CentOS 7. It may be the case this was true of CentOS 6 as well and it just slipped by.<br /> *''11 Nov 2017'' - The latest versions of CentOS 6 and 7 had no problems using the packages above. I did not test Java 8 or Java 9, but Java 8 has no problems in development configurations. Tomcat7, 8, and 9 from RPMs all work as well, at least on my development machines.<br /> <br /> ==== Autotools ====<br /> The autotools with CentOS 6 should be fine - the scripts will complain if otherwise. However, the newer autoconf/make support some nice features, so installing these newer version is not a waste. On a Test machine, this should not be done. If you don't install the new versions, make sure to use yum to install autoconf, automake and libtool. That is<br /> :''yum install autoconf automake libtool''<br /> If you don't want to use RPM for these, then:<br /> * download the latest versions of autoconf, automake, and libtool from GNU: http://ftp.gnu.org/gnu/<br /> * ./configure, make, make install<br /> <br /> ==== RPM Construction Tools ====<br /> * ''yum install rpm-devel rpm-build redhat-rpm-config''<br /> <br /> == Hyrax specific configuration ==<br /> Set up things that are specific to Hyrax<br /> <br /> It's best to use as much of the RPM 'ecosystem' as possible because this is what will be on other people's computers; using newer stuff, like the lastest HDF5 library is only going to cause incompatibilities. But to do that for Hyrax, you need to get beyond the stock set of repositories that ''yum'' searches. <br /> <br /> ==== Add the EPEL EL repo to yum ====<br /> * Information about EPEL/EL is here: https://fedoraproject.org/wiki/EPEL.<br /> * For CentOS 6, you likely want the rpm package here: http://mirror.pnl.gov/epel/6/i386/repoview/epel-release.html<br /> <br /> &lt;blockquote&gt;'''Note''' Use CentOS 6 if possible for Hyrax at this time (Sept 2014). We'll be moving to combined v.6 and v.7 support soon; v.5 is no longer supported. Some CentOS 5 information is included here because this was initially written back when it was the latest version.&lt;/blockquote&gt;<br /> <br /> ===== Centos 5.x =====<br /> But don't use this unless you really have too...<br /> <br /> * google EPEL/EL5 and <br /> * find &amp; download ''epel-release-5.4.noarch.rpm''<br /> * install it: ''rpm -i epel-release-5.4.noarch.rpm''<br /> <br /> ===== Centos 6.x =====<br /> Use this until we start supporting CentOS 7<br /> <br /> * google EPEL/EL6 and <br /> * find &amp; download ''epel-release-6.5.noarch.rpm''<br /> * install it: ''rpm -i epel-release-6.5.noarch.rpm''<br /> <br /> ==== Data type dependencies: NetCDF, HDF4, HDF5 ====<br /> * ''yum install netcdf-devel'' (note: On CentOS-6.2 you'll get netcdf-4 and on CentOS-5.3 you'll get netcdf-3)<br /> * ''yum install hdf-devel'' (note: 'hdf', not 'hdf4')<br /> * ''yum install hdf5-devel''<br /> <br /> ==== Other dependencies ====<br /> * ''yum install libicu-devel''<br /> * ''yum install cfitsio-devel''<br /> * ''yum install cppunit-devel''<br /> <br /> ==== ... If you're feeling lucky ====<br /> As with the 'build packages,' here are all of the 'hyrax-specific' packages so you can use just one Yum command<br /> :''netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''.<br /> Note that you'll still need to set up the EPEL repository.<br /> <br /> == VMware configuration tips ==<br /> It's best if you enable a shared directory so that you can pass stuff back and forth between the host OS and the Guest OS (i.e., the Virtual machine). That will only work if you have the ''VMware tools'' installed in/on the guest.<br /> <br /> * Install VMware-tools. This process varies, but it's pretty easy for all platforms. On VMware Fusion, look under the ''Virtual Machine'' menu for the item that says ''Install VMware Tools''. This will download lump of code and, for CentOS, mount it on '/media/VMware Tools' (yes, there's a space in the directory name). Copy the *.tar.gz file to some place like your home directory, unpack it and read the INSTALL file. For an initial installation, the typical process is to run ''vmware-install.pl''.<br /> * Under the configuration/options menu (the little wrench thing in Fusion), choose ''Sharing''. Make a folder with an obvious name (e.g., ''vmware'') and turn sharing on. Now, anything you put in there when on either the OS will be available to the other OS. On linux, this directory is located at ''/mnt/hgfs/&lt;&lt;name&gt;&gt;''.<br /> <br /> == Support for DEB packages ==<br /> It is possible to use ''fpm'' to build debian packages from RPM packages. See [https://ask.fedoraproject.org/en/question/52141/how-to-convert-rpm-package-to-deb/ how-to-convert-rpm-package-to-deb] for an explanation. In summary, make a few directories to hold parts of the RPM, load files from the RPM into them and then use ''fpm'' to build the debian package. In addition to these steps, extract the package description text to a file and pass that into ''fpm'' and look at the dependencies for the RPMs, find the corresponding Ubuntu dependencies and provide them to ''fpm''.<br /> <br /> == problems not addressed ==<br /> * There is no JUnit for the OLFS build<br /> * There is no graphviz so the ''doc'' targets for libdap and bes will fail</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13313 2022-01-20T16:45:40Z

<p>Kneumiller: /* Setup CentOS-8 */</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside the hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13312 2022-01-20T16:44:47Z

<p>Kneumiller: </p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run '''make gdal''' and '''make hdf4''' inside of hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13311 2022-01-20T16:42:46Z

<p>Kneumiller: </p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> <br /> Once you run through the rest of the hyrax build make sure to run make gdal and make hdf4 inside of hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13310 2022-01-20T16:41:55Z

<p>Kneumiller: </p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> :Once you run through the rest of the hyrax build make sure to run make gdal and make hdf4 inside of hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13309 2022-01-20T16:40:10Z

<p>Kneumiller: </p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> ;Tell the machine where to find the tirpc libraries<br /> :&lt;tt&gt;'''export CPPFLAGS=-I/usr/include/tirpc LDADD=-ltirpc'''&lt;/tt&gt;<br /> <br /> ;Once you run through the rest of the hyrax build make sure to run make gdal and make hdf4 inside of hyrax-dependencies<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13308 2022-01-20T16:31:19Z

<p>Kneumiller: </p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;You will need to enable power-tools for this setup<br /> :&lt;tt&gt;'''yum config-manager --set-enabled powertools'''&lt;/tt&gt;<br /> <br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=Hyrax_GitHub_Source_Build&diff=13307 2022-01-20T16:29:17Z

<p>Kneumiller: Adding CentOS 8 build steps</p> <hr /> <div>This describes how to get and build Hyrax from our GitHub repositories. Hyrax is a data server that implements the DAP2 and DAP4 protocols, works with a number of different data formats and supports a wide variety of customization options from tailoring the look of the server's web pages to complex server-side processing operations. This page describes how to build the server's source code. If you're working on a Linux or OS/X computer, the process is similar so we describe only the linux case; we do not support building the server on Windows operating systems.<br /> <br /> To build and install the server, you need to perform three steps:<br /> # Set up the computer to build source code (Install a Java compiler; install a C/C++ compiler; add some other tools)<br /> # Build the C++ DAP library (''libdap4'') and the Hyrax BES daemon<br /> # Build the Hyrax OLFS web application<br /> <br /> Quick links if you already know the process:<br /> * [https://github.com/opendap/hyrax new all-in-one repo that uses shell scripts]<br /> * [https://github.com/opendap/libdap libdap git repo]<br /> * [https://github.com/opendap/bes BES git repo]<br /> * [https://github.com/opendap/olfs OLFS git repo]<br /> * [https://github.com/opendap/hyrax-dependencies Hyrax dependencies]<br /> <br /> = Set up a CentOS machine to build code =<br /> == Setup CentOS-7 ==<br /> Note that I don't like clicking around to different pages to follow simple directions, so what follows is a short version of the CentOS 6 configuration information we've compiled for people that help us by building RPM packages for Hyrax. You can use this to extrapolate how to configure Ubuntu and OSX (We routinely build on those platforms as well). The complete instructions are in [[ConfigureCentos | Configure CentOS]] and describe how to to set up a CentOS 6 machine to build software. What follows is the condensed version:<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Load a basic software development environment:<br /> :&lt;tt&gt;'''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit'''&lt;/tt&gt; (it's likely that you can use more recent versions of Java)<br /> :&lt;tt&gt;'''yum install git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> ; The whole thing, with java-1.8.0<br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc'''&lt;/tt&gt;<br /> <br /> <br /> ;If you're going to build RPMs<br /> :&lt;tt&gt;'''yum install rpm-devel rpm-build redhat-rpm-config'''&lt;/tt&gt;<br /> <br /> ;Optional<br /> :Download, unpack, build and install the GNU autotools (''but '''don't''' do this unless the versions installed using yum don't work'')<br /> * autoconf &lt;tt&gt;'''[http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz autoconf-2.69.tar.gz]'''&lt;/tt&gt;<br /> * automake &lt;tt&gt;'''[http://ftp.gnu.org/gnu/automake/automake-1.14.1.tar.gz automake-1.14.1.tar.gz]'''&lt;/tt&gt; <br /> * libtool &lt;tt&gt;'''[http://ftp.gnu.org/gnu/libtool/libtool-2.4.2.tar.gz libtool-2.4.2.tar.gz]'''&lt;/tt&gt;<br /> :build them (&lt;tt&gt;'''''./configure; make; sudo make install '''''&lt;/tt&gt; - this should take no more than three minutes).<br /> <br /> == Setup CentOS-8 ==<br /> The CentOS-8 setup is very similar to CentOS-7, but there are some minor differences.<br /> <br /> ;Update the VM<br /> :&lt;tt&gt;'''yum -y update'''&lt;/tt&gt;<br /> <br /> ;Use java-1.8.0 instead of java-1.7.0 if possible.<br /> ;For CentOS 8 you will need the additional packages of openjpeg2-devel, jasper-devel, and libtirpc-devel<br /> <br /> :&lt;tt&gt;'''yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison cmake autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel cppunit cppunit-devel vim bc openjpeg2-devel jasper-devel libtirpc-devel'''&lt;/tt&gt;<br /> <br /> = A semi-automatic build =<br /> <br /> Use git to clone the https://github.com/opendap/hyrax project and follow the short instructions in the README file.<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Summarized here, those instructions are:<br /> ;use bash: The shell scripts in this repo assume you are using bash.<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development: ''source spath.sh''<br /> ;clone the three code repos for the server plus the hyrax dependencies: ''./hyrax_clone.sh -v''<br /> ;build the code, including the dependencies: ''./hyrax_build.sh -v''<br /> ;test the server: Start the BES using ''besctl start''<br /> :Start the OLFS using''./build/apache-tomcat-7.0.57/bin/startup.sh''<br /> :Test the server by loooking at ''&lt;nowiki&gt;http://localhost:8080/opendap&lt;/nowiki&gt;'' in a browser. You should see a directory named ''data'' and following that link should lead to more data. The server will be accessible to clients other than a web browser.<br /> :To test the BES function independently of the front end, use ''bescmdln'' and give it the ''show version;'' command, you should see output about different components and their versions. <br /> :Use ''exit'' to leave the command line test client.<br /> <br /> As described in the README file that is part of the ''hyrax'' repo, there are some other scripts in the repo and some options to the ''clone'' and ''build'' script that you can investigate by using -h (help).<br /> <br /> = The manual build = <br /> <br /> In the following, we describe only the build process for CentOS; the one for OS/X is similar and we note the differences where they are significant.<br /> <br /> == Get Hyrax from GitHub ==<br /> Use git to clone the https://github.com/opendap/hyrax project and follow the instructions on this page (which differ a bit from ones in the project's README)<br /> :'''&lt;tt&gt;git clone https://github.com/opendap/hyrax'''&lt;/tt&gt;<br /> <br /> Once you have the ''hyrax'' project cloned:<br /> ;set up some environment variables so the server will build an install locally, something that streamlines development<br /> :&lt;tt&gt;'''source spath.sh'''&lt;/tt&gt;<br /> ;clone the three code repos for the server plus the hyrax dependencies<br /> :&lt;tt&gt;'''./hyrax_clone.sh -v'''&lt;/tt&gt;<br /> ;proceed with the rest of the build as described in the following sections of this page<br /> <br /> == Important Note ==<br /> &lt;font color=&quot;red&quot;&gt;Many of the problems people have with the build stem from not setting the shell correctly for the build.&lt;/font&gt;<br /> In the above section, ''make sure'' you run '''source spath.sh''' before you run any of the building/compiling/testing commands that use the source code or build files. While the ''$prefix'' and ''$PATH'' environment variables are simple to set up, they are needed by most users. When you exit a terminal window and then open a new one, make sure to (re)source the ''spath.sh'' file in the new shell. You don't have to source spath.sh every time you enter the ''hyrax'' directory, but you must run it for every new instance of the shell.<br /> <br /> == Compile the Hyrax dependencies ==<br /> Use git to clone the hyrax-dependencies:<br /> git clone https://github.com/opendap/hyrax-dependencies<br /> And then build it. Unlike many source packages, there is no need to run a configure script, just ''make'' will do. However, the Makefile in this package expects ''$prefix'' to be set as described above. It will put all of the Hyrax server dependencies in a subdirectory called ''deps''. To build the dependencies for building RPMs, use ''make -j9 for-static-rpm''.<br /> ;(make sure you're in the directory set to ''$prefix'')<br /> &lt;tt&gt;<br /> ;git clone https://github.com/opendap/hyrax-dependencies<br /> ; cd hyrax-dependencies<br /> ; make --jobs=9<br /> : ''The --jobs=N runs a parallel build with at most N simultaneous compile operations. This will result in a huge performance improvement on multi-core machines. '''-jN''' is the short form for the option.''<br /> ;cd ..: ''Go back up to '''$prefix''' ''<br /> &lt;/tt&gt;<br /> <br /> '''Note:''' You can get some of the ''dependencies'' for Hyrax like ''netCDF'' from the EPEL repository, but the versions are often older than Hyrax needs. Contact us if you want information about using EPEL. At the risk of throwing people a curve ball, here's a synopsis of the process. Don't do this unless you know EPEL well. Use [http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm epel-release-6-8.noarch.rpm] and install it using ''sudo yum install epel-release-6-8.noarch.rpm''. Then install packages needed to read various file formats: ''yum install netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''<br /> <br /> == Build ''libdap'' and the ''BES'' daemon ==<br /> <br /> ==== Get and build libdap4 ====<br /> ;WARNING: If you have ''libdap'' already, uninstall it before proceeding.<br /> Build, test and install libdap4 into $prefix:<br /> ;git clone https://github.com/opendap/libdap4<br /> ;cd libdap4<br /> ;autoreconf -fiv<br /> ;./configure --prefix=$prefix --enable-developer <br /> ;make -j9<br /> ;make check -j9<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Get and build the BES and all of the modules shipped with Hyrax ====<br /> Build, test and install the BES and its modules<br /> ;git clone https://github.com/opendap/bes: Clone the BES from GitHub<br /> ;cd bes<br /> ;git submodule update --init: This will clone some additional modules into the directory ''modules''; you need to do this! (Previously it was an optional step). See [http://git-scm.com/docs/git-submodule git submodule] for information about all you can do with git's submodule command. Also note that this does not checkout a particular branch for the submodules; the modules are left in the 'detached head' state. To checkout a particular branch like 'master', which is important if you'll be making changes to that code, use ''git submodule foreach 'git checkout master' ''. <br /> ;autoreconf --force --install --verbose: You can use -fiv instead of the long options. These mean, when starting from a freshly cloned repo, run all of the autotools commands and install all of the needed scripts.<br /> ;./configure --prefix=$prefix --with-dependencies=$prefix/deps --enable-developer<br /> : Notes:<br /> :* The --with-deps... is not needed if you load the dependencies from RPMs or otherwise have them installed an generally accessible on the build machine.<br /> :* The --enable-developer option will compile in all of the debugging code which may affect performance even if the debugging output is not enabled.<br /> ;make -j9<br /> ;make check -j9: Some tests may fail and adding ''-k'' ignores that and keeps make marching along. ''Note that you must run '''make''' before '''make check''' in the bes code''.<br /> ;make install<br /> ;cd ..: Go back up to ''$prefix''<br /> <br /> ==== Test the BES ====<br /> Start the BES and verify that all of the modules build correctly.<br /> ;besctl start: Start the BES. Given that ''$prefix/bin'' is on your ''$PATH'', this should start the BES. You will not need to be root if you used the ''--enable-developer'' switch with configure (as shown above), otherwise you should run ''sudo besctl start'' with the caveat that as root ''$prefix/bin'' will probably not be n your ''$PATH''.<br /> :If there's an error (e.g., you tried to start as a regular user but need to be root), edit bes.conf to be a real user (yourself?) in a real group (use 'groups' to see which groups you are in) and also check that the bes.log file is ''not'' owned by root. <br /> :Restart.<br /> ;bescmdln: Now that the BES is running, start the BES testing tool<br /> ;show version;: Send the BES the version command to see if it's running<br /> :Take a quick look at the output. There should be entries for libdap, bes and all of the modules.<br /> ;exit: Exit the testing tool<br /> <br /> Note that even though you have exited the ''bescmdln'' test tool, the BES is still running. That's fine - we'll use it in just a bit - but if you want to shut it down, use ''besctl stop'', or ''besctl pids'' to see the daemon's processes. If the BES is not stopping, ''besctl kill'' will stop all BES processes without waiting for them to complete their current task.<br /> <br /> == Build the Hyrax ''OLFS'' web application ==<br /> The OLFS is a java servlet built using ant. The OLFS is a java servlet web application and runs with Tomcat, Glassfish, etc. You need a copy of Tomcat, but our servlet does not work with the RPM version of Tomcat. Get [http://tomcat.apache.org/download-70.cgi Tomcat 7 from Apache]. Note that if you built the dependencies from source using the ''hyrac-dependencies-1.10.tar'' then there is a copy of Tomcat in the ''hyrax-dependecies/extra_downloads directory. You can unpack the Tomcat tar file in ''$prefix''. I'll assume you have the Apache Tomcat tar file in ''$prefix''.<br /> <br /> ;tar -xzf apache-tomcat-7.0.57.tar.gz: Expand the Tomcat tar ball<br /> ;git clone https://github.com/opendap/olfs: Get the OLFS source code<br /> ;cd olfs: change directory to the OLFS source<br /> ;ant server: Build it<br /> ;cp build/dist/opendap.war ../apache-tomcat-7.0.57/webapps/: Copy the opendap web archive to the tomcat webapps direcotry.<br /> ;cd ..: Go up to ''$prefix''<br /> ;./apache-tomcat-7.0.57/bin/startup.sh: Start Tomcat<br /> <br /> == Test the server ==<br /> You can test the server several ways, but the most fun is to use a web browser. The URL ''http://&lt;machine&gt;:8080/opendap'' should return a page pointing to a collection of test datasets bundled with the server. You can also use ''curl'', ''wget'' or any application that can read from OpenDAP servers (e.g., Matlab, Octave, ArcGIS, IDL, ...).<br /> <br /> == Stopping the server ==<br /> Stop both the BES and Apache<br /> <br /> ;besctl stop<br /> ;./apache-tomcat-7.0.57/bin/shutdown.sh<br /> <br /> Note that there is also a ''hyraxctl'' script that provides a way to start and stop Hyrax without you (or ''init.d'') having to type separate commands for both the BES and OLFS. This script is part of the BES software you cloned from git.<br /> <br /> == Building select parts of the BES ==<br /> Building just the BES and one of more of its handlers/modules is not at all hard to do with a checkout of code from git. In the above section on building the BES, simply skip the step where the submodules are cloned (''git submodule update --init'') and link configure.ac to ''configure_standard.ac''. The rest of the process is as shown. The end result is a BES daemon without any of the standard Hyrax modules (but support for DAP will be built if ''libdap'' is found by the configure script).<br /> <br /> To build modules for the BES, simply go to ''$prefix'', clone their git repo and build them, taking care to pass set ''$prefix'' when calling the module's ''configure'' script. <br /> <br /> Note that it is easy to combine the 'build it all' and 'build just one' processes so that a complete Hyrax BES can be built in one go and then a new module/handler not included in the BES git repo can be built and used. Each module we have on GitHub has a ''configure.ac'', ''Makefile.am'', etc., that will support both kinds of builds and [[Configuration of BES Modules]] explains how to take a module/handler that builds as a standalone module and tweak the build scripts so that it's fully integrated into the Hyrax BES build, too.<br /> <br /> = Building on Ubuntu =<br /> This was tested using Xenial (Ubuntu 16)<br /> <br /> ''sudo apt-get update''<br /> <br /> Packages needed:<br /> <br /> ''sudo apt-get install ...''<br /> <br /> '''ant junit git flex bison autoconf automake libtool emacs openssl bzip2 libjpeg-dev libxml2-dev curl libicu-dev vim bc make cmake uuid-dev libcurl4-openssl-dev libicu-dev g++ zlib1g-dev libcppunit-dev libssl-dev'''</div>

Kneumiller https://docs.opendap.org/index.php?title=ConfigureCentos&diff=13303 2022-01-20T00:17:10Z

<p>Kneumiller: Added CentOS 8 instructions</p> <hr /> <div>[[HowTo_guides|&lt;&lt; back to HowTo Guides]]<br /> <br /> This describes how to set up a vanilla CentOS computer so that we can build RPM binaries for the Hyrax data server. This was taken from notes describing how I configured a machine with a fresh install of CentOS 5.3 to build Hyrax. Augmented for CentOS 6. <br /> <br /> If you're installing this on a ''virtual'' host using VMware, see the section at the end of this document for some tips on making that VM run smoothly.<br /> <br /> == CentOS versions ==<br /> <br /> These instructions have been amended to cover:<br /> <br /> * CentOS-6.9 (64-bit)<br /> * CentOS-7.? (64-bit)<br /> * CentOS-8 (64-bit)<br /> <br /> == Initial configuration ==<br /> <br /> Set up general build stuff that you need. The RPM packages you need are listed here. Using ''yum'' (''sudo yum install &lt;package&gt;'') is the easiest way to install these packages on CentOS/RedHat. The package names for Ubuntu are a bit different, but are easy to figure out. Our CI builds are done on Ubuntu 14 (as of Nov 2017), so it's certainly possible to build there:<br /> * java-1.7.0-openjdk<br /> * java-1.7.0-openjdk-devel<br /> * ant<br /> * ant-junit<br /> * junit<br /> * git <br /> * gcc-c++ <br /> * flex <br /> * bison <br /> * autoconf <br /> * automake <br /> * libtool <br /> * emacs <br /> * openssl-devel <br /> * libuuid-devel <br /> * readline-devel <br /> * zlib-devel <br /> * bzip2, bzip2-devel <br /> * libjpeg-devel <br /> * libxml2-devel <br /> * curl-devel <br /> * libicu-devel<br /> * vim<br /> * bc<br /> <br /> If you're feeling lucky, you can put all of the packages on a single line<br /> :''yum install java-1.7.0-openjdk java-1.7.0-openjdk-devel ant ant-junit junit git gcc-c++ flex bison autoconf automake libtool emacs openssl-devel libuuid-devel readline-devel zlib-devel bzip2 bzip2-devel libjpeg-devel libxml2-devel curl-devel libicu-devel vim bc''<br /> <br /> '''Notes:''' <br /> *''Java 1.7, Ant and CentOS 6''. I found that ant 1.7 (the default with CentOS 6) would not work with Java 1.8. To get our code to build with Java 1.7, I used ''alternatives --config java'' to configure 1.7 and then ''export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75.x86_64''. I could not compile the olfs otherwise. <br /> *''7 June 2016 - CentOS 7 RPM Build''. I had to add '''bzip2-devel''', '''libicu-devel''', and '''bc''' to the set of packages to get the RPM to build on CentOS 7. It may be the case this was true of CentOS 6 as well and it just slipped by.<br /> *''11 Nov 2017'' - The latest versions of CentOS 6 and 7 had no problems using the packages above. I did not test Java 8 or Java 9, but Java 8 has no problems in development configurations. Tomcat7, 8, and 9 from RPMs all work as well, at least on my development machines.<br /> <br /> ==== Autotools ====<br /> The autotools with CentOS 6 should be fine - the scripts will complain if otherwise. However, the newer autoconf/make support some nice features, so installing these newer version is not a waste. On a Test machine, this should not be done. If you don't install the new versions, make sure to use yum to install autoconf, automake and libtool. That is<br /> :''yum install autoconf automake libtool''<br /> If you don't want to use RPM for these, then:<br /> * download the latest versions of autoconf, automake, and libtool from GNU: http://ftp.gnu.org/gnu/<br /> * ./configure, make, make install<br /> <br /> ==== RPM Construction Tools ====<br /> * ''yum install rpm-devel rpm-build redhat-rpm-config''<br /> <br /> == Hyrax specific configuration ==<br /> Set up things that are specific to Hyrax<br /> <br /> It's best to use as much of the RPM 'ecosystem' as possible because this is what will be on other people's computers; using newer stuff, like the lastest HDF5 library is only going to cause incompatibilities. But to do that for Hyrax, you need to get beyond the stock set of repositories that ''yum'' searches. <br /> <br /> ==== Add the EPEL EL repo to yum ====<br /> * Information about EPEL/EL is here: https://fedoraproject.org/wiki/EPEL.<br /> * For CentOS 6, you likely want the rpm package here: http://mirror.pnl.gov/epel/6/i386/repoview/epel-release.html<br /> <br /> &lt;blockquote&gt;'''Note''' Use CentOS 6 if possible for Hyrax at this time (Sept 2014). We'll be moving to combined v.6 and v.7 support soon; v.5 is no longer supported. Some CentOS 5 information is included here because this was initially written back when it was the latest version.&lt;/blockquote&gt;<br /> <br /> ===== Centos 5.x =====<br /> But don't use this unless you really have too...<br /> <br /> * google EPEL/EL5 and <br /> * find &amp; download ''epel-release-5.4.noarch.rpm''<br /> * install it: ''rpm -i epel-release-5.4.noarch.rpm''<br /> <br /> ===== Centos 6.x =====<br /> Use this until we start supporting CentOS 7<br /> <br /> * google EPEL/EL6 and <br /> * find &amp; download ''epel-release-6.5.noarch.rpm''<br /> * install it: ''rpm -i epel-release-6.5.noarch.rpm''<br /> <br /> ===== Centos 8.x =====<br /> * yum update<br /> * yum config-manager --set-enabled powertools<br /> * yum install openjpeg2-devel<br /> * yum install jasper-devel<br /> * yum install libtirpc-devel<br /> * export CPPFLAGS=-I/usr/include/tirpc<br /> * export LDADD=-ltirpc<br /> <br /> <br /> ==== Data type dependencies: NetCDF, HDF4, HDF5 ====<br /> * ''yum install netcdf-devel'' (note: On CentOS-6.2 you'll get netcdf-4 and on CentOS-5.3 you'll get netcdf-3)<br /> * ''yum install hdf-devel'' (note: 'hdf', not 'hdf4')<br /> * ''yum install hdf5-devel''<br /> <br /> ==== Other dependencies ====<br /> * ''yum install libicu-devel''<br /> * ''yum install cfitsio-devel''<br /> * ''yum install cppunit-devel''<br /> <br /> ==== ... If you're feeling lucky ====<br /> As with the 'build packages,' here are all of the 'hyrax-specific' packages so you can use just one Yum command<br /> :''netcdf-devel hdf-devel hdf5-devel libicu-devel cfitsio-devel cppunit-devel rpm-devel rpm-build''.<br /> Note that you'll still need to set up the EPEL repository.<br /> <br /> == VMware configuration tips ==<br /> It's best if you enable a shared directory so that you can pass stuff back and forth between the host OS and the Guest OS (i.e., the Virtual machine). That will only work if you have the ''VMware tools'' installed in/on the guest.<br /> <br /> * Install VMware-tools. This process varies, but it's pretty easy for all platforms. On VMware Fusion, look under the ''Virtual Machine'' menu for the item that says ''Install VMware Tools''. This will download lump of code and, for CentOS, mount it on '/media/VMware Tools' (yes, there's a space in the directory name). Copy the *.tar.gz file to some place like your home directory, unpack it and read the INSTALL file. For an initial installation, the typical process is to run ''vmware-install.pl''.<br /> * Under the configuration/options menu (the little wrench thing in Fusion), choose ''Sharing''. Make a folder with an obvious name (e.g., ''vmware'') and turn sharing on. Now, anything you put in there when on either the OS will be available to the other OS. On linux, this directory is located at ''/mnt/hgfs/&lt;&lt;name&gt;&gt;''.<br /> <br /> == Support for DEB packages ==<br /> It is possible to use ''fpm'' to build debian packages from RPM packages. See [https://ask.fedoraproject.org/en/question/52141/how-to-convert-rpm-package-to-deb/ how-to-convert-rpm-package-to-deb] for an explanation. In summary, make a few directories to hold parts of the RPM, load files from the RPM into them and then use ''fpm'' to build the debian package. In addition to these steps, extract the package description text to a file and pass that into ''fpm'' and look at the dependencies for the RPMs, find the corresponding Ubuntu dependencies and provide them to ''fpm''.<br /> <br /> == problems not addressed ==<br /> * There is no JUnit for the OLFS build<br /> * There is no graphviz so the ''doc'' targets for libdap and bes will fail</div>

Kneumiller