ConfigureAmazonLinuxAMI: Difference between revisions
Line 241: | Line 241: | ||
<font size="2"><code> | <font size="2"><code> | ||
: [root@ip-10-152-156-244 shrew]# export LD_LIBRARY_PATH=/home/ec2-user/opendap/shrew/deps/icu-3.6/lib/ | : [root@ip-10-152-156-244 shrew]# export LD_LIBRARY_PATH=/home/ec2-user/opendap/shrew/deps/icu-3.6/lib/ | ||
: [root@ip-10-152-156-244 shrew]# besctl | : [root@ip-10-152-156-244 shrew]# besctl start | ||
: Starting the BES | |||
: OK: Successfully started the BES | : OK: Successfully started the BES | ||
: PID: 11975 UID: 0 | : PID: 11975 UID: 0 |
Revision as of 00:25, 19 September 2014
How to configure an Amazon Linux AMI for EC2 to build Hyrax
This describes how to set up a vanilla Amazon Linux AMI virtual machine so that we can build the Hyrax data server.
Amazon Linux AMI versions
These instructions have been amended to cover:
- Amazon Linux AMI 2012.09
Initial configuration
Set up general build stuff that you need:
- yum install java-1.6.0-openjdk java-1.6.0-openjdk-devel
- yum install junit
- yum install ant
- yum install ant-junit.noarch
- yum install make
- yum install subversion
- yum install gcc-c++
- yum install flex
- yum install bison
- yum install curl-devel
- yum install libxml2-devel
- yum install libjpeg-devel
- yum install zlib-devel
- yum install readline-devel
- yum install libuuid-devel
- yum install openssl-devel
Other dependencies
- yum install libicu-devel
Autotools
- download the latest versions of autoconf, automake and libtool
- ./configure, make, make install
Building Hyrax from the shrew project
In a (ba)sh shell:
- Check out the shrew project
svn co https://scm.opendap.org/sv/trunk/shrew
- Change your working directory to the shrew directory.
cd shrew
- Source the spath file
. spath
- Build the dependancies
cd src/dependencies
make
Using the shrew top level Makefile
- Change your working directory to the shrew directory.
cd $prefix
- Copy the Makefile.am.master to Makefile.am
cp Makefile.am.master Makefile.am
- Edit the Makefile.am to be certain that the various dependancies are correctly configured.
vim Makefile.am
- (Since I am trying to get a new package up I edited this file, mostly I think you should just be able to make the copy and proceed.)
- Engage the autotools process and see how far you get...
autoreconf -vif
./configure --prefix=$prefix
make
If the above fails then you need to build each Hyrax component by hand.'
Building by hand
- Build and install libdap
cd $prefix/src/libdap
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the BES
cd $prefix/src/bes
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the dap-server module
cd $prefix/src/modules/dap-server
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the csv_handler module
cd $prefix/src/modules/csv_handler
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the fileout_gdal module
cd $prefix/src/modules/fileout_gdal
autoreconf -vif
./configure --prefix=$prefix --with-gdal=$prefix/deps/gdal-12.15.12
make install
- Build and install the fileout_netcdf module
cd $prefix/src/modules/fileout_netcdf
autoreconf -vif
./configure --prefix=$prefix --with-netcdf=$prefix/deps/netcdf-4.1.2
make install
- Build and install the fits_handler module
cd $prefix/src/modules/fits_handler
autoreconf -vif
./configure --prefix=$prefix --with-cfits=$prefix/deps/cfitsio
make install
- Build and install the freeform_handler module
cd $prefix/src/modules/freeform_handler
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the gateway_module module
cd $prefix/src/modules/gateway_module
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the gdal_handler module
cd $prefix/src/modules/fileout_netcdf
autoreconf -vif
./configure --prefix=$prefix --with-gdal=$prefix/deps/gdal-12.15.12
make install
- Build and install the hdf4_handler module
cd $prefix/src/modules/hdf4_handler
autoreconf -vif
./configure --prefix=$prefix --with-hdf4=$prefix/deps/hdf-4.2.8
make install
- Build and install the hdf5_handler module
cd $prefix/src/modules/hdf5_handler
autoreconf -vif
./configure --prefix=$prefix --with-hdf5=$prefix/deps/hdf5-1.8.6
make install
- Build and install the ncml_module module
cd $prefix/src/modules/ncml_module
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the netcdf_handler module
cd $prefix/src/modules/fileout_netcdf
autoreconf -vif
./configure --prefix=$prefix --with-netcdf=$prefix/deps/netcdf-4.1.2
make install
- Build and install the xml_data_handler module
cd $prefix/src/modules/xml_data_handler
autoreconf -vif
./configure --prefix=$prefix
make install
- Build and install the ugrid_functions module
cd $prefix/src/modules/ugrid_functions
autoreconf -vif
./configure --prefix=$prefix --with-gridfields=$prefix/deps/gridfields-1.0.1
make install
Building the ncml_handler when incompatible versions of ICU are resident on the system
Is there a system ICU?
Check to see what's on the machine in terms of ICU:
- [root@ip-10-152-156-244 shrew]# rpm -qa | grep icu
- libicu-4.2.1-9.9.amzn1.x86_64
- libicu-devel-4.2.1-9.9.amzn1.x86_64
- [root@ip-10-152-156-244 shrew]#
(no need to be root)
So we can see that icu is there but it's the version 4 code and that is massively incompatible with the version 3 code. You probably knew all of that, I think we talked about it, but I just thought I'd show the useful 'rpm -qa' (query all) command.
Where for art thou, ICU files?
Here's how to see where stuff is:
- [ec2-user@ip-10-152-156-244 shrew]$ rpm -qil libicu-devel-4.2.1-9.9.amzn1.x86_64Name : libicu-devel Relocations: (not relocatable)
- Version : 4.2.1 Vendor: Amazon.com
- Release : 9.9.amzn1 Build Date: Mon 09 Jan 2012 04:52:37 PM UTC
- Install Date: Wed 30 Jan 2013 05:39:05 AM UTC Build Host: build-31003.build
- Group : Development/Libraries Source RPM: icu-4.2.1-9.9.amzn1.src.rpm
- .
- .
- .
- Description :
- Includes and definitions for developing with icu.
- /usr/bin/icu-config
- /usr/include/layout
- /usr/include/layout/LEFontInstance.h
- /usr/include/layout/LEGlyphFilter.h
- /usr/include/layout/LEGlyphStorage.h
- /usr/include/layout/LEInsertionList.h
- /usr/include/layout/LELanguages.h
- /usr/include/layout/LEScripts.h
- /usr/include/layout/LESwaps.h
- /usr/include/layout/LETypes.h
- /usr/include/layout/LayoutEngine.h
- /usr/include/layout/ParagraphLayout.h
- /usr/include/layout/RunArrays.h
- /usr/include/layout/loengine.h
- /usr/include/layout/playout.h
- /usr/include/layout/plruns.h
- /usr/include/unicode
- /usr/include/unicode/basictz.h
- /usr/include/unicode/bms.h
- /usr/include/unicode/bmsearch.h
- .
- .
- .
So if configure looks for icu-config, it'll find the one for icu 4.x which our handler cannot use - at least not without some tweaking things. I think the includes are all different (nice, eh?)
Make the build find the ICU in our dependencies
Put our icu-3.6/bin dir on the PATH in front of everything:
- [ec2-user@ip-10-152-156-244 shrew]$ export PATH=/home/ec2-user/opendap/shrew/deps/icu-3.6/bin:$PATH
And configure in the ncml_handler should work just fine
But when you try to start the server, massive library not found crap. That's because on linux you have to run ldconfig for a library to be found 'automatically'. As a convenience to users, icu left this step out…
Make the dynamic library load find our depencencies ICU at runtime
Add our icu-3.6/lib dir to LD_LIBRARY_PATH
- [root@ip-10-152-156-244 shrew]# export LD_LIBRARY_PATH=/home/ec2-user/opendap/shrew/deps/icu-3.6/lib/
- [root@ip-10-152-156-244 shrew]# besctl start
- Starting the BES
- OK: Successfully started the BES
- PID: 11975 UID: 0
- [root@ip-10-152-156-244 shrew]#
Start tomcat and the server is up!
http://ec2-54-242-224-73.compute-1.amazonaws.com:8080/opendap/data/ncml/fnoc1_improved.ncml
Why we don't have this these issues on CentOS is likely that they use icu-3.6 and even if we build our own version, it's not being used. In fact, the reason we used 3.6 and not 4.2 was that most linux distros came with 3.6.