Difference between revisions of "Developer Info"
m (→OPeNDAP Workshops)
m (→AWS Tips)
|Line 50:||Line 50:|
==== AWS Tips ====
==== AWS Tips ====
===== Growing a CentOS Root Partition on an AWS EC2 Instance =====
===== Growing a CentOS Root Partition on an AWS EC2 Instance =====
Revision as of 21:38, 10 April 2017
- OPeNDAP's GitHub repositories: OPeNDAP's software is available using GitHub in addition to the doenloads from our website.
- Continuous Integration builds: Software that is built whenever new changes are pushed to the master branch. These builds are done on the Travis-CI system.
- test.opendap.org: Test servers with data files.
- We use the Coverity static system to look for common software defects, information on Hyrax is spread across three projects:
- 1 Developer Guidelines and Information
- 2 General development information
- 3 OPeNDAP Workshops
- 4 OPeNDAP Development process information
- 5 TRANSFERED FROM HYRAX INSTALL PAGE
- 6 Old information
1 Developer Guidelines and Information
The OPeNDAP FAQ has a pretty good section on developer's questions.
2 General development information
These pages contain general information relevant to anyone working with our software:
- Git Hacks and Tricks: Information about using git and/or GitHub that seems useful and maybe not all that obvious.
- Valgrind - How to generate suppression files for valgrind This will quiet valgrind, keeping it from telling you OS/X or Linux (or the BES) is leaking memory.
- Migrating source code from SVN to git: How to move a large project from SVN to git and keep the history, commits, branches and tags.
- Eclipse - Detailed information about running Eclipse on OSX from the Mozzilla project. Updated in 2017, this is really good but be aware that it's specific to Mozilla so some of the tips don't apply. Hyrax (i.e., libdap4 and BES) also use their own build system (autotools + make) so most of the configuration information here is very apropos. See also How to use Eclipse with Hyrax Source Code below.
- RPM Guide The best one I'm found so far...
3 OPeNDAP Workshops
- The APAC/BOM Workshops: This workshop spanned several days and covered a number of topics, including information for SAs and Developers. Oct 2007.
- ESIP Federation Server Workshop: This half-day workshop focused on server installation and configuration. Summer 2008
- Server Functions: This one-day workshop is all about writing and debugging server-side functions. It also contains a wealth of information about Hyrax, the BES and debugging tricks for the server. Spring 2012. Updated Fall 2014 for presentation to Ocean Networks Canada.
4 OPeNDAP Development process information
These pages contain information about how we'd like people working with us to use our various on-line tools.
- Hyrax GitHub Source Build This explains how to clone our software from GitHub and build our code using a shell like bash. It also explains how to build the BES and all of the Hyrax 'standard' handlers in one operation, as well as how to build just the parts you need without cloning the whole set of repos. Some experience with 'git submodule' will make this easier, although the page explains everything.
- How to Plan a Release Sprint A general template for planning a release in Jira or the equivalent.
- Making a Release. Once software is ready for distribution, use this checklist to make sure you do all of the steps needed to make a release.
- Making RPM Distributions. Follow these steps to create an RPM distribution of the software.
- Making a Release of OLFS. Follow these steps to create the three .jar files needed for the OLFS release. Includes information on how to build the OLFS and how to run the tests.
- Bug Prioritization. How we prioritize bugs in our software.
4.1 Software process issues:
- How to configure a CentOS machine for production of RPM binaries - Updated 12/2014 to include information regarding git.
- How to use Eclipse with Hyrax Source Code Assuming you have cloned our Hyrax code from GitHub, this explains how to setup eclipse so you can work fairly easily and switch back and forth between the shell, emacs and eclipse.
- How to add timing instrumentation to your BES code.
- How to write unit tests using CppUnit
- How to use valgrind with unit tests
- How to copyright software written for OPeNDAP
- Managing public and private keys using gpg
- How to Setup Secure Email and Sign Software Distributions
- How to Handle Email-list Support Questions
- Security Policy and Related Procedures
- Software version numbers
- Development Guidelines
4.2 AWS Tips
4.2.1 Growing a CentOS Root Partition on an AWS EC2 Instance
- Install the EPEL6 repo
- Install the cloud-utils-growpart package:
[bash]# yum install cloud-utils-growpart
- Determine the root partition device using the lsblk command:
[bash]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 150G 0 disk └─xvda1 202:1 0 8G 0 part /
- Grow the root partition:
[bash]# growpart /dev/xdva 1
- At this point lsblk will not show a change, but if you use parted you'll see that the deed has indeed been done:
[bash]# parted GNU Parted 2.1 Using /dev/xvda Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) unit s (parted) print Model: Xen Virtual Block Device (xvd) Disk /dev/xvda: 314572800s Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 2048s 314568764s 314566717s primary ext4 boot
- The only thing remaining is to resize the filesystem. This will be done automatically by the cloud-init tools after a reboot:
[bash]# reboot now
- When the system is back up, login and use lsblk to confirm your change:
[bash]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 150G 0 disk └─xvda1 202:1 0 150G 0 part /
4.2.2 How Shutoff the CentOS firewall
If you trust the AWS firewall, then you'll probably want to shut off the now redundant Linux firewall:
- Stop the ipchains service: service ipchains stop
- Stop the iptables service: service iptables stop
- Stop the ipchains service from starting when you restart the server: chkconfig ipchains off
- Stop the iptables service from starting when you restart the server: chkconfig iptables off
5 TRANSFERED FROM HYRAX INSTALL PAGE
5.1 For Software Developers
- Hyrax GitHub Source Build We have moved the Hyrax source code to GitHub; please don't use the old, read-only SVN repo to build.
- How to use Eclipse with Hyrax Source Code Note that this is a a work in progress, but it will help with some of the odd steps that Eclipse seems to require.
5.1.1 BES Development Information
- How to debug the BES
- BES - Debugging Using besstandalone
- How to create your own BES Module
- Hyrax Module Integration: How to configure your module so it's easy to add to Hyrax instances (pdf)
- Starting and stopping the BES
- Running the BES command line client
- BES Client commands
- BES XML Commands
- Extending your BES Module
- Example BES Modules - the Hello World example and the CSV data handler
- BES communication protocol using PPT (point to point transport)
- BES Administrative Commands
5.2 Reference documentation
6 Old information
Note: The following is all about using Subversion and is out of date as of November 2014 when we switched to git. There are still good ideas here...
- How to merge code
- Using the SVN trunk, branches and tags to manage releases.
- Making a Branch of Shrew for a Server Release. Releases should be made from the trunk and moved to a branch once they are 'ready' so that development can continue on the trunk and so that we can easily go back to the software that mad up a release, fix bugs, and (re)release those fixes. In general, it's better to fix things like build issues, etc., discovered in the released software on the trunk and merge those down to the release branch to maintain consistency, re-release, etc. This also means that virtually all new feature development should take place on special feature branches, not the trunk.
- Hyrax Package for OS-X. This describes how to make a new OS/X 'metapackage' for Hyrax.
- Making Windows XP distributions. Follow these directions to make Windows XP binaries.
- Making a Matlab Ocean Toolbox Release. Follow these steps when a new Matlab GUI version is ready to be released.
- Eclipse - How to Setup Eclipse in a Shrew Checkout This includes some build instructions
- How to configure a Linux machine to build Hyrax from SVN
- How to configure a SUSE machine for production of RPM binaries
- How to configure an Amazon Linux AMI for EC2 Instance To Build Hyrax
- Notes from setting up Hyrax on our new web host
- Subversion 1.7 documentation -- The official Subversion documentation; PDF and HTML.
- OPeNDAP's Use of Trac -- How to use Trac's various features in the software development process.