OLFSReleaseGuide: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
(40 intermediate revisions by 2 users not shown)
Line 3: Line 3:
== Overview ==
== Overview ==
This document describes how to make releases of the OLFS.   
This document describes how to make releases of the OLFS.   
For this document, ''$svn'' refers to the svn root, presumably "https://scm.opendap.org/svn".


About version numbers:  
About version numbers:  
Line 26: Line 24:
Make sure that all of the developers working on the code have checked in their working copies of the trunk, and that all development branches whose functionality/features/codeBase are to included in the release have been merged to the trunk and committed.  
Make sure that all of the developers working on the code have checked in their working copies of the trunk, and that all development branches whose functionality/features/codeBase are to included in the release have been merged to the trunk and committed.  


=== Check out the OLFS code from the subversion trunk ===
=== Check out the master branch of the OLFS code from GitHub ===
In a new directory get a fresh copy of the trunk:
In a new directory get a fresh copy of the trunk:
<font size="2"><source lang="sh">svn co https://scm.opendap.org/svn/trunk/olfs</source></font>
<font size="2"><source lang="bash">git clone https://github.com/opendap/olfs</source></font><!-- svn co https://scm.opendap.org/svn/trunk/olfs -->


=== Build the OLFS ===
=== Build the OLFS ===
In the checked out OLFS directory run the command:
In the checked out OLFS directory run the command:
<font size="2"><source lang="sh">% ant server</source></font>
<font size="2"><source lang="bash">% ant server</source></font>


=== Run the make-check script located in the olfs directory ===
=== Run the tests ===
This script will run a series of tests between the OLFS and an already installed BES. The BES should be configured with the netcdf_handler, freeform_handler, and hdf4_handler modules. If it's not already running the make-check script will start it. Or you can start the BES yourself. This requires that hyrax is running on localhost:8080.
''Updated Dec 29th 2020''


<font size="2"><source lang="sh">% make-check</source></font>
The Hyrax regression tests utilize an autotools/automake/autotest test system. In order to to run the tests you must have the Hyrax server running. Build and install the BES first. Then build and install the OLFS into a Tomcat instance and start that instance.


''Note: You can use the script '''mkTestPatch''' to see the diffs of any broken tests.''
Once you have the OLFS built, installed, and running do the following:
* If you have not done so already, clone the hyrax_regression_tests project:
: <code>gitclone https://github.com/opendap/hyrax_regression_tests</code>
* Change directory into the top of <code>hyrax_regression_tests</code> and perform the following:
: <code>autoreconf -vif</code>
: <code>./configure</code>
: <code>make check</code>
The tests may also be run directly:
: <code>./testsuite </code>


== Determine OLFS Version Number ==
Either command will run the tests in a traditional autotest test harness, against the server you should have running at http://localhost:8080/opendap/
 
The <code>testsuite</code> supports several notable command line options:
 
* <code>--besdev=yes|no</code> Was the BES built using --enable-developer?
* <code>--hyraxurl=hyrax-service-endpoint-url</code> Run the various tests (DAP2/4, w10n, wcs, etc.) against the Hyrax instance located at the specified endpoint URL. (default: http://localhost:8080/opendap)
* <code>--netrc=netrc_file_name</code> Run tests using the specified netrc file (ala cURL) so that the test can authenticate to access the server. (default: ~/.netrc)
 
== Determine Version Numbers ==
You will need to know the OLFS version and the Hyrax version of the release to build the distribution and to update the various release related files.
 
=== OLFS Version Number ===
# Look in the NEWS and README files to determine the previous version number.
# Look in the NEWS and README files to determine the previous version number.
# Determine new version number by looking at the changes to the code (start by reviewing the ChangeLog file)
# Determine new version number by looking at the changes to the code (start by reviewing the ChangeLog file)
Line 51: Line 68:
#* Update the release related files  
#* Update the release related files  
#* Build the versioned software distribution bundles.
#* Build the versioned software distribution bundles.
=== Hyrax Version ===
; Need a set of criteria or a link to them here.


== Edit the release related files ==
== Edit the release related files ==


; ChangeLog
; ChangeLog
: Update the '''ChangeLog''' file using the script ''update_cl.sh'' which can be found in the ''svn-tools'' project. (e.g., ~/svn-tools/update_cl.sh ChangeLog). If you're making the first ChangeLog entries, then you'll need to create the ChangeLog file first. Here's a key tip: When you're making the commit log entries, always include the file name(s) of the affected files in your entry. Then to write the NEWS file, just read over the new ChangeLog entries and summarize. In the old days of CVS, the logs automatically included the names of the changed files, but subversion doesn't do that. A little discipline with those log entries will go a long way toward streamlining the release process!
:Use the script <tt>gitlog-to-changelog</tt> (which can be found with Google) to update the '''ChangeLog''' file by running it using the <tt>--since="<date>"</tt> option with a date one day later in time than the newest entry in the current ChangeLog. Then, to write the NEWS file, just read over the new ChangeLog entries and summarize.
: '''Tip''' - This command gets the entire gitlog and reformats it to change log, wrapping and indenting long lines to under 80 characters. Be sure to update the date to the latest date in the current ChangeLog file.:
:: '''<tt>gitlog-to-changelog --since="1970-01-01" 2>&1  | fold -s -w 72 | awk '{if(!match($0,"^[0-9]|^[\\t]")){printf("\t%s\n",$0);}else{print $0;}}' -</tt>'''
 


; README
; README.md
: Update version number.
: Update version number.
: Review content to make sure it's current.  
: Review content to make sure it's current.  
Line 67: Line 90:
; install.html
; install.html
: Update version number.
: Update version number.
: Review content to make sure it's current.  
: Review content to make sure it's current.


== Check in changes ==
== Check in changes ==


In the local copy of the OLFS trunk in which you updated the '''ChangeLog''' and other files:
In the local copy of the OLFS master branch in which you updated the '''ChangeLog''' and other files:
   
   
<font size="2"><source lang="sh">svn ci -m "olfs: Updating olfs version and release files to version 1.2.3"</source></font>
<font size="2"><source lang="bash">git commit -a -m "olfs: Updating olfs version and release files to version 1.2.3"</source></font>


== The release branch ==
Once this is done, use github to tag the code you just committed both with its version and the version of hyrax to which it belongs:
 
==== Tag the release ====
<font size="2"><source lang="bash">
git tag -m "olfs-<numbers>" -a olfs-<numbers>
git push origin olfs-<numbers>
</source></font>
 
==== If this is part of Hyrax, also tag this point in the master branch with the Hyrax release number ====
<font size="2"><source lang="bash">
git tag -m "hyrax-<numbers>" -a hyrax-<numbers>
git push origin hyrax-<numbers>
</source></font>
 
;Note: Instead of tagging all of the modules, use the saved commit hashes that git tracks for submodules. This cuts down on the bookkeeping for releases and removes one source of error; recording the wrong version for one of the 17 or so modules. thus, there is no need to tag the code in the modules.
 
<!-- == The release branch ==




Line 88: Line 127:
: Since every release of the OLFS is tied to a release of the Hyrax server we create OLFS release branches based the name of the Hyrax release with which they are associated.  
: Since every release of the OLFS is tied to a release of the Hyrax server we create OLFS release branches based the name of the Hyrax release with which they are associated.  


<font size="2"><source lang="sh">svn cp -m "olfs: Creating OLFS release branch for hyrax-4.5" $svn/trunk/olfs $svn/branch/olfs/release/hyrax-4.5</source></font>
<font size="2"><source lang="bash">svn cp -m "olfs: Creating OLFS release branch for hyrax-4.5" $svn/trunk/olfs $svn/branch/olfs/release/hyrax-4.5</source></font>


=== Otherwise (the release branch exists) so merge the trunk to the branch ===
=== Otherwise (the release branch exists) so merge the trunk to the branch ===


Checkout a copy of the release branch:
Checkout a copy of the release branch:
<font size="2"><source lang="sh">svn co $svn/branch/olfs/release/hyrax-4.5</source></font>
<font size="2"><source lang="bash">svn co $svn/branch/olfs/release/hyrax-4.5</source></font>


In this new local copy of the release branch merge from the trunk like this:
In this new local copy of the release branch merge from the trunk like this:
<font size="2"><source lang="sh">
<font size="2"><source lang="bash">
svn merge $svn/trunk/package
svn merge $svn/trunk/olfs
</source></font>
</source></font>
You need to run this command from within a working copy and the changes are made to the working copy. You need to commit them to add them to the repository.
You need to run this command from within a working copy and the changes are made to the working copy. You need to commit them to add them to the repository.
<font size="2"><source lang="sh">
<font size="2"><source lang="bash">
svn ci -m "olfs: Merged trunk to hyrax-4.5 release branch."
svn ci -m "olfs: Merged trunk to hyrax-4.5 release branch."
</source></font>
</source></font>
Line 116: Line 155:
And the new release branch is named ''hyrax-4.5'' then add a 0 as the revision number for a tag name of ''hyrax-4.5.0''
And the new release branch is named ''hyrax-4.5'' then add a 0 as the revision number for a tag name of ''hyrax-4.5.0''


<font size="2"><source lang="sh">svn cp -m "olfs: Tagging OLFS release for hyrax-4.5.0" $svn/branch/olfs/release/hyrax-4.5 $svn/tags/olfs/hyrax-4.5.0</source></font>
<font size="2"><source lang="bash">svn cp -m "olfs: Tagging OLFS release for hyrax-4.5.0" $svn/branch/olfs/release/hyrax-4.5 $svn/tags/olfs/hyrax-4.5.0</source></font>


=== Example: Releasing a new revision an existing release ===
=== Example: Releasing a new revision an existing release ===
Line 122: Line 161:
And the most recent release tag was ''hyrax-4.5.6'' then add a increment the revision field for a tag name of ''hyrax-4.5.7''
And the most recent release tag was ''hyrax-4.5.6'' then add a increment the revision field for a tag name of ''hyrax-4.5.7''


<font size="2"><source lang="sh">svn cp -m "olfs: Tagging OLFS release for hyrax-4.5.7" $svn/branch/olfs/release/hyrax-4.5 $svn/tags/olfs/hyrax-4.5.7</source></font>
<font size="2"><source lang="bash">svn cp -m "olfs: Tagging OLFS release for hyrax-4.5.7" $svn/branch/olfs/release/hyrax-4.5 $svn/tags/olfs/hyrax-4.5.7</source></font>
-->
 
=== Make The Release ===
* Make the release in GitHub.
** Look at previous releases for title format.
** Use the NEWS entry for the description.
* Get The DOI for the release from Zendodo.
** Use the search api to find <cod>olfs xx.yy.zz</code>
** Copy the markup for the DOI badge and then
* Update the Release Page in Github with the DOI markup.


== Build the versioned distribution bundles ==
== Build the versioned distribution bundles ==
 
<!--
Build the OLFS distribution files by running ant and providing both the Hyrax version and OLFS version.  
Build the OLFS distribution files by running ant and providing both the Hyrax version and OLFS version.  


First get a copy of the new tagged version from svn:
First get a copy of the new tagged version from svn:


<font size="2"><source lang="sh">svn co $svn/tags/olfs/hyrax-4.5.0</source></font>
<font size="2"><source lang="bash">svn co $svn/tags/olfs/hyrax-4.5.0</source></font>
 
-->
In the local copy of the tag build the software:
In the local copy of the <!-- tag build the -->software:


<font size="2"><source lang="sh">ant -DOLFS_VERSION=1.2.3 -DHYRAX_VERSION=4.5.0 DISTRO</source></font>
<font size="2"><source lang="bash">ant -DOLFS_VERSION=1.2.3 -DHYRAX_VERSION=4.5.0 DISTRO</source></font>


This will create the three compressed tar files for the release in the $olfs/build/dist directory.
This will create the three compressed tar files for the release in the $olfs/build/dist directory.
Line 141: Line 190:
;olfs-1.2.3-webapp.tgz: Contains the instructions and .war file that will be used in the tomcat webapps directory.
;olfs-1.2.3-webapp.tgz: Contains the instructions and .war file that will be used in the tomcat webapps directory.


Use GPG to detach sign each of these three files and you're ready to upload them to the website for distribution.  
Use [[SecureEmail#Encrypting_files | GPG to detach sign]] each of these three files and you're ready to upload them to the website for distribution.  


The OLFS has historically been published here: http://www.opendap.org/pub/olfs/
The OLFS has historically been published here: http://www.opendap.org/pub/olfs/

Revision as of 15:15, 30 December 2020

<< back to HowTo Guides

Overview

This document describes how to make releases of the OLFS.

About version numbers:

  • We use Hyrax <major>.<minor> version numbers for release branches.
  • We use <major>.<minor>.<revision> version numbers for the olfs code version.

New versions of a release (only the <revision> field in the version number is changed) should contain:

  • Bug fixes.
  • Updated tests.
  • Changes to production rules.

These changes should be made on the trunk and the merged to the release branch.

Since major development is only happening on development branches, very little on the trunk should change until a development branch is merged into the trunk. If new features are added or the code API changes there should be a new release made, which means make a new release branch.

This page is based on the branching and merging guidelines found here.

Make sure the code is ready to release

Make sure that all of the developers working on the code have checked in their working copies of the trunk, and that all development branches whose functionality/features/codeBase are to included in the release have been merged to the trunk and committed.

Check out the master branch of the OLFS code from GitHub

In a new directory get a fresh copy of the trunk:

git clone https://github.com/opendap/olfs

Build the OLFS

In the checked out OLFS directory run the command:

% ant server

Run the tests

Updated Dec 29th 2020

The Hyrax regression tests utilize an autotools/automake/autotest test system. In order to to run the tests you must have the Hyrax server running. Build and install the BES first. Then build and install the OLFS into a Tomcat instance and start that instance.

Once you have the OLFS built, installed, and running do the following:

  • If you have not done so already, clone the hyrax_regression_tests project:
gitclone https://github.com/opendap/hyrax_regression_tests
  • Change directory into the top of hyrax_regression_tests and perform the following:
autoreconf -vif
./configure
make check

The tests may also be run directly:

./testsuite

Either command will run the tests in a traditional autotest test harness, against the server you should have running at http://localhost:8080/opendap/

The testsuite supports several notable command line options:

  • --besdev=yes|no Was the BES built using --enable-developer?
  • --hyraxurl=hyrax-service-endpoint-url Run the various tests (DAP2/4, w10n, wcs, etc.) against the Hyrax instance located at the specified endpoint URL. (default: http://localhost:8080/opendap)
  • --netrc=netrc_file_name Run tests using the specified netrc file (ala cURL) so that the test can authenticate to access the server. (default: ~/.netrc)

Determine Version Numbers

You will need to know the OLFS version and the Hyrax version of the release to build the distribution and to update the various release related files.

OLFS Version Number

  1. Look in the NEWS and README files to determine the previous version number.
  2. Determine new version number by looking at the changes to the code (start by reviewing the ChangeLog file)
    • Versioning rules: What the MAJOR.MINOR.REVISION string means:
      • No interfaces changed, only implementations (good): ==> Increment REVISION.
      • Interfaces added, none removed (good): ==> Increment MINOR, set REVISION to 0.
      • Interfaces removed or changed (BAD, breaks upward compatibility): ==> Increment MAJOR, set MINOR and REVISION to 0.
  3. Remember the new version number so that it can be used to:
    • Update the release related files
    • Build the versioned software distribution bundles.

Hyrax Version

Need a set of criteria or a link to them here.

Edit the release related files

ChangeLog
Use the script gitlog-to-changelog (which can be found with Google) to update the ChangeLog file by running it using the --since="<date>" option with a date one day later in time than the newest entry in the current ChangeLog. Then, to write the NEWS file, just read over the new ChangeLog entries and summarize.
Tip - This command gets the entire gitlog and reformats it to change log, wrapping and indenting long lines to under 80 characters. Be sure to update the date to the latest date in the current ChangeLog file.:
gitlog-to-changelog --since="1970-01-01" 2>&1 | fold -s -w 72 | awk '{if(!match($0,"^[0-9]|^[\\t]")){printf("\t%s\n",$0);}else{print $0;}}' -


README.md
Update version number.
Review content to make sure it's current.
NEWS
Add new version number and,
Summarize the new additions to the change log. There may be many checkins associated with new feature development. You can summarize those as "Added 'X' feature."
install.html
Update version number.
Review content to make sure it's current.

Check in changes

In the local copy of the OLFS master branch in which you updated the ChangeLog and other files:

git commit -a -m "olfs: Updating olfs version and release files to version 1.2.3"

Once this is done, use github to tag the code you just committed both with its version and the version of hyrax to which it belongs:

Tag the release

git tag -m "olfs-<numbers>" -a olfs-<numbers> 
git push origin olfs-<numbers>

If this is part of Hyrax, also tag this point in the master branch with the Hyrax release number

git tag -m "hyrax-<numbers>" -a hyrax-<numbers> 
git push origin hyrax-<numbers>
Note
Instead of tagging all of the modules, use the saved commit hashes that git tracks for submodules. This cuts down on the bookkeeping for releases and removes one source of error; recording the wrong version for one of the 17 or so modules. thus, there is no need to tag the code in the modules.


Make The Release

  • Make the release in GitHub.
    • Look at previous releases for title format.
    • Use the NEWS entry for the description.
  • Get The DOI for the release from Zendodo.
    • Use the search api to find <cod>olfs xx.yy.zz
    • Copy the markup for the DOI badge and then
  • Update the Release Page in Github with the DOI markup.

Build the versioned distribution bundles

In the local copy of the software:

ant -DOLFS_VERSION=1.2.3 -DHYRAX_VERSION=4.5.0 DISTRO

This will create the three compressed tar files for the release in the $olfs/build/dist directory.

olfs-1.2.3-doc.tgz
Contains the javadoc documentation for the release
olfs-1.2.3-src.tgz
Contains the source files for the release
olfs-1.2.3-webapp.tgz
Contains the instructions and .war file that will be used in the tomcat webapps directory.

Use GPG to detach sign each of these three files and you're ready to upload them to the website for distribution.

The OLFS has historically been published here: http://www.opendap.org/pub/olfs/