Difference between revisions of "How to Make a Release"

From OPeNDAP Documentation
(Steps For A Minor Release (ex: 1.16.5 -> 1.16.6))
m (Send out a notice.)
 
(37 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
This is an updated set of notes regarding a software release. It is tailored to a release of Hyrax, but the same process can be used for any of our code.
 +
 +
# Look for any tickets that have been bound to the label 'fix release x.y.z' and make sure those are completed. Be sure to use both:
 +
#* The HyraxKB Kanban board in OPeNDAP's JIRA.
 +
#* The Hyrax board in our NASA JIRA.
 +
# When we are ready to release, follow the process outlined in the sections below.
 +
# Once done, close the board
 +
 
== When we decide to release software ==
 
== When we decide to release software ==
 
Use the developers at opendap.org list to send out notices to all the developers about pending releases.
 
Use the developers at opendap.org list to send out notices to all the developers about pending releases.
Line 11: Line 19:
  
 
Maybe run Coverity - depends on the scope of the changes.
 
Maybe run Coverity - depends on the scope of the changes.
 
== Overview ==
 
This is an updated set of notes regarding a software release. It is tailored to a release of Hyrax, but the same process can be used for any of our code.
 
 
# Look for any tickets that have been bound to the label 'fix release x.y.z' and make sure those are completed. Use the Kanban board.
 
# When we are ready to release, follow the process outlined in the section below.
 
# Once done, close the board
 
  
 
== Release Prerequisites  ==
 
== Release Prerequisites  ==
Line 29: Line 30:
 
== Steps For A Minor Release (ex: 1.16.5 -> 1.16.6) ==
 
== Steps For A Minor Release (ex: 1.16.5 -> 1.16.6) ==
 
; Notes
 
; Notes
:* ''Committing and pushing the various changes made for the source releases will trigger a binary build and those binaries will have the version numbers that making the source distributions update.''
+
* ''Committing and pushing the various changes made for the source releases will trigger a binary build and those binaries will have the version numbers made in the source distributions update.''
:* ''Perform these steps in the order stated as downstream activities rely on items created in earlier steps, such as the creation of the DOIs for libdap and BES are required when updating the OLFS README.md, ''
+
 
 +
* ''Perform these steps in the order stated as downstream activities rely on items created in earlier steps, such as the creation of the DOIs for libdap and BES are required when updating the OLFS README.md, ''
  
 
=== Perform the [[Jira Release Process]] ===
 
=== Perform the [[Jira Release Process]] ===
  
=== Source release for hyrax-dependencies ===
+
 
# Make sure the ''hyrax-dependencies'' project is up to date and tar balls on www.o.o. If there have been changes/updates since the last Hyrax release then :
+
=== hyrax-dependencies: [[Source Release For hyrax-dependencies]] ===
# Update version number for the ''hyrax-dependencies'' in the Makefile, for example "1.xx"
 
# Save, commit, and push the changes to master branch.
 
# Once the ''hyrax-dependencies'' CI build is finished, trigger builds CI in both ''libdap4'' and ''bes'' by pushing change(s) to the master branch of each.
 
# Tag, and push the tag.
 
#* ''git tag -m "version-<number>" -a <numbers>''
 
#* ''git push origin <numbers>''
 
# Since this is part of Hyrax, also tag this point in the master branch with the Hyrax release number:
 
#* ''git tag -m "hyrax-<number>" -a hyrax-<numbers>''
 
#* ''git push origin hyrax-<numbers>''
 
# Make 'release' in github
 
## Goto the [https://github.com/OPENDAP/hyrax-dependencies/tags GitHub 'tags' page for ''hyrax-dependencies''].
 
## Enter a title for the release (looks at previous releases for examples)
 
## Copy the most recent text from the NEWS file into the describe field
 
## Click Update this release or Save draft, this will trigger a 'archive and DOI' process on the Zenodo system.
 
  
 
=== libdap: [[Source Release for libdap]] ===
 
=== libdap: [[Source Release for libdap]] ===
 +
  
 
=== BES: [[Source Release for BES]] ===
 
=== BES: [[Source Release for BES]] ===
  
=== OLFS:  [[OLFSReleaseGuide | Make the OLFS release Source and Binary release files]] ===
+
=== OLFS:  [[OLFSReleaseGuide | Source and Binary release for OLFS]] ===
Follow these steps to create the three .jar files needed for the OLFS release.
 
  
=== Upload the source distributions ===
+
=== [[Signing And Publishing C++ Binary Distributions]] ===
Make and upload the source distributions for hyrax-dependencies, libdap, BES and, OLFS - delay signing until later.  
+
: '''Note:''' ''This was already done for the java OLFS in the previous steps.''
  
 +
=== [[HyraxDockerReleaseGuide | Build And Publish Docker Images For The Release]] ===
  
 
=== Install the new server on test.opendap.org ===
 
=== Install the new server on test.opendap.org ===
# For the C++ software, Transfer the binary packages built using the CI/CD process. Log into the ''www.opendap.org'' web site host using ssh and the ''opendap'' account, and then use ''curl -O ...'' or ''wget --no-check-certificate ...'' to transfer the RPMs and Debian packages from our ''opendap.travis.build'' S3 bucket. The S3 bucket can be found at: https://s3.amazonaws.com/opendap.travis.build/index.html
 
#
 
# Update the links on the web page made/copies/edited above. Test each link - source and binary. Once all of those work...
 
# Sign the packages (source and binary). Do this by downloading the packages, signing and then uploading the ''.sig'' files - no need to upload the actual packages. See [[SecureEmail | Secure Email]] for a hack to batch sign packages.
 
# Follow the instructions in [[HyraxDockerReleaseGuide | Hyrax Docker Release Guide]] to create the docker images on docker-hub.
 
  
=== Update the release page on website ===
+
== [[Update The Hyrax Release Web Page]] ==
but don't publish it. For this, copy the X.Y page and make it X.Y.n. In the future, we might use only a single page for release X.Y and simply amend it with information for version ''.1'', ''.2'', and so on. Don't bother to edit the links, just get the text in place.
+
== Send out a notice. ==
 +
Here's a sample email message. I sent this to: "OPeNDAP Inc." <opendap-tech@opendap.org>, opendap-announce@opendap.org, opendap@opendap.org
 +
 
 +
Subject: We are pleased to announce the release of Hyrax 1.16.8
 +
 
 +
<pre>
 +
Greetings!
 +
 
 +
OPeNDAP is pleased to announce the release of Hyrax 1.16.8. This version of the DAP2 and DAP4 data server is available as RedHat 8 and CentOS 7 RPM packages, Docker containers and source code via GitHub. See https://www.opendap.org/software/hyrax/1.16#SoftwareDownloads for the details.
 +
 
 +
Hyrax is Open Source.
 +
 
 +
Below are the changes/updates for this version of the server.
 +
 
 +
DMR++ Improvements - Direct subsetting access to data in S3
 +
 
 +
• Various improvements for supporting FillValue in dmr++ lifecycle.
 +
• Improved support for arrays of type String.
 +
• Fixed trusted url bug in DMZ parser.
 +
• Added support for "empty" valued scalars with associated _FillValue metadata.
 +
• get_dmrpp Improvements
 +
• Added support for S3 hosted granules to get_dmrpp
 +
bes
 +
 
 +
• Support for RHEL8
 +
• Improved the error messages returned to the client
 +
• Refactored get_dmrpp, application. Some test features are still broken but core functionality is working now.
 +
• Improved support for more GES DISC level 3 and level 4 products
 +
• Added coverage the support for the AIRS level 3 and GLDAS level products.
 +
• Patched bug in HDF5Array.cc introduced by the std::vector refactor
 +
• Added time.h header to ppt/SocketUtilities.cc
 +
• Modified fileout_netcdf handler to allow netcdf-3 responses to be up to 4GB in size.
 +
This behavior can be reverted by setting FONc.NC3ClassicFormat=true in the BES configuration
 +
(aka /etc/bes/site.conf file)
 +
olfs
 +
 
 +
• Improved the DAP4 Data Request Form (now the default form for the server)
 +
• Patched bug where unexpected Authentication headers would trigger a redirect loop.
 +
• Fixed the broken service that was failing to deliver flat (not-data) files to clients.
 +
• Made the "Get As NetCDF-3" an "Get As DAP2 Binary" buttons on the DAP4 Data Request Form context sensitive. If the dataset in question contains variables whose data types are found in DAP4 and not in DAP2/NetCDF-3 then the buttons are disabled. A more complete solution is envisioned where the projected variables are assessed and if only DAP2/NetCDF-3 types are selected then the buttons would would be enabled. This fix is only a step in that more dynamic direction.
 +
• Tested on Tomcat-9.0.x
 +
 
 +
Thank you for your continued support,
 +
 
 +
The Hyrax development team: Nathan Potter, Dan Holloway, Sam Lloyd, Kodi Neumiller, Kent Yang and James Gallagher
  
# Publish the release page. Check the links - there are links in two menus that have to be set manually.  
+
Acknowledgement: OPeNDAP thanks NASA and Raytheon for continued support of the Hyrax Data Server.
=== Send out a notice. ===
+
 +
James Gallagher
 +
jgallagher@opendap.org
 +
</pre>
  
=== Time to complete by release ===
+
== Time to complete by release ==
 
Update and of the release checklists and the 'Time to complete...' information below.
 
Update and of the release checklists and the 'Time to complete...' information below.
  

Latest revision as of 17:08, 29 July 2022

This is an updated set of notes regarding a software release. It is tailored to a release of Hyrax, but the same process can be used for any of our code.

  1. Look for any tickets that have been bound to the label 'fix release x.y.z' and make sure those are completed. Be sure to use both:
    • The HyraxKB Kanban board in OPeNDAP's JIRA.
    • The Hyrax board in our NASA JIRA.
  2. When we are ready to release, follow the process outlined in the sections below.
  3. Once done, close the board

1 When we decide to release software

Use the developers at opendap.org list to send out notices to all the developers about pending releases.

1.1 Planned release

If this is a planned release, send out a notice at least two weeks in advance to anyone who likely has made changes to the software. More lead time is, of course, better. This will allow developers time to get their code and documentation changes into github and onto the master branch in time for the release.

Run Coverity!

1.2 Emergency releases

If this is an 'emergency' release, then send out a notice to developers as soon as the decision to release is made, since this will give a chance for other developers to lets us know if there are new changes in the master branch that are ready for release. if we don't get an email from those developers of a particular component, then we should assume that the code might not be 100% ready for release and we should use the version tagged with/for the last release if possible.

Maybe run Coverity - depends on the scope of the changes.

2 Release Prerequisites

Only start this process when hyrax-dependencies, libdap, bes and OLFS master branches all building and passing their tests.

3 Steps For A Major Release (ex: 1.16.x -> 1.17.0)

  1. Security Review for libdap, BES and OLFS Now Handled By TravisCI
  2. Write a new X.Y web page. See the web site admin notes for info about this. It has important information about the menu links. Base this on the previous release page's structure/content.
  3. Follow the remaining steps for a minor release (below).

4 Steps For A Minor Release (ex: 1.16.5 -> 1.16.6)

Notes
  • Committing and pushing the various changes made for the source releases will trigger a binary build and those binaries will have the version numbers made in the source distributions update.
  • Perform these steps in the order stated as downstream activities rely on items created in earlier steps, such as the creation of the DOIs for libdap and BES are required when updating the OLFS README.md,

4.1 Perform the Jira Release Process

4.2 hyrax-dependencies: Source Release For hyrax-dependencies

4.3 libdap: Source Release for libdap

4.4 BES: Source Release for BES

4.5 OLFS: Source and Binary release for OLFS

4.6 Signing And Publishing C++ Binary Distributions

Note: This was already done for the java OLFS in the previous steps.

4.7 Build And Publish Docker Images For The Release

4.8 Install the new server on test.opendap.org

5 Update The Hyrax Release Web Page

6 Send out a notice.

Here's a sample email message. I sent this to: "OPeNDAP Inc." <opendap-tech@opendap.org>, opendap-announce@opendap.org, opendap@opendap.org

Subject: We are pleased to announce the release of Hyrax 1.16.8

Greetings!

OPeNDAP is pleased to announce the release of Hyrax 1.16.8. This version of the DAP2 and DAP4 data server is available as RedHat 8 and CentOS 7 RPM packages, Docker containers and source code via GitHub. See https://www.opendap.org/software/hyrax/1.16#SoftwareDownloads for the details.

Hyrax is Open Source.

Below are the changes/updates for this version of the server.

DMR++ Improvements - Direct subsetting access to data in S3

	• Various improvements for supporting FillValue in dmr++ lifecycle.
	• Improved support for arrays of type String.
	• Fixed trusted url bug in DMZ parser.
	• Added support for "empty" valued scalars with associated _FillValue metadata.
	• get_dmrpp Improvements
	• Added support for S3 hosted granules to get_dmrpp
bes

	• Support for RHEL8
	• Improved the error messages returned to the client
	• Refactored get_dmrpp, application. Some test features are still broken but core functionality is working now.
	• Improved support for more GES DISC level 3 and level 4 products
	• Added coverage the support for the AIRS level 3 and GLDAS level products.
	• Patched bug in HDF5Array.cc introduced by the std::vector refactor
	• Added time.h header to ppt/SocketUtilities.cc
	• Modified fileout_netcdf handler to allow netcdf-3 responses to be up to 4GB in size.
		This behavior can be reverted by setting FONc.NC3ClassicFormat=true in the BES configuration
		(aka /etc/bes/site.conf file)
olfs

	• Improved the DAP4 Data Request Form (now the default form for the server)
	• Patched bug where unexpected Authentication headers would trigger a redirect loop.
	• Fixed the broken service that was failing to deliver flat (not-data) files to clients.
	• Made the "Get As NetCDF-3" an "Get As DAP2 Binary" buttons on the DAP4 Data Request Form context sensitive. If the dataset in question contains variables whose data types are found in DAP4 and not in DAP2/NetCDF-3 then the buttons are disabled. A more complete solution is envisioned where the projected variables are assessed and if only DAP2/NetCDF-3 types are selected then the buttons would would be enabled. This fix is only a step in that more dynamic direction.
	• Tested on Tomcat-9.0.x

Thank you for your continued support,

The Hyrax development team: Nathan Potter, Dan Holloway, Sam Lloyd, Kodi Neumiller, Kent Yang and James Gallagher

Acknowledgement: OPeNDAP thanks NASA and Raytheon for continued support of the Hyrax Data Server.
—
James Gallagher
jgallagher@opendap.org

7 Time to complete by release

Update and of the release checklists and the 'Time to complete...' information below.

  • Hyrax 1.12.2: The total time was planned as 3w 1d 5h and turned out as 1w 2d 2h 10m.
  • Hyrax 1.15.1: 3d calendar time
  • Hyrax 1.16.2: 3d effort