Developer Info: Difference between revisions

From OPeNDAP Documentation
⧼opendap2-jumptonavigation⧽
(25 intermediate revisions by 2 users not shown)
Line 1: Line 1:


* [https://github.com/OPENDAP OPeNDAP's GitHub repositories]: OPeNDAP's software is available using GitHub in addition to the doenloads from our website.
* [https://github.com/OPENDAP OPeNDAP's GitHub repositories]: OPeNDAP's software is available using GitHub in addition to the doenloads from our website.
** Before 2015 we hosted our own SVN repository. It's still online and available, but for read-only access, at [https://scm.opendap.org/svn https://scm.opendap.org/svn].
* [https://travis-ci.org/OPENDAP Continuous Integration builds]: Software that is built whenever new changes are pushed to the master branch. These builds are done on the Travis-CI system.
* [https://travis-ci.org/OPENDAP Continuous Integration builds]: Software that is built whenever new changes are pushed to the master branch. These builds are done on the Travis-CI system.
* [http://test.opendap.org/ test.opendap.org]: Test servers with data files.
* [http://test.opendap.org/ test.opendap.org]: Test servers with data files.
Line 8: Line 9:
** [https://scan.coverity.com/projects/opendap-libdap4?tab=overview libdap - The implementation of DAP2 and DAP4]
** [https://scan.coverity.com/projects/opendap-libdap4?tab=overview libdap - The implementation of DAP2 and DAP4]


== Developer Guidelines and Information ==
== OPeNDAP's FAQ ==
The [http://www.opendap.org/faq-page OPeNDAP FAQ] has a pretty good section on developer's questions.
The [http://www.opendap.org/faq-page OPeNDAP FAQ] has a pretty good section on developer's questions.
== General development information ==
These pages contain general information relevant to anyone working with our software:
* [[Git Hacks and Tricks]]: Information about using git and/or GitHub that seems useful and maybe not all that obvious.
* [https://wiki.wxwidgets.org/Valgrind_Suppression_File_Howto Valgrind - How to generate suppression files for valgrind] This will quiet valgrind, keeping it from telling you OS/X or Linux (or the BES) is leaking memory.
* [[Migrating source code from SVN to git]]: How to move a large project from SVN to git and keep the history, commits, branches and tags.
* [https://developer.mozilla.org/en-US/docs/Eclipse_CDT Eclipse - Detailed information about running Eclipse on OSX from the Mozzilla project]. Updated in 2017, this is really good but be aware that it's specific to Mozilla so some of the tips don't apply. Hyrax (i.e., libdap4 and BES) also use their own build system (autotools + make) so most of the configuration information here is very apropos. See also [[How to use Eclipse with Hyrax Source Code]] below.
* [https://jfearn.fedorapeople.org/en-US/RPM/4/html/RPM_Guide/index.html RPM Guide] The best one I'm found so far...


== OPeNDAP Workshops ==
== OPeNDAP Workshops ==
Line 24: Line 16:
* [http://www.opendap.org/about/workshops-and-presentations/2008-07-15 ESIP Federation Server Workshop]: This half-day workshop focused on server installation and configuration. Summer 2008
* [http://www.opendap.org/about/workshops-and-presentations/2008-07-15 ESIP Federation Server Workshop]: This half-day workshop focused on server installation and configuration. Summer 2008
* [[A One-day Course on Hyrax Development | Server Functions]]: This one-day workshop is all about writing and debugging server-side functions. It also contains a wealth of information about Hyrax, the BES and debugging tricks for the server. Spring 2012. Updated Fall 2014 for presentation to Ocean Networks Canada.
* [[A One-day Course on Hyrax Development | Server Functions]]: This one-day workshop is all about writing and debugging server-side functions. It also contains a wealth of information about Hyrax, the BES and debugging tricks for the server. Spring 2012. Updated Fall 2014 for presentation to Ocean Networks Canada.
== libdap4 and BES Reference documentation ==
* [https://opendap.github.io/bes/html/ BES Reference]
* [https://opendap.github.io/libdap4/html/ libdap Reference]
== BES Development Information ==
* [[Hyrax - Logging Configuration|Logging Configuration]]
* [[BES_-_How_to_Debug_the_BES| How to debug the BES]]
* [[BES - Debugging Using besstandalone]]
* [[Hyrax - Create BES Module | How to create your own BES Module]]
* Hyrax Module Integration: How to configure your module so it's easy to add to Hyrax instances ([[:File:HyraxModuleIntegration-1.2.pdf|pdf]])
* [[Hyrax - Starting and stopping the BES| Starting and stopping the BES]]
* [[Hyrax - Running bescmdln | Running the BES command line client]]
* [[Hyrax - BES Client commands| BES Client commands]]. The page [[BES_XML_Commands | BES XML Commands]] repeats this info for a bit more information on the return values. Most of the commands don't return anything unless they return an error and are expected to be used in a group where a ''get'' command closes out the request and obviously does return a response of some kind (maybe an error).
* [[Hyrax:_BES_Administrative_Commands| BES Administrative Commands]]
* [[Hyrax - Extending BES Module | Extending your BES Module]]
* [[Hyrax - Example BES Modules | Example BES Modules]] - the Hello World example and the CSV data handler
* [[Hyrax - BES PPT | BES communication protocol using PPT (point to point transport)]]
* [[Australian BOM Software Developer's Agenda and Presentations|Software Developers Workshop]]


== OPeNDAP Development process information  ==
== OPeNDAP Development process information  ==
Line 29: Line 42:


* [[Hyrax GitHub Source Build]] This explains how to clone our software from GitHub and build our code using a shell like bash. It also explains how to build the BES and all of the Hyrax 'standard' handlers in one operation, as well as how to build just the parts you need without cloning the whole set of repos. Some experience with 'git submodule' will make this easier, although the page explains everything.
* [[Hyrax GitHub Source Build]] This explains how to clone our software from GitHub and build our code using a shell like bash. It also explains how to build the BES and all of the Hyrax 'standard' handlers in one operation, as well as how to build just the parts you need without cloning the whole set of repos. Some experience with 'git submodule' will make this easier, although the page explains everything.
* [[ReleaseSprintNotes | How to Plan a Release Sprint]] A general template for planning a release in Jira or the equivalent.
* [[ReleaseGuide | Making a Release]]. Once software is ready for distribution, use this checklist to make sure you do all of the steps needed to make a release.
* [[RPM |Making RPM Distributions]]. Follow these steps to create an RPM distribution of the software.
* [[OLFSReleaseGuide| Making a Release of OLFS]]. Follow these steps to create the three .jar files needed for the OLFS release. Includes information on how to build the OLFS and how to run the tests.
* [[Bug Prioritization]]. How we prioritize bugs in our software.
* [[Bug Prioritization]]. How we prioritize bugs in our software.


===== Software process issues: =====
===Making a Release ===
* [[How_to_Make_a_Release]] A general template for making a release. This references some of the pages below.
====The Release Process====
# [[Source_Release_for_libdap | Making a source release of libdap]]
# [[ReleaseGuide | Making a source release of the BES]].
# [[OLFSReleaseGuide| Make the OLFS release WAR file]]. Follow these steps to create the three .jar files needed for the OLFS release. Includes information on how to build the OLFS and how to run the tests.
# [[HyraxDockerReleaseGuide|Make the official Hyrax Docker image for the release]] When the RPMs and the WAR file(s) are built and pushed to their respective download locations, make the Docker image of the release.
 
====Supplemental release guides====
<font color="red">Old - use the packages built using the Continuous Delivery process</font>
# [[RPM |Make the RPM Distributions]]. Follow these steps to create an RPM distribution of the software. '''Note:''' ''Now we use packages built using CI/CD, so this checklist is no longer needed.''
 
== Software process issues: ==
* [[ConfigureCentos| How to configure a CentOS machine for production of RPM binaries]] - Updated 12/2014 to include information regarding git.
* [[ConfigureCentos| How to configure a CentOS machine for production of RPM binaries]] - Updated 12/2014 to include information regarding git.
* [[How to use Eclipse with Hyrax Source Code]] Assuming you have cloned our Hyrax code from GitHub, this explains how to setup eclipse so you can work fairly easily and switch back and forth between the shell, emacs and eclipse.
* [[How to use Eclipse with Hyrax Source Code]] Assuming you have cloned our Hyrax code from GitHub, this explains how to setup eclipse so you can work fairly easily and switch back and forth between the shell, emacs and eclipse.
Line 41: Line 62:
* [[UnitTests| How to write unit tests using CppUnit]]
* [[UnitTests| How to write unit tests using CppUnit]]
* [[valgrind| How to use valgrind with unit tests]]
* [[valgrind| How to use valgrind with unit tests]]
* [[Debugging the distcheck target]] Yes, this gets its own page...
* [[CopyRights| How to copyright software written for OPeNDAP]]
* [[CopyRights| How to copyright software written for OPeNDAP]]
* [[Managing public and private keys using gpg]]
* [[Managing public and private keys using gpg]]
Line 53: Line 75:
* [[How Shutoff the CentOS firewall]]
* [[How Shutoff the CentOS firewall]]


= TRANSFERED FROM HYRAX INSTALL PAGE =
== General development information ==
== For Software Developers ==
These pages contain general information relevant to anyone working with our software:
* [[Hyrax GitHub Source Build]] We have moved the Hyrax source code to GitHub; please don't use the old, read-only SVN repo to build.
* [[How to use Eclipse with Hyrax Source Code]] Note that this is a a work in progress, but it will help with some of the odd steps that Eclipse seems to require.
 
===== BES Development Information =====
* [[Hyrax - Logging Configuration|Logging Configuration]]
 
* [[BES_-_How_to_Debug_the_BES| How to debug the BES]]
* [[BES - Debugging Using besstandalone]]
* [[Hyrax - Create BES Module | How to create your own BES Module]]
* Hyrax Module Integration: How to configure your module so it's easy to add to Hyrax instances ([[:File:HyraxModuleIntegration-1.2.pdf|pdf]])
* [[Hyrax - Starting and stopping the BES| Starting and stopping the BES]]
* [[Hyrax - Running bescmdln | Running the BES command line client]]
* [[Hyrax - BES Client commands| BES Client commands]]
* [[BES_XML_Commands | BES XML Commands]]
* [[Hyrax - Extending BES Module | Extending your BES Module]]
* [[Hyrax - Example BES Modules | Example BES Modules]] - the Hello World example and the CSV data handler
* [[Hyrax - BES PPT | BES communication protocol using PPT (point to point transport)]]
* [[Hyrax:_BES_Administrative_Commands| BES Administrative Commands]]
 
* [[Australian BOM Software Developer's Agenda and Presentations|Software Developers Workshop]]
 
<!-- There is a link to this on the main opendap.org page. I'll put a copy of that on the Developer Info page, where it's more appropriate. jhrg * [[Australian BOM Software Developer's Agenda and Presentations|Software Developers Workshop]] -->
 
==Reference documentation==
 
* [http://old.opendap.org/api/bes/html/index.html BES Reference]
* [http://old.opendap.org/api/libdap/html/index.html libdap Reference]


* [[Git Hacks and Tricks]]: Information about using git and/or GitHub that seems useful and maybe not all that obvious.
* [[Using a debugger for C++ with Eclipse on OS/X]] Short version: use lldbmi2 **Add info**
* [[Using ASAN]] Short version, look [https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger at the Google/GitHub pages] for useful environment variables **add text** On Centos, use yum install llvm to get the 'symbolizer' and try ''ASAN_OPTIONS=symbolize=1 ASAN_SYMBOLIZER_PATH=$(shell which llvm-symbolizer)''
* [[How to use ''Instruments'' on OS/X to profile]] Updated 7/2018
* [https://wiki.wxwidgets.org/Valgrind_Suppression_File_Howto Valgrind - How to generate suppression files for valgrind] This will quiet valgrind, keeping it from telling you OS/X or Linux (or the BES) is leaking memory.
* [[Migrating source code from SVN to git]]: How to move a large project from SVN to git and keep the history, commits, branches and tags.
* [https://developer.mozilla.org/en-US/docs/Eclipse_CDT Eclipse - Detailed information about running Eclipse on OSX from the Mozzilla project]. Updated in 2017, this is really good but be aware that it's specific to Mozilla so some of the tips don't apply. Hyrax (i.e., libdap4 and BES) also use their own build system (autotools + make) so most of the configuration information here is very apropos. See also [[How to use Eclipse with Hyrax Source Code]] below.
* [https://jfearn.fedorapeople.org/en-US/RPM/4/html/RPM_Guide/index.html RPM Guide] The best one I'm found so far...
* [https://autotools.io/index.html Autotools Myth busters] The best info on autotools I've found yet (covers ''autoconf'', ''automake'', ''libtool'' and ''pkg-config'').
* The [https://www.gnu.org/software/autoconf/autoconf.html autoconf] manual
* The [https://www.gnu.org/software/automake/ automake] manual
* The [https://www.gnu.org/software/libtool/ libtool] manual
* A good [https://lldb.llvm.org/lldb-gdb.html gdb to lldb cheat sheet] for those of us who know ''gdb'' but not ''lldb''.


= Old information =
= Old information =

Revision as of 21:56, 6 December 2018

OPeNDAP's FAQ

The OPeNDAP FAQ has a pretty good section on developer's questions.

OPeNDAP Workshops

  • The APAC/BOM Workshops: This workshop spanned several days and covered a number of topics, including information for SAs and Developers. Oct 2007.
  • ESIP Federation Server Workshop: This half-day workshop focused on server installation and configuration. Summer 2008
  • Server Functions: This one-day workshop is all about writing and debugging server-side functions. It also contains a wealth of information about Hyrax, the BES and debugging tricks for the server. Spring 2012. Updated Fall 2014 for presentation to Ocean Networks Canada.

libdap4 and BES Reference documentation

BES Development Information

OPeNDAP Development process information

These pages contain information about how we'd like people working with us to use our various on-line tools.

  • Hyrax GitHub Source Build This explains how to clone our software from GitHub and build our code using a shell like bash. It also explains how to build the BES and all of the Hyrax 'standard' handlers in one operation, as well as how to build just the parts you need without cloning the whole set of repos. Some experience with 'git submodule' will make this easier, although the page explains everything.
  • Bug Prioritization. How we prioritize bugs in our software.

Making a Release

The Release Process

  1. Making a source release of libdap
  2. Making a source release of the BES.
  3. Make the OLFS release WAR file. Follow these steps to create the three .jar files needed for the OLFS release. Includes information on how to build the OLFS and how to run the tests.
  4. Make the official Hyrax Docker image for the release When the RPMs and the WAR file(s) are built and pushed to their respective download locations, make the Docker image of the release.

Supplemental release guides

Old - use the packages built using the Continuous Delivery process

  1. Make the RPM Distributions. Follow these steps to create an RPM distribution of the software. Note: Now we use packages built using CI/CD, so this checklist is no longer needed.

Software process issues:

AWS Tips

General development information

These pages contain general information relevant to anyone working with our software:

Old information

Note: The following is all about using Subversion and is out of date as of November 2014 when we switched to git. There are still good ideas here...