OPeNDAP Documentation - User contributions [en]

https://docs.opendap.org/api.php?action=feedcontributions&feedformat=atom&user=PatrickWest OPeNDAP Documentation - User contributions [en] 2026-04-16T09:11:43Z User contributions MediaWiki 1.38.4 https://docs.opendap.org/index.php?title=BES_-_Modules&diff=9939 BES - Modules 2013-09-23T22:38:44Z

<p>PatrickWest: </p> <hr /> <div>The following is a list of modules that can be dynamically loaded into the BES.<br /> * default BES module<br /> * dap and dapcmd modules<br /> * netcdf data handler<br /> * [[BES_-_Modules_-_The_HDF4_Handler|hdf4 data handler]]<br /> * hdf5 data handler<br /> * CSV Handler ''Update this and then edit www.opendap.org/download/csv_handler.html''<br /> * [[BES - Modules - Gateway Module|gateway handler]] (Provides interoperability between Hyrax and other web services)<br /> * [[The_FreeForm_Data_Handler|freeform data handler]]<br /> * [[BES - Modules - FileOut Netcdf|file out netcdf]]<br /> * [[BES - Modules - NcML Module|NcML data handler]]<br /> <br /> [[Category:BES Modules|BES Modules]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9592 Google Summer of Code 2013 Ideas 2013-03-27T18:56:08Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> <br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> * [[Development of improved Hyrax User Interfaces]]: Browsers, JavaScript<br /> * [[KML responses for Hyrax]]: C++<br /> * [[Packaging and Installation]]: Linux, RPM, yum, aptitude, scripting<br /> * [[Python and PHP clients to BES]]: Python and PHP<br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9591 Google Summer of Code 2013 Ideas 2013-03-27T18:53:23Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> <br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> * [[Development of improved Hyrax User Interfaces]]: Browsers, JavaScript<br /> * [[Extending support for Matlab]]: C; Matlab a plus<br /> * [[KML responses for Hyrax]]: C++<br /> * [[Packaging and Installation]]: Linux, RPM, yum, aptitude, scripting<br /> * [[Python and PHP clients to BES]]: Python and PHP<br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9590 Google Summer of Code 2013 Ideas 2013-03-27T10:01:30Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> <br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> * [[Development of improved Hyrax User Interfaces]]: Browsers, JavaScript<br /> * [[Extending support for Matlab]]: C; Matlab a plus<br /> * [[KML responses for Hyrax]]: C++<br /> * [[GeoTIFF responses for Hyrax]]: C++<br /> * [[Packaging and Installation]]: Linux, RPM, yum, aptitude, scripting<br /> * [[Python and PHP clients to BES]]: Python and PHP<br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9589 Google Summer of Code 2013 Ideas 2013-03-27T09:59:27Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> <br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> * [[Development of improved Hyrax User Interfaces]]: Browsers, JavaScript<br /> * [[Extending support for Matlab]]: C; Matlab a plus<br /> * [[KML responses for Hyrax]]: C++<br /> * [[GeoTIFF responses for Hyrax]]: C++<br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9588 Google Summer of Code 2013 Ideas 2013-03-27T09:59:03Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> <br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> <br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9587 Google Summer of Code 2013 Ideas 2013-03-27T09:58:55Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png]]<br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> <br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=File:GSoC2013_300x200.png&diff=9586 File:GSoC2013 300x200.png 2013-03-27T09:58:28Z

<p>PatrickWest: </p> <hr /> <div></div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9585 Google Summer of Code 2013 Ideas 2013-03-27T09:54:43Z

<p>PatrickWest: </p> <hr /> <div>[[Image :GSoC2013 300x200.png|right]]<br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> <br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Google_Summer_of_Code_2013_Ideas&diff=9584 Google Summer of Code 2013 Ideas 2013-03-27T09:53:07Z

<p>PatrickWest: Created page with "right OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 20..."</p> <hr /> <div>[[Image :GSoC2012 300x200.png|right]]<br /> OPeNDAP would like to welcome all the [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 Summer of Code] (SoC) students for the 2013 iteration of the SoC program! On this page you'll find information about project ideas we have along with information to help you apply to work on our software. The projects range from web site implementation to complex problems in data systems. Several of the projects listed are focused on adding new features to our data server. However, we also have a set of client tools and GUI-based applications that would provide an ample basis for summer projects. If you don't see anything below in the [[#Ideas]] section don't let that stop you from starting a conversation with one of the mentors listed below ([[#Mentors]]) because you may have an idea, talent or skill that is the spark of a great project. Don't hesitate to email us! <br /> <br /> We would like to stress that while the ideas included here can be the basis of your project, you can also propose something completely different - maybe you've used our software and wish it included a particular feature.<br /> <br /> Before submitting an application to GSoC with OPeNDAP as your mentoring organization, make sure you read our [[Google Summer of Code FAQ|GSoC FAQ]] as well as the [http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page Google] FAQ, both of which provide some pointers on how to write a successful application.<br /> <br /> <br /> <br /> = Ideas =<br /> <br /> <br /> = Mentors =<br /> <br /> * Patrick West (westp at rpi.edu): The Hyrax data server, C, C++, Java, Apache Modules, Semantic Web, OWL (Web Ontology Language), RDBMS, Java<br /> <br /> = Proposal template =<br /> <br /> Each proposal should be a text document that should contain the following sections:<br /> ;Synopsis: A one-paragraph description of the proposal<br /> ;Project Description: A complete description of what you will do including a logical breakdown of the project into smaller parts and how the project will be completed on time<br /> ;Profit for OPeNDAP Users: How will users of OPeNDAP's open source software benefit from this project?<br /> ;Success Criteria: What's the target you are trying to reach with this project so that success can easily be measured<br /> ;Roadmap: Describe the different development phases you expect to go through. For example: <br /> # Add a description of my project to the OPeNDAP developer's wiki<br /> # Solicit community feedback<br /> # Develop base-level features<br /> # Testing and alpha release<br /> # Develop additional features <br /> # ...<br /> ;Biography: Tell us about yourself, why you want to do this and how to contact you<br /> <br /> '''Note''': Proposals without contact information ''will not be accepted''.<br /> <br /> [[Category:Google Summer of Code]]</div>

PatrickWest https://docs.opendap.org/index.php?title=MediaWiki:Sidebar&diff=9397 MediaWiki:Sidebar 2012-09-17T18:06:26Z

<p>PatrickWest: </p> <hr /> <div>* navigation<br /> ** mainpage|Home<br /> ** Documentation|Documentation<br /> ** Developer Info|Developer Info<br /> ** Development|Projects<br /> ** OPULS Development|OPULS/DAP4<br /> ** Working_Groups|Working Groups<br /> ** Meetings|Meetings<br /> ** https://scm.opendap.org/svn/|Subversion<br /> ** https://scm.opendap.org/trac/|Trac<br /> ** Document Templates|Document Templates<br /> ** helppage|help</div>

PatrickWest https://docs.opendap.org/index.php?title=MediaWiki:Sidebar&diff=9396 MediaWiki:Sidebar 2012-09-17T18:05:52Z

<p>PatrickWest: </p> <hr /> <div>* navigation<br /> ** mainpage|Home<br /> ** Documentation|Documentation<br /> ** Developer Info|Developer Info<br /> ** Development|Projects<br /> ** OPULS Development|OPULS/DAP4<br /> ** Working_Groups|Working Groups<br /> ** Meetings|Meetings<br /> ** https://scm.opendap.org/svn/|Subversion<br /> ** [https://scm.opendap.org/trac/ Trac]<br /> ** Document Templates|Document Templates<br /> ** helppage|help</div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=7603 BES XML Commands 2012-03-20T21:16:56Z

<p>PatrickWest: </p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Error Response ==<br /> <br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> Where Type is one of the following:<br /> * 1. Internal Error - the error is internal to the BES Server<br /> * 2. Internal Fatal Error - error is fatal, can not continue<br /> * 3. Syntax User Error - the requester has a syntax error in request or config<br /> * 4. Forbidden Error - the requester is forbidden to see the resource<br /> * 5. Not Found Error - the resource can not be found<br /> <br /> If debugging is enabled during build then the Error object will include the file name and line number where the exception was thrown.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response XML Document:<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a [[BES_XML_Commands#BES_Error_Response|BESError]].<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a [[BES_XML_Commands#BES_Error_Response|BESError]].<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a [[BES_XML_Commands#BES_Error_Response|BESError]].<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a [[BES_XML_Commands#BES_Error_Response|BESError]].<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServiceDescriptions /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=7601 BES XML Commands 2012-03-20T21:15:48Z

<p>PatrickWest: </p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Error Response ==<br /> <br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> Where Type is one of the following:<br /> * 1. Internal Error - the error is internal to the BES Server<br /> * 2. Internal Fatal Error - error is fatal, can not continue<br /> * 3. Syntax User Error - the requester has a syntax error in request or config<br /> * 4. Forbidden Error - the requester is forbidden to see the resource<br /> * 5. Not Found Error - the resource can not be found<br /> <br /> If debugging is enabled during build then the Error object will include the file name and line number where the exception was thrown.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response XML Document:<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a [[BES_XML_Commands#BES_Error_Response|BESError]].<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServiceDescriptions /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=7599 BES XML Commands 2012-03-20T21:15:03Z

<p>PatrickWest: </p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Error Response ==<br /> <br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> Where Type is one of the following:<br /> * 1. Internal Error - the error is internal to the BES Server<br /> * 2. Internal Fatal Error - error is fatal, can not continue<br /> * 3. Syntax User Error - the requester has a syntax error in request or config<br /> * 4. Forbidden Error - the requester is forbidden to see the resource<br /> * 5. Not Found Error - the resource can not be found<br /> <br /> If debugging is enabled during build then the Error object will include the file name and line number where the exception was thrown.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response XML Document:<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServiceDescriptions /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=User:PatrickWest&diff=7473 User:PatrickWest 2012-03-05T18:36:31Z

<p>PatrickWest: </p> <hr /> <div>Patrick West<BR><br /> Senior Software Engineer, Rensselaer Polytechnic Institute, Tetherless World Constellation<BR><br /> [mailto:westp@rpi.edu email]</div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=6973 BES XML Commands 2012-01-24T20:30:07Z

<p>PatrickWest: /* Response */</p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response:<br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> <br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServiceDescriptions /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> <format name="dap2"/><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=6972 BES XML Commands 2012-01-24T20:27:48Z

<p>PatrickWest: /* showServices */</p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response:<br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> <br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServiceDescriptions /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=BES_XML_Commands&diff=6970 BES XML Commands 2012-01-20T20:18:34Z

<p>PatrickWest: </p> <hr /> <div>[[Category:Development|BES XML Commands]][[Category:Hyrax Development|BES XML Commands]]<br /> == BES XML Command syntax. ==<br /> <br /> The BES will accept commands encoded in XML documents (BES XML Commands), and provide responses in kind. Some requests specifically indicate a non XML response (such as a DAP2 binary response) in which case the response will be as requested. <br /> <br /> === Requests ===<br /> All elements mentioned in the following are in the http://xml.opendap.org/ns/bes/1.0# namespace unless otherwise noted.<br /> <br /> # Each request document '''must''' have a <request> root element.<br /> # Each <request> '''must''' contain one or more BesCommand elements. <br /> # A <request> '''may''' contain multiple BesCommands as long as zero more of those commands returns a response.<br>Examples (expand with abbreviated xml):<br /> #* we can do a set context, set container, define and a get das in the same request document as only the get das request command returns a response.<br /> #* There can not be two show commands within the request document, or a show and a get, or multiple gets. <br /> #. Each request element '''must''' have an attribute ''reqID'' the value of which will be used in the response document. There is no guarantee that the value of ''reqID'' be unique within the operational domain of the BES. (It might be unique within the software of the requesting client, but that's of no concern to the BES).<br /> <br /> Question: Do we may want to allow reqID to be an optional attribute for each individual command. If present it would (''can it?'') be preserved in the response (or error) object as appropriate.<br /> <br /> When we (eventually) develop a schema for this we can use it to "enforce" these restrictions.<br /> <br /> === Responses ===<br /> <br /> Need a description and such here.<br /> <br /> == BES Command Set ==<br /> <br /> === setContext ===<br /> <br /> Example: <br /> <setContext name="contextName>Value</setContext><br /> <br /> Changes the state of the BES for the current client connection. This allows the client to ask the BES to utilize various response formats.<br /> <br /> ====''name'' attribute====<br /> Identifies which context value is being set.<br /> <br /> '''dap_format''' context<br /> : Value:<br /> :* ''Major.Minor'' where both ''Major'' and ''Minor'' are integer values.<br /> <br /> '''errors''' context<br /> : Current Values:<br /> :* ''xml'' -<br /> :* ''dap2'' - When error context is set to ''dap2'' then all errors will returned as DAP2 error objects (definitely '''not''' XML).<br /> <br /> : Proposed Values:<br /> :* ''dap'' - When error context is set to ''dap'' then all errors will returned as DAP error objects. The version of the DAP that error must conform to is controlled by the dap_format context. It is possible (likely) that in the future DAP errors will be XML documents.<br /> :* ''bes'' - Returns a BES Error response:<br /> <BES><br /> <response reqID="####"><br /> <BESError><br /> <Type>3</Type><br /> <Message>Unable to find command for showVersions</Message><br /> <Administrator>ndp@opendap.org</Administrator><br /> </BESError><br /> </response><br /> </BES><br /> <br /> <br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="errors">dap2</setContext><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === setContainer ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> ----<br /> <br /> === define ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <define name="d" space="default"><br /> <constraint>a valid default ce</constraint><br /> <container name="c1"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <container name="c2"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError.<br /> <br /> <br /> <br /> ----<br /> <br /> === get ===<br /> <br /> '''This needs to be expanded to illuminate the missing details from the previoius command set:'''<br /> * get 'type' for 'definition' using 'URL';<br /> <br /> Type:<br /> * '''dds''' -<br /> * '''das''' -<br /> * '''dods''' - <br /> * '''stream''' -<br /> * '''ascii''' -<br /> * '''html_form''' -<br /> * '''info_page''' -<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <get type="data_product" definition="def_name" returnAs="name" url="url" /><br /> </request><br /> <br /> ===== Response Example =====<br /> <br /> Explain about DAP2 responses etc...<br /> <br /> ===Multiple command example ===<br /> Multiple command transaction resulting in a DDS (non XML DAP2) response:<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <setContext name="error">dap2</setContext><br /> <setContainer name="c" space="catalog">data/nc/fnoc1.nc</setContainer><br /> <define name="d" space="default"><br /> <container name="c"><br /> <constraint>a valid ce</constraint><br /> <attributes>list of attributes</attributes><br /> </container><br /> <aggregate handler="someHandler" cmd="someCommand" /><br /> </define> <br /> <get type="dds" definition="d" returnAs="name" /><br /> </request><br /> <br /> <br /> <br /> ----<br /> <br /> === Show version ===<br /> <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showVersion /><br /> </request><br /> <br /> ===== Response =====<br /> <br /> Current:<br /> <showVersion><br /> <response><br /> <DAP><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </DAP><br /> <BES><br /> <lib><br /> <name>libdap</name><br /> <version>3.5.3</version><br /> </lib><br /> <lib><br /> <name>bes</name><br /> <version>3.1.0</version><br /> </lib><br /> </BES><br /> <Handlers><br /> <lib><br /> <name>libnc-dods</name><br /> <version>0.9</version><br /> </lib><br /> </Handlers><br /> </response><br /> </showVersion><br /> <br /> <br /> Proposed:<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showVersion><br /> <service name="dap"><br /> <version>2.0</version><br /> <version>3.0</version><br /> <version>3.2</version><br /> </service><br /> <library name="bes">3.5.3</library><br /> <library name="libdap">3.10.0</library><br /> <module name="netcdf_handler">3.7.9</module><br /> <module name="freeform_handler">3.7.9</module><br /> </showVersion><br /> </response><br /> <br /> ----<br /> <br /> === Show help ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showHelp /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showHelp><br /> <module name="bes" version="3.6.2"><html xmlns= http://www.w3.org/1999/xhtml >Help Information</html></module><br /> <module name="dap" version="3.10.1">Help Information</module><br /> <module name="netcdf_handler" version="3.7.9">Help Information including supported responses</module><br /> </showHelp><br /> </response><br /> <br /> ----<br /> <br /> === showProcess ===<br /> <br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command. <br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showProcess /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showProcess><br /> <process pid="10831" /><br /> </showProcess><br /> </response><br /> <br /> ----<br /> <br /> === showConfig ===<br /> This is available only if the BES is compiled in developer mode. A 'production' BES does not support this command.<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showConfig /><br /> </request><br /> <br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showConfig><br /> <file>/Users/pwest/opendap/chunking/etc/bes/bes.conf</file><br /> <key name="BES.CacheDir">/tmp</key><br /> ....<br /> </showConfig><br /> </response><br /> ----<br /> <br /> === showStatus ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showStatus /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showStatus><br /> <status>MST Thu Dec 18 11:51:36 2008</status><br /> </showStatus><br /> </response><br /> <br /> ----<br /> <br /> === showContainers ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContainers /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showContainers><br /> <store name="volatile"><br /> <container name="c" type="nc">data/nc/fnoc1.nc</container><br /> </store><br /> </showContainers><br /> </response><br /> <br /> ----<br /> <br /> === deleteContainer(s), deleteDefinition(s) ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <deleteContainers store="storeName" /><br /> <deleteContainer name="containerName" store="storeName" /><br /> <deleteDefinitions store="storeName" /><br /> <deleteDefinition name="defName" store="storeName" /><br /> </request><br /> ===== Response Example =====<br /> <br /> Normally no response. May return a BESError<br /> <br /> <br /> <br /> ----<br /> <br /> === showDefinitions ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showDefinitions /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showDefinitions><br /> <store name="volatile"><br /> <definition name="d"><br /> <container name="c" type="nc" constraint="">data/nc/fnoc1.nc</container><br /> <aggregation handler="agg">aggregation_command</aggregation><br /> </definition><br /> </store><br /> </showDefinitions><br /> </response><br /> <br /> ----<br /> <br /> === showContext ===<br /> ===== Request Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID ="####" ><br /> <showContext /><br /> </request><br /> ===== Response Example =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID ="####" ><br /> <showContext><br /> <context name="name1">value1</context><br /> <context name="name2">value2</context><br /> ...<br /> <context name="namen">valuen</context><br /> <showContext><br /> </response><br /> <br /> ----<br /> <br /> === showCatalog ===<br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showCatalog node="[catalog:]nodeName" /><br /> </request><br /> <br /> The catalog name is optional, defaulting to the default catalog specified in the BES configuration file. So if you had a catalog named rdh you could specify node="rdh:/" and it would give you the root node for the rdh catalog.<br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####" ><br /> <showCatalog><br /> <dataset name="nc/test" size="408" lastModified="2006-01-04T19:48:24" catalog="catalog" node="true" count="5"><br /> <dataset name="test.nc" size="12148" lastModified="2005-09-29T16:31:28" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="testfile.nc" size="43392" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPat.nc" size="262452" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatDbl.nc" size="2097464" lastModified="2005-09-29T16:31:28" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> <dataset name="TestPatFlt.nc" size="1048884" lastModified="2005-09-29T16:31:27" catalog="catalog" node="false"><br /> <service>DAP</service><br /> </dataset><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ----<br /> <br /> === showInfo ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showInfo node="nodeName /><br /> </request><br /> <br /> ===== Current Response =====<br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="true"><br /> <name>nc/test</name><br /> <size>408</size><br /> <lastmodified><br /> <date>2006-01-04</date><br /> <time>19:48:24</time><br /> </lastmodified><br /> <count>5</count><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showInfo><br /> <dataset thredds_container="false"><br /> <name>nc/test/TestPatFlt.nc</name><br /> <size>1048884</size><br /> <lastmodified><br /> <date>2005-09-29</date><br /> <time>16:31:27</time><br /> </lastmodified><br /> </dataset><br /> </showInfo><br /> </response><br /> <br /> ===== Proposed Response =====<br /> <dataset name="testfile.nc" size="43392" lastModified="YYYY-MM-DDThh:mm:ss" catalog="catalog" <br /> node="true|false" count="#ofChildDatSets"><br /> <service>DAP</service><br /> </dataset><br /> <br /> === showServices ===<br /> <br /> ===== Request =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <request reqID="####" ><br /> <showServices /><br /> </request><br /> <br /> ===== Response =====<br /> <?xml version="1.0" encoding="UTF-8"?><br /> <response reqID="####"><br /> <showServices><br /> <service name="DAP"><br /> <command name="ddx"><br /> <description>Words For Humans</description><br /> </command><br /> <command name="dds"><br /> <description>Words For Humans</description><br /> </command><br /> </service><br /> </showServices><br /> </response></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_BES_Build_Instructions&diff=6496 Hyrax - BES Build Instructions 2011-06-09T19:56:32Z

<p>PatrickWest: </p> <hr /> <div>How to build the BES from either a source code distribution, or directly from our [http://subversion.tigris.org/ Subversion] repository. These instructions are for the latest release of the BES, which is version 3.9.0 used for Hyrax 1.6.2.<br /> <br /> -----<br /> =Dependancies=<br /> <br /> ==[http://directory.fsf.org/gcc.html GCC, the GNU compiler]==<br /> <br /> '''''Minimum''''': gcc 4.x<br /> <br /> '''''Recommended''''': gcc 4.2.1<br /> <br /> This software was developed and tested using gcc 4.0.1 on Intel OS-X. The default compiler was used, version 4.0.1. You can also obtain the compiler using such tools as [http://darwinports.opendarwin.org/ DarwinPorts] on OS-X, package managers on Unix-like machines.<br /> <br /> ==[http://directory.fsf.org/autoconf.html autoconf]==<br /> <br /> '''''Minimum''''': autoconf 2.61<br /> <br /> '''''Recommended''''': autoconf 2.65<br /> <br /> This software was developed and tested using autoconf 2.63 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/automake.html automake]==<br /> <br /> '''''Minimum''''': automake 1.10<br /> <br /> '''''Recommended''''': automake 1.11.1<br /> <br /> This software was developed and tested using automake 1.11 on Intel OS-X. The reason for requiring 1.10 is that there is an issue with building some of the configuration and tests for the BES.<br /> <br /> ==[http://directory.fsf.org/libtool.html libtool]==<br /> <br /> '''''Minimum''''': libtool 1.5.18<br /> <br /> '''''Recommended''''': libtool 2.2.6<br /> <br /> This software was developed and tested using libtool 2.2.6 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/readline.html readline]==<br /> <br /> This library allows users to edit command lines as they are being typed in. This is used by the BES command line application bescmdln in the interactive mode (the default mode).<br /> <br /> '''''Minimum''''': readline 4.3<br /> <br /> '''''Recommended''''': readline 4.3<br /> <br /> This software was developed and tested using readline 4.3 on Intel OS-X.<br /> <br /> ==[http://xmlsoft.org/ libxml2]==<br /> <br /> Libxml2 is the XML C parser and toolkit developed for the Gnome project (but usable outside of the Gnome platform), it is free software available under the MIT License. This is used by the BES to parse incoming request documents and to create xml response documents.<br /> <br /> '''''Minimum''''': readline 2.6.16<br /> <br /> '''''Recommended''''': readline 2.7.3<br /> <br /> This software was developed and tested using libxml2 2.7.3 on Intel OS-X.<br /> <br /> ==[http://opendap.org/download/hyrax.html libdap]==<br /> <br /> The BES does not specifically require the libdap libraries. But, to get Dap 2 responses you must have libdap. With libdap you get the DAS, DDS, DataDDS, DDX responses as well as catalog support for THREDDS.<br /> <br /> '''''Minimum''''': libdap 3.11.0<br /> <br /> '''''Recommended''''': libdap 3.11.0<br /> <br /> This software was developed and tested using libdap 3.9.2 on Intel OS-X.<br /> <br /> =Optional Dependencies=<br /> <br /> ==[http://www.openssl.org/ openssl]==<br /> <br /> Openssl is used for a secure server. If you specify BES.ServerSecure=yes (default is no) in the [[Hyrax - BES Configuration|BES Configuration]] file then a client will be required to authenticate with the BES.<br /> <br /> '''''Minimum''''': openssl 0.9.7<br /> <br /> '''''Recommended''''': openssl 0.9.7<br /> <br /> This software was developed and tested using openssl 0.9.7i on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://www.gnu.org/software/dejagnu/ dejagnu]==<br /> <br /> In order to run many of the tests in the BES you will need dejagnu. We are in the process of converting our tests to use autotest, which will remove this dependency. If you wish to run 'make check', then you will need dejagnu.<br /> <br /> '''''Minimum''''': dejagnu 1.4.4<br /> <br /> '''''Recommended''''': dejagnu 1.4.4<br /> <br /> This BES software was tested using dejagnu 1.4.4 on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://sourceforge.net/projects/cppunit/ cppunit]==<br /> <br /> In order to run many of the tests in the BES you will need cppunit. If you wish to run all of the tests using 'make check', then you will need cppunit, although it is not required.<br /> <br /> '''''Minimum''''': cppunit 1.12.1<br /> <br /> '''''Recommended''''': cppunit 1.12.1<br /> <br /> This BES software was tested using cppunit 1.12.1 on Intel OS-X.<br /> <br /> ----<br /> <br /> =Getting The Source Code=<br /> <br /> ==As A Source Code Distribution==<br /> The most recent source code distribution will available at both the [http://opendap.org/download/hyrax.html Hyrax download page] and the [http://opendap.org/download/BES.html BES download page]. If you are attempting to build the BES in conjunction with the OLFS - for example you are building Hyrax - then get both the source distributions from the same Hyrax release on the [http://opendap.org/download/hyrax.html Hyrax download page], other wise you may experience compatibility issues. <br /> <br /> Go get the source code bundle. It will be a file named something like bes-''x.x.x''.tar.gz<br /> <br /> Unpack the bundle using the command:<br /> <br /> tar zxvf bes-''x.x.x''.tar.gz<br /> <br /> This will unpack the source code distribution into a directory called ''bes.x.x.x''<br /> <br /> <br /> ==From Subversion==<br /> * Check out the software for the BES from our [http://subversion.tigris.org/ Subversion] repository located at http://scm.opendap.org/svn/trunk/bes:<br /> svn co http://scm.opendap.org/svn/trunk/bes<br /> <br /> <br /> ----<br /> <br /> =Building the code=<br /> Building the BES software should be very simple.<br /> <br /> # Enter the top level directory of the bes source code tree (''bes'' if you got the source code from Subversion, ''bes-.x.x.x'' if you got it from a distribution bundle.)<br /><br /><br /> # If you checked out the latest from the Subversion repository, run the command:<br /><br /><code>autoreconf --force</code><br /><br /> This will create the configuration scripts<br /><br /><br /> # Run the command:<br /><br /><code>./configure</code><br /><br /> This will run the configuration script to make sure that you have all the required dependencies met and will also generate Makefiles and other configuration files needed to build the BES and to run the BES.<br />You may want to change some of the configuration options. Please refer to the README and INSTALL files in the top level directory for configuration options. You can also run <br /><br /><code>./configure --help</code><br /><br /> to get more options. The most commonly used option is the --prefix=<directory> option, which tells the build to install the bes into the specified directory. The default directory is /usr/local.<br /><br /><br /> # Run the command:<br /><br /><code>make</code><br /><br /> This will build the BES code<br /><br /><br /> # Run the command:<br /><br /><code>make check</code><br /><br /> This will test to make sure that you have a successful build of the BES.<br /><br /><br /> <br /> ----<br /> <br /> =Installing the BES from source code=<br /> Run the command:<br /><br /><code>make install</code><br /><br /> This will install the BES into /usr/local if no --prefix was specified on the configure line, or in the specified directory if you did use the --prefix option.<br /><br /><br /> <br /> Make sure that the BES bin directory is in your PATH. For example, if you installed the BES in the default location, the directory /usr/local/bin should be on your path. If you installed it in a non-standard location, then you will need to update your PATH environment variable. For example, if you installed the bes into /usr/local/bes, then make sure that /usr/local/bes/bin is on your PATH.<br /> <br /> ----<br /> <br /> =Testing the install=<br /> Try running the following to make sure that the install was successful.<br /> <br /> bes-config --version<br /> <br /> You should see the version of the bes that you just installed.<br /> <br /> Now, let's try to run the BES. Try typing the following on the command line:<br /> <br /> besctl start<br /> <br /> You should see something like this:<br /> <br /> BES install directory: /usr/local<br /> BES configuration file: /usr/local/etc/bes/bes.conf<br /> Starting the BES daemon<br /> Unix socket name: /tmp/opendap.socket<br /> PID: 10491 UID: 6779<br /> <br /> And then you should be able to run the bescmdln application. After running it you should see a prompt. Type in the command <code>show version;</code>, and then type in the command <code>exit</code>.<br /> <br /> <pre><br /> bescmdln -h localhost -p 10002<br /> <br /> <br /> Type 'exit' to exit the command line client and 'help' or '?' to display the help screen<br /> <br /> BESClient&gt; show version;<br /> &lt;?xml version="1.0" encoding="UTF-8"?&gt;<br /> &lt;showVersion&gt;<br /> &lt;response&gt;<br /> &lt;BES&gt;<br /> &lt;lib&gt;<br /> &lt;name&gt;bes&lt;/name&gt;<br /> &lt;version&gt;3.4.1&lt;/version&gt;<br /> &lt;/lib&gt;<br /> &lt;/BES&gt;<br /> &lt;Handlers&gt;<br /> &lt;DAP&gt;<br /> &lt;version&gt;2.0&lt;/version&gt;<br /> &lt;version&gt;3.0&lt;/version&gt;<br /> &lt;version&gt;3.2&lt;/version&gt;<br /> &lt;/DAP&gt;<br /> &lt;/Handlers&gt;<br /> &lt;/response&gt;<br /> &lt;/showVersion&gt;<br /> BESClient&gt; exit<br /> </pre></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6345 Hyrax Catalogs 2011-04-21T14:14:40Z

<p>PatrickWest: /* Questions */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> * Might want to move the catalog stuff into the bes_dispatch library instead of the bes_dap library<br /> * In the setContainer request, look at the value of the element (the node) and see if it starts with a catalog name (if store/space="catalog")<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> ** [<font color="red">JG: Can you remind me what the JGOFS directory holds - the catalog is the collection of served data sets, yes?</font>]<br /> ** This is just an example. Change the name from jgofs to foobar. What do I do if there's a catalog names foobar and a directory in the root of the default catalog named foobar? In the response to a showCatalog, do I include them both? I say yes.<br /> ** As for your jgofs question, directories are shown and can be traversed, just like the default catalog. But no files are listed. The jgofs objects are listed in the .remoteobjects file, if there is one in that directory.<br /> * <font color="red">JG: If we do this, will the SQL handler work? Almost? (and we can change the color or these questions once you all see them... I don't wan to make it all ugly.</font><br /> ** I'm afraid I have not yet had time to look at the SQL handler, so am not sure.<br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the collection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> <br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6344 Hyrax Catalogs 2011-04-21T14:14:02Z

<p>PatrickWest: /* Questions */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> * Might want to move the catalog stuff into the bes_dispatch library instead of the bes_dap library<br /> * In the setContainer request, look at the value of the element (the node) and see if it starts with a catalog name (if store/space="catalog")<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> ** [<font color="red">JG: Can you remind me what the JGOFS directory holds - the catalog is the collection of served data sets, yes?</font>]<br /> ** This is just an example. Change the name from jgofs to foobar. What do I do if there's a catalog names foobar and a directory in the root of the default catalog named foobar? In the response to a showCatalog, do I include them both? I say yes.<br /> ** As for your jgofs question, directories are shown and can be traversed, just like the default catalog. But no files are listed. The jgofs objects are listed in the .remoteobjects file, if there is one in that directory.<br /> * <font color="red">JG: If we do this, will the SQL handler work? Almost? (and we can change the color or these questions once you all see them... I don't wan to make it all ugly.</font><br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the collection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> <br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6340 Hyrax Catalogs 2011-04-20T23:44:24Z

<p>PatrickWest: /* What does the Patrick do differently */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> * Might want to move the catalog stuff into the bes_dispatch library instead of the bes_dap library<br /> * In the setContainer request, look at the value of the element (the node) and see if it starts with a catalog name (if store/space="catalog")<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6339 Hyrax Catalogs 2011-04-20T23:43:49Z

<p>PatrickWest: /* What does the BES do differently */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> * Might want to move the catalog stuff into the bes_dispatch library instead of the bes_dap library<br /> * Might want to remove the non-catalog version of BES.RootDirectory (etc...) and just use the catalog stuff<br /> * In the setContainer request, look at the value of the element (the node) and see if it starts with a catalog name (if store/space="catalog")<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6338 Hyrax Catalogs 2011-04-20T23:42:56Z

<p>PatrickWest: /* What does the Patrick do differently */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used (unless we drop that out of the BES as well)<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> * Might want to move the catalog stuff into the bes_dispatch library instead of the bes_dap library<br /> * Might want to remove the non-catalog version of BES.RootDirectory (etc...) and just use the catalog stuff<br /> * In the setContainer request, look at the value of the element (the node) and see if it starts with a catalog name (if store/space="catalog")<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6337 Hyrax Catalogs 2011-04-20T23:40:25Z

<p>PatrickWest: /* What does the BES do differently */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> * in a setContainer request, store/space="catalog" still needs to be used (unless we drop that out of the BES as well)<br /> ** the value of the setContainer element needs to also be parsed to see if there is a catalog name at the front<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6336 Hyrax Catalogs 2011-04-20T23:37:39Z

<p>PatrickWest: /* What does the Patrick do differently */</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> ** remove this attribute<br /> * Somewhere (perhaps in the XML request parsing) grab the node attribute in showCatalog, grab the first part (if there is one), and see if a catalog by that name exists<br /> ** also need to see if it's something in the root of the default catalog (SUCKS)<br /> * When showing the root, instead of displaying the names of the catalogs, display the root of the default catalog, and the names of the other catalogs<br /> ** if there's a duplicate between the root of the default and a catalog name, leave it in. Make the admins change this<br /> * When requesting a node, if the node is in the root of the default catalog AND there's a catalog with that name, then log an error message using BESLog<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6335 Hyrax Catalogs 2011-04-20T23:33:43Z

<p>PatrickWest: </p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, each possibly representing a different repository or collection of data. For example, one catalog could be the jgofs repository, another catalog a basic directory listing of data files (default catalog does this), another could represent the CEDAR Catalog of data (based on a relational database).<br /> <br /> Unfortunately when the OLFS was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> <br /> If the OLFS had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Solution ==<br /> The following is the agreed upon solution by James Gallagher, Nathan Potter, and Patrick West on April 20, 2011.<br /> <br /> The key is to maintain backward compatibility.<br /> <br /> So let's say we have three catalogs.<br /> <br /> # The default catalog named '''catalog'''<br /> # A jgofs catalog named '''jgofs'''<br /> # A netcdf catalog named '''nc'''<br /> <br /> Under the default catalog (which say points to $prefix/share/hyrax) has a single subdirectory called '''data'''.<br /> <br /> The request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> <br /> Would have the result:<br /> <br /> <?xml version="1.0" encoding="ISO-8859-1"?><br /> <response reqID="some_unique_value" xmlns="http://xml.opendap.org/ns/bes/1.0#"><br /> <showCatalog><br /> <dataset count="3" name="/" node="true"><br /> <dataset count="6" lastModified="2010-05-13T04:35:17" name="data" node="true" size="102"/><br /> <dataset count="2" lastModified="2011-04-18T18:42:44" name="jgofs" node="true" size="136"/><br /> <dataset count="9" lastModified="2011-04-20T21:10:51" name="nc" node="true" size="374"/><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> Points to bring out:<br /> * The default catalog root is returned as part of the result. The data directory has 6 elements under it (let's say), it is a node (has sub directories)<br /> * The other two catalogs (jgofs and nc) are both listed here as part of the root of the default catalog<br /> <br /> A subsequent request to show the data node would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="data"/><br /> </bes:request><br /> <br /> Whereas a request to show the root node of the jgofs catalog would look like:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> ===So what about name clashes===<br /> The default catalog always wins. So, if the root of the default catalog, in this case, had a directory called jgofs, then the request:<br /> <br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="jgofs"/><br /> </bes:request><br /> <br /> would return the contents of the jgofs directory in the default catalog, not the root node of the jgofs catalog.<br /> <br /> ===What does the OLFS do differently===<br /> * Nothing<br /> <br /> ===What does the BES do differently===<br /> * Drop the node="catalog:dir" notation and use node="catalog/dir" or, in the case of the default catalog node="dir"<br /> * If the first part of the node (before a slash, or whole thing if no slash) matches the name of a catalog, then use the catalog<br /> * Unless where the name matches an element in the default catalog root, at which point the default catalog wins<br /> <br /> ===What does the Patrick do differently===<br /> * Currently, in doing ncml and jgofs work, modified the request to include a catalog="" attribute in a showCatalog request<br /> <br /> ===Questions===<br /> * Should the request for the root node include both the jgofs directory and the jgofs catalog? Would be confusing to user, so would say no. But might be a good thing if users send messages to the data admins to point this out.<br /> <br /> <br /> == Old Notes with Comments ==<br /> <br /> === Possible Solutions ===<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> === Patrick seez ===<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ===Comments===<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Fixing_the_(currently)_broken_Hyrax_catalogs&diff=6334 Fixing the (currently) broken Hyrax catalogs 2011-04-20T23:06:45Z

<p>PatrickWest: Fixing the (currently) broken Hyrax catalogs moved to Hyrax Catalogs: James said so</p> <hr /> <div>#REDIRECT [[Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6333 Hyrax Catalogs 2011-04-20T23:06:45Z

<p>PatrickWest: Fixing the (currently) broken Hyrax catalogs moved to Hyrax Catalogs: James said so</p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, all of which have default names that can be remapped if needed through the BES configuration.(<font color="red">Patrick: Is that true? If not it should be.</font> - yes, this is true. Could be called something different in the configuration file)<br /> <br /> Unfortunately when Hyrax was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> : <font color="red">Patrick: Is that true? How do you "do" a bes:showCatalog for a particular catalog? It's not documented here: [[BES XML Commands]]</font> - [[User:ndp]]<br /> : I changed the documentation at the BES XML Commands page and modified the request document above for showing the root node of the catalog named "catalog". This needs to change. Should be able to say <bes:showCatalog catalog="catname" node="/" /> - [[User:pwest]]<br /> <br /> If Hyrax had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Possible Solutions ==<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> == Patrick seez ==<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ==Comments==<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6323 Hyrax Catalogs 2011-04-17T06:00:22Z

<p>PatrickWest: </p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, all of which have default names that can be remapped if needed through the BES configuration.(<font color="red">Patrick: Is that true? If not it should be.</font> - yes, this is true. Could be called something different in the configuration file)<br /> <br /> Unfortunately when Hyrax was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> : <font color="red">Patrick: Is that true? How do you "do" a bes:showCatalog for a particular catalog? It's not documented here: [[BES XML Commands]]</font> - [[User:ndp]]<br /> : I changed the documentation at the BES XML Commands page and modified the request document above for showing the root node of the catalog named "catalog". This needs to change. Should be able to say <bes:showCatalog catalog="catname" node="/" /> - [[User:pwest]]<br /> <br /> If Hyrax had been written to support the implicit catalog organization in the BES then the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Possible Solutions ==<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> == Patrick seez ==<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I think that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ==Comments==<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6322 Hyrax Catalogs 2011-04-17T05:39:02Z

<p>PatrickWest: </p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, all of which have default names that can be remapped if needed through the BES configuration.(<font color="red">Patrick: Is that true? If not it should be.</font> - yes, this is true. Could be called something different in the configuration file)<br /> <br /> Unfortunately when Hyrax was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog/><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL has been reduced to a BES resource ID of ''/''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/" /><br /> </bes:request><br /> : <font color="red">Patrick: Is that true? How do you "do" a bes:showCatalog for a particular catalog? It's not documented here: [[BES XML Commands]]</font> - [[User:ndp]]<br /> : I changed the documentation at the BES XML Commands page and modified the request document above for showing the root node of the catalog named "catalog". This needs to change. Should be able to say <bes:showCatalog catalog="catname" node="/" /> - [[User:pwest]]<br /> <br /> If Hyrax had been written to support the implicit catalog organization in the BES the the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Possible Solutions ==<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> == Patrick seez ==<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I thing that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ==Comments==<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_Catalogs&diff=6321 Hyrax Catalogs 2011-04-17T05:28:53Z

<p>PatrickWest: </p> <hr /> <div>== Hyrax/BES catalog integration issues ==<br /> <br /> Up until now Hyrax has been relying on only the default catalog in the BES. The default catalog is named '''catalog''' (Unfortunately for those of us reading and writing about it the word is both a proper noun and a regular noun in this context - confusing to say the least) The BES supports any number of other catalogs, all of which have default names that can be remapped if needed through the BES configuration.(<font color="red">Patrick: Is that true? If not it should be.</font> - yes, this is true. Could be called something different in the configuration file)<br /> <br /> Unfortunately when Hyrax was developed it was written to rely on only the default BES catalog. This manifests in the Hyrax URLs as follows:<br /> <br /> The catalog URL ''<nowiki>http://localhost:8080/opendap/data/nc</nowiki>'' refers to the top level in the default BES catalog. Accessing this URL issues the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/data/nc" /><br /> </bes:request><br /> Notice that:<br /> * The bes:showCatalog element does not specify a particular catalog, thus it implicitly specifies the default catalog '''catalog'''.<br /> * The catalog URL ahs been reduced to a BES resource ID of ''/data/nc''<br /> <br /> Which is equivalent to the BES command:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="catalog:/data/nc" /><br /> </bes:request><br /> : <font color="red">Patrick: Is that true? How do you "do" a bes:showCatalog for a particular catalog? It's not documented here: [[BES XML Commands]]</font> - [[User:ndp]]<br /> : I changed the documentation at the BES XML Commands page and modified the request document above for showing the root node of the catalog named "catalog". This needs to change. Should be able to say <bes:showCatalog catalog="catname" node="/" /> - [[User:pwest]]<br /> <br /> If Hyrax had been written to support the implicit catalog organization in the BES the the correct catalog URL would have been:<br /> ''<nowiki>http://localhost:8080/opendap/catalog/data/nc</nowiki>''<br /> <br /> Changing this presents backward compatibility issues for Hyrax in that it is important that Hyrax not change the access URL's to existing data sets, catalogs, or inventories.<br /> <br /> == Possible Solutions ==<br /> In order to acheive this compatibility the following schemes have been proposed:<br /> <br /> ==== OLFS DispatchHandler for each catalog ====<br /> <br /> For each catalog that gets added to the BES a new OLFS DispatchHandler is written. This really would entail making a copy of all the classes in the java package ''opendap.bes'' and modifying them slightly to allow for the different catalog names. It's a crummy solution because it grows the code base in a crazy way - lot's of replicate functionality that becomes a real problem to maintain as changes to the BES API will have to be implemented in multiple places in the OLFS code base. This, I predict won't happen and things will constantly broken. <br /> <br /> It also means that every time someone writes a BES module that provides a complete DAP services stack and a new catalog that Hyrax will be unable to take advantage of it without having a java program written to support it. That's a bloody waste of resources.<br /> <br /> ==== BES interprets resource ID's ====<br /> <br /> In lieu of specifying the catalog name explicitly in the various BES requests, the BES will look at the first part of the resource ID and compare it to its list of catalogs. If the first part of the resource ID matches a catalog name then the BES will pass the request to that catalog. Otherwise it will pass the request to the default catalog.<br /> <br /> ===== Keep existing URLs working =====<br /> <br /> For example, in the catalog URL ''<nowiki> http://localhost:8080/opendap/data/nc</nowiki>''<br /> Would produce a resource ID of ''/data/nc'' Unless the BES actually contains a catalog named ''data'' then a catalog request for the resource ID ''/data/nc'' will be routed to the default catalog. <br /> <br /> ===== Make the top level catalog include all of the catalogs =====<br /> <br /> When a request is made for the default top-level catalog:<br /> <bes:request xmlns:bes="http://xml.opendap.org/ns/bes/1.0#" reqID="[http-8080-1:16:bes_request]"><br /> <bes:setContext name="errors">xml</bes:setContext><br /> <bes:showCatalog node="/" /><br /> </bes:request><br /> The BES will return a catalog containing an merging of the catalogs by name (as catalog ''nodes'') and a listing of the top-level of the default catalog. For example if the BES has the ''rdh'', ''cedar'', and ''jgofs'' catalogs installed, and the default catalog contains data sets, regular files, and nodes the response might look like this:<br /> <br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset <font color=red>catalog="catalog" (What would this be in this example?)</font> count="13" lastModified="2009-03-24T19:02:46" name="/" node="true" size="578"><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> <br /> <dataset catalog="catalog" lastModified="2008-11-19T20:56:15" name="Test" node="true" size="918" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:34" name="Trunk Monkey" node="true" size="238" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:31:56" name="bears.nc" node="false" size="852"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2008-09-11T16:27:59" name="coverage" node="true" size="102" /><br /> <dataset catalog="catalog" lastModified="2009-01-30T20:10:16" name="data" node="true" size="510" /><br /> <dataset catalog="catalog" lastModified="2009-03-25T17:58:54" name="fanfang" node="true" size="306" /><br /> <dataset catalog="catalog" lastModified="2009-03-24T19:01:22" name="fanfang.tar.gz" node="false" size="9055684" /><br /> <dataset catalog="catalog" lastModified="2008-12-05T19:23:30" name="namespaceAttributesTest.nc" node="false" size="3396"><br /> <serviceRef>dap</serviceRef><br /> </dataset><br /> <dataset catalog="catalog" lastModified="2009-02-12T21:25:11" name="netcdf" node="true" size="136" /><br /> <dataset catalog="catalog" lastModified="2008-09-17T21:45:19" name="robots.txt" node="false" size="25" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="sst" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-12-30T21:47:58" name="thredds" node="true" size="68" /><br /> <dataset catalog="catalog" lastModified="2008-05-29T23:32:04" name="wcs" node="true" size="68" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> If there is a conflict between catalog name(s) and holdings in the default catalog '''catalog''' the offending catalog name(s) can be remapped using the BES configuration.<br /> <br /> '''GAH!'''<br /> <br /> '''I still think that this is totally busted and I don't see how to fix it with out simply rewriting the OLFS to correctly work with the different catalogs in the BES which WILL change all of the URL's. The fact is that we (well ok, I) didn't get it right. And this is a pretty intractable problem otherwise. Fixing it in one of these half-baked ways just begs for the whole thing to come crashing down around us, not to mention the burden of creating an even MORE COMPLEX install and configuration procedure for our users. Maybe one of you can see a way out of this box... [[User:Ndp|ndp]] 13:35, 7 May 2009 (PDT)'''<br /> <br /> <br /> == Patrick seez ==<br /> <br /> ==== In the BES ====<br /> <br /> instead of looking for the catalog as part of the node name, look for an attribute of the showCatalog element called catalog. If not present and ( the node attribute is not present or is empty or is slash ) then I return the list of catalogs.<br /> <br /> If not present and the node is present and the node is not empty and the node is not slash then I default the catalog name to the default catalog specified in the BES configuration file.<br /> <br /> If present then I use that catalog name.<br /> <br /> In the response for <showCatalog /> I give you the list of the catalogs where the default catalog is marked with default="true".<br /> <br /> Or perhaps there's a new command <showCatalogs /> that gives the list of catalogs?<br /> <br /> ==== In the OLFS ====<br /> <br /> When the OLFS starts up make a <showCatalog /> request to get the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/ then you show them the list of catalogs.<br /> <br /> When someone browses to www.example.com/opendap/catalogname then you show them the root node of that catalog<br /> <br /> When someone browses to www.example.com/opendap/data/nc/ (no catalog name is present) then this is an old URL, you pass this off to the BES without a catalog name, and the BES will use the default catalog.<br /> <br /> When someone browses to www.example.com/opendap/data/nc/fnoc1.nc.das (no catalog name is present) then, again this is an old URL. Given the list of catalogs in the beginning you use the default catalog name.<br /> <br /> How can you tell if the catalog name is not present ... when the OLFS started up you asked for the list of catalogs. If data is not the name of a catalog then you know we are using the default catalog.<br /> <br /> Or ... if possible ... redirect the old URLs to one with the catalog name. Not sure if you can do this. So, if someone browses to www.example.com/opendap/data/nc then you redirect that to www.example.com/opendap/catalog/data/nc, and the same for www.example.com/opendap/data/nc/fnoc1.nc.das you redirect to www.example.com/opendap/catalog/data/nc/fnoc1.nc<br /> <br /> What do you think?<br /> <br /> I don't like the idea of when asked for the root node, if no catalog is specified then I give you the root for the default catalog and the names of the other catalogs. I thing that if you ask for no catalog and no node then you should get the list of catalogs, including the default.<br /> <br /> <showCatalog /><br /> <response xmlns="http://xml.opendap.org/ns/bes/1.0#" reqID="some_unique_value"><br /> <showCatalog><br /> <dataset catalog="root" count="4" lastModified="2009-03-24T19:02:46" name="/" node="true" size="-1"><br /> <dataset catalog="catalog" lastModified="2003-09-11T16:27:59" name="catalog" node="true" size="-1" default="true" /><br /> <dataset catalog="rdh" lastModified="2003-09-11T16:27:59" name="rdh" node="true" size="-1" /><br /> <dataset catalog="jgofs" lastModified="2001-05-29T23:32:04" name="jgofs" node="true" size="-1" /><br /> <dataset catalog="cedar" lastModified="2017 -12-30T21:47:58" name="cedar" node="true" size="-1" /><br /> </dataset><br /> </showCatalog><br /> </response><br /> <br /> ==Comments==<br /> <font color="red">Why does this matter? Do we need to use multiple catalogs for something? [[User:Jimg|jimg]] 10:43, 12 August 2009 (PDT)<br /> <br /> :Yes. As we grow Hyrax we will need to support additional catalogs and services. Because both the BES and the OLFS can independently (and as part of their regular usage pattern) modify the catalog by adding either services which are connected to a URL or catalog content that is mapped into the URL space. <br /> <br /> :I guess one way to think of this is that the catalog for a particular instance of Hyrax is the colection of all URLs that are valid. (Where valid means that they return an HTTP status of 200.) Can't we consider the collection of URL stings to represent a tree? Doesn't it define a collection of named nodes? Some which are leaves and some which contain more stuff?<br /> :--[[User:Ndp|ndp]] 09:21, 7 March 2010 (PST)<br /> </font><br /> [[User:pwest | Patrick]]<br /> <br /> [[Category:Development|Fixing the Broken Hyrax Catalogs]][[Category:Hyrax Development|Fixing the Broken Hyrax Catalogs]]</div>

PatrickWest https://docs.opendap.org/index.php?title=NCML_Service&diff=6301 NCML Service 2011-03-08T18:28:59Z

<p>PatrickWest: </p> <hr /> <div>Service allows for the reading of THREDDS catalogs, and ones that contain embedded NCML elements, passing that element to the BES for storage using the NCML id from the THREDDS catalog as a dataset name.<br /> <br /> [http://tw.rpi.edu/web/project/ESG-CET/workinggroups/aggregation RPI ESG Aggregation documentation]</div>

PatrickWest https://docs.opendap.org/index.php?title=NCML_Service&diff=6299 NCML Service 2011-03-08T17:07:35Z

<p>PatrickWest: New page: Service allows for the reading of THREDDS catalogs, and ones that contain embedded NCML elements, passing that element to the BES for storage using the NCML id from the THREDDS catalog as ...</p> <hr /> <div>Service allows for the reading of THREDDS catalogs, and ones that contain embedded NCML elements, passing that element to the BES for storage using the NCML id from the THREDDS catalog as a dataset name.</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_BES_Build_Instructions&diff=6256 Hyrax - BES Build Instructions 2011-02-03T18:54:02Z

<p>PatrickWest: /* From Subversion */</p> <hr /> <div>How to build the BES from either a source code distribution, or directly from our [http://subversion.tigris.org/ Subversion] repository. These instructions are for the latest release of the BES, which is version 3.9.0 used for Hyrax 1.6.2.<br /> <br /> -----<br /> =Dependancies=<br /> <br /> ==[http://directory.fsf.org/gcc.html GCC, the GNU compiler]==<br /> <br /> '''''Minimum''''': gcc 4.x<br /> <br /> '''''Recommended''''': gcc 4.2.1<br /> <br /> This software was developed and tested using gcc 4.0.1 on Intel OS-X. The default compiler was used, version 4.0.1. You can also obtain the compiler using such tools as [http://darwinports.opendarwin.org/ DarwinPorts] on OS-X, package managers on Unix-like machines.<br /> <br /> ==[http://directory.fsf.org/autoconf.html autoconf]==<br /> <br /> '''''Minimum''''': autoconf 2.61<br /> <br /> '''''Recommended''''': autoconf 2.65<br /> <br /> This software was developed and tested using autoconf 2.63 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/automake.html automake]==<br /> <br /> '''''Minimum''''': automake 1.10<br /> <br /> '''''Recommended''''': automake 1.11.1<br /> <br /> This software was developed and tested using automake 1.11 on Intel OS-X. The reason for requiring 1.10 is that there is an issue with building some of the configuration and tests for the BES.<br /> <br /> ==[http://directory.fsf.org/libtool.html libtool]==<br /> <br /> '''''Minimum''''': libtool 1.5.18<br /> <br /> '''''Recommended''''': libtool 2.2.6<br /> <br /> This software was developed and tested using libtool 2.2.6 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/readline.html readline]==<br /> <br /> This library allows users to edit command lines as they are being typed in. This is used by the BES command line application bescmdln in the interactive mode (the default mode).<br /> <br /> '''''Minimum''''': readline 4.3<br /> <br /> '''''Recommended''''': readline 4.3<br /> <br /> This software was developed and tested using readline 4.3 on Intel OS-X.<br /> <br /> ==[http://opendap.org/download/hyrax.html libdap]==<br /> <br /> The BES does not specifically require the libdap libraries. But, to get Dap 2 responses you must have libdap. With libdap you get the DAS, DDS, DataDDS, DDX responses as well as catalog support for THREDDS.<br /> <br /> '''''Minimum''''': libdap 3.11.0<br /> <br /> '''''Recommended''''': libdap 3.11.0<br /> <br /> This software was developed and tested using libdap 3.9.2 on Intel OS-X.<br /> <br /> =Optional Dependencies=<br /> <br /> ==[http://www.openssl.org/ openssl]==<br /> <br /> Openssl is used for a secure server. If you specify BES.ServerSecure=yes (default is no) in the [[Hyrax - BES Configuration|BES Configuration]] file then a client will be required to authenticate with the BES.<br /> <br /> '''''Minimum''''': openssl 0.9.7<br /> <br /> '''''Recommended''''': openssl 0.9.7<br /> <br /> This software was developed and tested using openssl 0.9.7i on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://www.gnu.org/software/dejagnu/ dejagnu]==<br /> <br /> In order to run many of the tests in the BES you will need dejagnu. We are in the process of converting our tests to use autotest, which will remove this dependency. If you wish to run 'make check', then you will need dejagnu.<br /> <br /> '''''Minimum''''': dejagnu 1.4.4<br /> <br /> '''''Recommended''''': dejagnu 1.4.4<br /> <br /> This BES software was tested using dejagnu 1.4.4 on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://sourceforge.net/projects/cppunit/ cppunit]==<br /> <br /> In order to run many of the tests in the BES you will need cppunit. If you wish to run all of the tests using 'make check', then you will need cppunit, although it is not required.<br /> <br /> '''''Minimum''''': cppunit 1.12.1<br /> <br /> '''''Recommended''''': cppunit 1.12.1<br /> <br /> This BES software was tested using cppunit 1.12.1 on Intel OS-X.<br /> <br /> ----<br /> <br /> =Getting The Source Code=<br /> <br /> ==As A Source Code Distribution==<br /> The most recent source code distribution will available at both the [http://opendap.org/download/hyrax.html Hyrax download page] and the [http://opendap.org/download/BES.html BES download page]. If you are attempting to build the BES in conjunction with the OLFS - for example you are building Hyrax - then get both the source distributions from the same Hyrax release on the [http://opendap.org/download/hyrax.html Hyrax download page], other wise you may experience compatibility issues. <br /> <br /> Go get the source code bundle. It will be a file named something like bes-''x.x.x''.tar.gz<br /> <br /> Unpack the bundle using the command:<br /> <br /> tar zxvf bes-''x.x.x''.tar.gz<br /> <br /> This will unpack the source code distribution into a directory called ''bes.x.x.x''<br /> <br /> <br /> ==From Subversion==<br /> * Check out the software for the BES from our [http://subversion.tigris.org/ Subversion] repository located at http://scm.opendap.org/svn/trunk/bes:<br /> svn co http://scm.opendap.org/svn/trunk/bes<br /> <br /> <br /> ----<br /> <br /> =Building the code=<br /> Building the BES software should be very simple.<br /> <br /> # Enter the top level directory of the bes source code tree (''bes'' if you got the source code from Subversion, ''bes-.x.x.x'' if you got it from a distribution bundle.)<br /><br /><br /> # If you checked out the latest from the Subversion repository, run the command:<br /><br /><code>autoreconf --force</code><br /><br /> This will create the configuration scripts<br /><br /><br /> # Run the command:<br /><br /><code>./configure</code><br /><br /> This will run the configuration script to make sure that you have all the required dependencies met and will also generate Makefiles and other configuration files needed to build the BES and to run the BES.<br />You may want to change some of the configuration options. Please refer to the README and INSTALL files in the top level directory for configuration options. You can also run <br /><br /><code>./configure --help</code><br /><br /> to get more options. The most commonly used option is the --prefix=<directory> option, which tells the build to install the bes into the specified directory. The default directory is /usr/local.<br /><br /><br /> # Run the command:<br /><br /><code>make</code><br /><br /> This will build the BES code<br /><br /><br /> # Run the command:<br /><br /><code>make check</code><br /><br /> This will test to make sure that you have a successful build of the BES.<br /><br /><br /> <br /> ----<br /> <br /> =Installing the BES from source code=<br /> Run the command:<br /><br /><code>make install</code><br /><br /> This will install the BES into /usr/local if no --prefix was specified on the configure line, or in the specified directory if you did use the --prefix option.<br /><br /><br /> <br /> Make sure that the BES bin directory is in your PATH. For example, if you installed the BES in the default location, the directory /usr/local/bin should be on your path. If you installed it in a non-standard location, then you will need to update your PATH environment variable. For example, if you installed the bes into /usr/local/bes, then make sure that /usr/local/bes/bin is on your PATH.<br /> <br /> ----<br /> <br /> =Testing the install=<br /> Try running the following to make sure that the install was successful.<br /> <br /> bes-config --version<br /> <br /> You should see the version of the bes that you just installed.<br /> <br /> Now, let's try to run the BES. Try typing the following on the command line:<br /> <br /> besctl start<br /> <br /> You should see something like this:<br /> <br /> BES install directory: /usr/local<br /> BES configuration file: /usr/local/etc/bes/bes.conf<br /> Starting the BES daemon<br /> Unix socket name: /tmp/opendap.socket<br /> PID: 10491 UID: 6779<br /> <br /> And then you should be able to run the bescmdln application. After running it you should see a prompt. Type in the command <code>show version;</code>, and then type in the command <code>exit</code>.<br /> <br /> <pre><br /> bescmdln -h localhost -p 10002<br /> <br /> <br /> Type 'exit' to exit the command line client and 'help' or '?' to display the help screen<br /> <br /> BESClient&gt; show version;<br /> &lt;?xml version="1.0" encoding="UTF-8"?&gt;<br /> &lt;showVersion&gt;<br /> &lt;response&gt;<br /> &lt;BES&gt;<br /> &lt;lib&gt;<br /> &lt;name&gt;bes&lt;/name&gt;<br /> &lt;version&gt;3.4.1&lt;/version&gt;<br /> &lt;/lib&gt;<br /> &lt;/BES&gt;<br /> &lt;Handlers&gt;<br /> &lt;DAP&gt;<br /> &lt;version&gt;2.0&lt;/version&gt;<br /> &lt;version&gt;3.0&lt;/version&gt;<br /> &lt;version&gt;3.2&lt;/version&gt;<br /> &lt;/DAP&gt;<br /> &lt;/Handlers&gt;<br /> &lt;/response&gt;<br /> &lt;/showVersion&gt;<br /> BESClient&gt; exit<br /> </pre></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_BES_Build_Instructions&diff=6255 Hyrax - BES Build Instructions 2011-02-03T18:53:01Z

<p>PatrickWest: </p> <hr /> <div>How to build the BES from either a source code distribution, or directly from our [http://subversion.tigris.org/ Subversion] repository. These instructions are for the latest release of the BES, which is version 3.9.0 used for Hyrax 1.6.2.<br /> <br /> -----<br /> =Dependancies=<br /> <br /> ==[http://directory.fsf.org/gcc.html GCC, the GNU compiler]==<br /> <br /> '''''Minimum''''': gcc 4.x<br /> <br /> '''''Recommended''''': gcc 4.2.1<br /> <br /> This software was developed and tested using gcc 4.0.1 on Intel OS-X. The default compiler was used, version 4.0.1. You can also obtain the compiler using such tools as [http://darwinports.opendarwin.org/ DarwinPorts] on OS-X, package managers on Unix-like machines.<br /> <br /> ==[http://directory.fsf.org/autoconf.html autoconf]==<br /> <br /> '''''Minimum''''': autoconf 2.61<br /> <br /> '''''Recommended''''': autoconf 2.65<br /> <br /> This software was developed and tested using autoconf 2.63 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/automake.html automake]==<br /> <br /> '''''Minimum''''': automake 1.10<br /> <br /> '''''Recommended''''': automake 1.11.1<br /> <br /> This software was developed and tested using automake 1.11 on Intel OS-X. The reason for requiring 1.10 is that there is an issue with building some of the configuration and tests for the BES.<br /> <br /> ==[http://directory.fsf.org/libtool.html libtool]==<br /> <br /> '''''Minimum''''': libtool 1.5.18<br /> <br /> '''''Recommended''''': libtool 2.2.6<br /> <br /> This software was developed and tested using libtool 2.2.6 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/readline.html readline]==<br /> <br /> This library allows users to edit command lines as they are being typed in. This is used by the BES command line application bescmdln in the interactive mode (the default mode).<br /> <br /> '''''Minimum''''': readline 4.3<br /> <br /> '''''Recommended''''': readline 4.3<br /> <br /> This software was developed and tested using readline 4.3 on Intel OS-X.<br /> <br /> ==[http://opendap.org/download/hyrax.html libdap]==<br /> <br /> The BES does not specifically require the libdap libraries. But, to get Dap 2 responses you must have libdap. With libdap you get the DAS, DDS, DataDDS, DDX responses as well as catalog support for THREDDS.<br /> <br /> '''''Minimum''''': libdap 3.11.0<br /> <br /> '''''Recommended''''': libdap 3.11.0<br /> <br /> This software was developed and tested using libdap 3.9.2 on Intel OS-X.<br /> <br /> =Optional Dependencies=<br /> <br /> ==[http://www.openssl.org/ openssl]==<br /> <br /> Openssl is used for a secure server. If you specify BES.ServerSecure=yes (default is no) in the [[Hyrax - BES Configuration|BES Configuration]] file then a client will be required to authenticate with the BES.<br /> <br /> '''''Minimum''''': openssl 0.9.7<br /> <br /> '''''Recommended''''': openssl 0.9.7<br /> <br /> This software was developed and tested using openssl 0.9.7i on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://www.gnu.org/software/dejagnu/ dejagnu]==<br /> <br /> In order to run many of the tests in the BES you will need dejagnu. We are in the process of converting our tests to use autotest, which will remove this dependency. If you wish to run 'make check', then you will need dejagnu.<br /> <br /> '''''Minimum''''': dejagnu 1.4.4<br /> <br /> '''''Recommended''''': dejagnu 1.4.4<br /> <br /> This BES software was tested using dejagnu 1.4.4 on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://sourceforge.net/projects/cppunit/ cppunit]==<br /> <br /> In order to run many of the tests in the BES you will need cppunit. If you wish to run all of the tests using 'make check', then you will need cppunit, although it is not required.<br /> <br /> '''''Minimum''''': cppunit 1.12.1<br /> <br /> '''''Recommended''''': cppunit 1.12.1<br /> <br /> This BES software was tested using cppunit 1.12.1 on Intel OS-X.<br /> <br /> ----<br /> <br /> =Getting The Source Code=<br /> <br /> ==As A Source Code Distribution==<br /> The most recent source code distribution will available at both the [http://opendap.org/download/hyrax.html Hyrax download page] and the [http://opendap.org/download/BES.html BES download page]. If you are attempting to build the BES in conjunction with the OLFS - for example you are building Hyrax - then get both the source distributions from the same Hyrax release on the [http://opendap.org/download/hyrax.html Hyrax download page], other wise you may experience compatibility issues. <br /> <br /> Go get the source code bundle. It will be a file named something like bes-''x.x.x''.tar.gz<br /> <br /> Unpack the bundle using the command:<br /> <br /> tar zxvf bes-''x.x.x''.tar.gz<br /> <br /> This will unpack the source code distribution into a directory called ''bes.x.x.x''<br /> <br /> <br /> ==From Subversion==<br /> * Check out the software for the BES from our [http://subversion.tigris.org/ Subversion] repository located at http://scm.opendap.org:8090/svn/trunk/bes:<br /> svn co http://scm.opendap.org:8090/svn/trunk/bes<br /> <br /> <br /> ----<br /> <br /> =Building the code=<br /> Building the BES software should be very simple.<br /> <br /> # Enter the top level directory of the bes source code tree (''bes'' if you got the source code from Subversion, ''bes-.x.x.x'' if you got it from a distribution bundle.)<br /><br /><br /> # If you checked out the latest from the Subversion repository, run the command:<br /><br /><code>autoreconf --force</code><br /><br /> This will create the configuration scripts<br /><br /><br /> # Run the command:<br /><br /><code>./configure</code><br /><br /> This will run the configuration script to make sure that you have all the required dependencies met and will also generate Makefiles and other configuration files needed to build the BES and to run the BES.<br />You may want to change some of the configuration options. Please refer to the README and INSTALL files in the top level directory for configuration options. You can also run <br /><br /><code>./configure --help</code><br /><br /> to get more options. The most commonly used option is the --prefix=<directory> option, which tells the build to install the bes into the specified directory. The default directory is /usr/local.<br /><br /><br /> # Run the command:<br /><br /><code>make</code><br /><br /> This will build the BES code<br /><br /><br /> # Run the command:<br /><br /><code>make check</code><br /><br /> This will test to make sure that you have a successful build of the BES.<br /><br /><br /> <br /> ----<br /> <br /> =Installing the BES from source code=<br /> Run the command:<br /><br /><code>make install</code><br /><br /> This will install the BES into /usr/local if no --prefix was specified on the configure line, or in the specified directory if you did use the --prefix option.<br /><br /><br /> <br /> Make sure that the BES bin directory is in your PATH. For example, if you installed the BES in the default location, the directory /usr/local/bin should be on your path. If you installed it in a non-standard location, then you will need to update your PATH environment variable. For example, if you installed the bes into /usr/local/bes, then make sure that /usr/local/bes/bin is on your PATH.<br /> <br /> ----<br /> <br /> =Testing the install=<br /> Try running the following to make sure that the install was successful.<br /> <br /> bes-config --version<br /> <br /> You should see the version of the bes that you just installed.<br /> <br /> Now, let's try to run the BES. Try typing the following on the command line:<br /> <br /> besctl start<br /> <br /> You should see something like this:<br /> <br /> BES install directory: /usr/local<br /> BES configuration file: /usr/local/etc/bes/bes.conf<br /> Starting the BES daemon<br /> Unix socket name: /tmp/opendap.socket<br /> PID: 10491 UID: 6779<br /> <br /> And then you should be able to run the bescmdln application. After running it you should see a prompt. Type in the command <code>show version;</code>, and then type in the command <code>exit</code>.<br /> <br /> <pre><br /> bescmdln -h localhost -p 10002<br /> <br /> <br /> Type 'exit' to exit the command line client and 'help' or '?' to display the help screen<br /> <br /> BESClient&gt; show version;<br /> &lt;?xml version="1.0" encoding="UTF-8"?&gt;<br /> &lt;showVersion&gt;<br /> &lt;response&gt;<br /> &lt;BES&gt;<br /> &lt;lib&gt;<br /> &lt;name&gt;bes&lt;/name&gt;<br /> &lt;version&gt;3.4.1&lt;/version&gt;<br /> &lt;/lib&gt;<br /> &lt;/BES&gt;<br /> &lt;Handlers&gt;<br /> &lt;DAP&gt;<br /> &lt;version&gt;2.0&lt;/version&gt;<br /> &lt;version&gt;3.0&lt;/version&gt;<br /> &lt;version&gt;3.2&lt;/version&gt;<br /> &lt;/DAP&gt;<br /> &lt;/Handlers&gt;<br /> &lt;/response&gt;<br /> &lt;/showVersion&gt;<br /> BESClient&gt; exit<br /> </pre></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_BES_Build_Instructions&diff=6254 Hyrax - BES Build Instructions 2011-02-03T18:52:18Z

<p>PatrickWest: </p> <hr /> <div>How to build the BES from either a source code distribution, or directly from our [http://subversion.tigris.org/ Subversion] repository. These instructions are for the latest release of the BES, which is version 3.7.2 used for Hyrax 1.5.2.<br /> <br /> -----<br /> =Dependancies=<br /> <br /> ==[http://directory.fsf.org/gcc.html GCC, the GNU compiler]==<br /> <br /> '''''Minimum''''': gcc 4.x<br /> <br /> '''''Recommended''''': gcc 4.2.1<br /> <br /> This software was developed and tested using gcc 4.0.1 on Intel OS-X. The default compiler was used, version 4.0.1. You can also obtain the compiler using such tools as [http://darwinports.opendarwin.org/ DarwinPorts] on OS-X, package managers on Unix-like machines.<br /> <br /> ==[http://directory.fsf.org/autoconf.html autoconf]==<br /> <br /> '''''Minimum''''': autoconf 2.61<br /> <br /> '''''Recommended''''': autoconf 2.65<br /> <br /> This software was developed and tested using autoconf 2.63 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/automake.html automake]==<br /> <br /> '''''Minimum''''': automake 1.10<br /> <br /> '''''Recommended''''': automake 1.11.1<br /> <br /> This software was developed and tested using automake 1.11 on Intel OS-X. The reason for requiring 1.10 is that there is an issue with building some of the configuration and tests for the BES.<br /> <br /> ==[http://directory.fsf.org/libtool.html libtool]==<br /> <br /> '''''Minimum''''': libtool 1.5.18<br /> <br /> '''''Recommended''''': libtool 2.2.6<br /> <br /> This software was developed and tested using libtool 2.2.6 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/readline.html readline]==<br /> <br /> This library allows users to edit command lines as they are being typed in. This is used by the BES command line application bescmdln in the interactive mode (the default mode).<br /> <br /> '''''Minimum''''': readline 4.3<br /> <br /> '''''Recommended''''': readline 4.3<br /> <br /> This software was developed and tested using readline 4.3 on Intel OS-X.<br /> <br /> ==[http://opendap.org/download/hyrax.html libdap]==<br /> <br /> The BES does not specifically require the libdap libraries. But, to get Dap 2 responses you must have libdap. With libdap you get the DAS, DDS, DataDDS, DDX responses as well as catalog support for THREDDS.<br /> <br /> '''''Minimum''''': libdap 3.11.0<br /> <br /> '''''Recommended''''': libdap 3.11.0<br /> <br /> This software was developed and tested using libdap 3.9.2 on Intel OS-X.<br /> <br /> =Optional Dependencies=<br /> <br /> ==[http://www.openssl.org/ openssl]==<br /> <br /> Openssl is used for a secure server. If you specify BES.ServerSecure=yes (default is no) in the [[Hyrax - BES Configuration|BES Configuration]] file then a client will be required to authenticate with the BES.<br /> <br /> '''''Minimum''''': openssl 0.9.7<br /> <br /> '''''Recommended''''': openssl 0.9.7<br /> <br /> This software was developed and tested using openssl 0.9.7i on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://www.gnu.org/software/dejagnu/ dejagnu]==<br /> <br /> In order to run many of the tests in the BES you will need dejagnu. We are in the process of converting our tests to use autotest, which will remove this dependency. If you wish to run 'make check', then you will need dejagnu.<br /> <br /> '''''Minimum''''': dejagnu 1.4.4<br /> <br /> '''''Recommended''''': dejagnu 1.4.4<br /> <br /> This BES software was tested using dejagnu 1.4.4 on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://sourceforge.net/projects/cppunit/ cppunit]==<br /> <br /> In order to run many of the tests in the BES you will need cppunit. If you wish to run all of the tests using 'make check', then you will need cppunit, although it is not required.<br /> <br /> '''''Minimum''''': cppunit 1.12.1<br /> <br /> '''''Recommended''''': cppunit 1.12.1<br /> <br /> This BES software was tested using cppunit 1.12.1 on Intel OS-X.<br /> <br /> ----<br /> <br /> =Getting The Source Code=<br /> <br /> ==As A Source Code Distribution==<br /> The most recent source code distribution will available at both the [http://opendap.org/download/hyrax.html Hyrax download page] and the [http://opendap.org/download/BES.html BES download page]. If you are attempting to build the BES in conjunction with the OLFS - for example you are building Hyrax - then get both the source distributions from the same Hyrax release on the [http://opendap.org/download/hyrax.html Hyrax download page], other wise you may experience compatibility issues. <br /> <br /> Go get the source code bundle. It will be a file named something like bes-''x.x.x''.tar.gz<br /> <br /> Unpack the bundle using the command:<br /> <br /> tar zxvf bes-''x.x.x''.tar.gz<br /> <br /> This will unpack the source code distribution into a directory called ''bes.x.x.x''<br /> <br /> <br /> ==From Subversion==<br /> * Check out the software for the BES from our [http://subversion.tigris.org/ Subversion] repository located at http://scm.opendap.org:8090/svn/trunk/bes:<br /> svn co http://scm.opendap.org:8090/svn/trunk/bes<br /> <br /> <br /> ----<br /> <br /> =Building the code=<br /> Building the BES software should be very simple.<br /> <br /> # Enter the top level directory of the bes source code tree (''bes'' if you got the source code from Subversion, ''bes-.x.x.x'' if you got it from a distribution bundle.)<br /><br /><br /> # If you checked out the latest from the Subversion repository, run the command:<br /><br /><code>autoreconf --force</code><br /><br /> This will create the configuration scripts<br /><br /><br /> # Run the command:<br /><br /><code>./configure</code><br /><br /> This will run the configuration script to make sure that you have all the required dependencies met and will also generate Makefiles and other configuration files needed to build the BES and to run the BES.<br />You may want to change some of the configuration options. Please refer to the README and INSTALL files in the top level directory for configuration options. You can also run <br /><br /><code>./configure --help</code><br /><br /> to get more options. The most commonly used option is the --prefix=<directory> option, which tells the build to install the bes into the specified directory. The default directory is /usr/local.<br /><br /><br /> # Run the command:<br /><br /><code>make</code><br /><br /> This will build the BES code<br /><br /><br /> # Run the command:<br /><br /><code>make check</code><br /><br /> This will test to make sure that you have a successful build of the BES.<br /><br /><br /> <br /> ----<br /> <br /> =Installing the BES from source code=<br /> Run the command:<br /><br /><code>make install</code><br /><br /> This will install the BES into /usr/local if no --prefix was specified on the configure line, or in the specified directory if you did use the --prefix option.<br /><br /><br /> <br /> Make sure that the BES bin directory is in your PATH. For example, if you installed the BES in the default location, the directory /usr/local/bin should be on your path. If you installed it in a non-standard location, then you will need to update your PATH environment variable. For example, if you installed the bes into /usr/local/bes, then make sure that /usr/local/bes/bin is on your PATH.<br /> <br /> ----<br /> <br /> =Testing the install=<br /> Try running the following to make sure that the install was successful.<br /> <br /> bes-config --version<br /> <br /> You should see the version of the bes that you just installed.<br /> <br /> Now, let's try to run the BES. Try typing the following on the command line:<br /> <br /> besctl start<br /> <br /> You should see something like this:<br /> <br /> BES install directory: /usr/local<br /> BES configuration file: /usr/local/etc/bes/bes.conf<br /> Starting the BES daemon<br /> Unix socket name: /tmp/opendap.socket<br /> PID: 10491 UID: 6779<br /> <br /> And then you should be able to run the bescmdln application. After running it you should see a prompt. Type in the command <code>show version;</code>, and then type in the command <code>exit</code>.<br /> <br /> <pre><br /> bescmdln -h localhost -p 10002<br /> <br /> <br /> Type 'exit' to exit the command line client and 'help' or '?' to display the help screen<br /> <br /> BESClient&gt; show version;<br /> &lt;?xml version="1.0" encoding="UTF-8"?&gt;<br /> &lt;showVersion&gt;<br /> &lt;response&gt;<br /> &lt;BES&gt;<br /> &lt;lib&gt;<br /> &lt;name&gt;bes&lt;/name&gt;<br /> &lt;version&gt;3.4.1&lt;/version&gt;<br /> &lt;/lib&gt;<br /> &lt;/BES&gt;<br /> &lt;Handlers&gt;<br /> &lt;DAP&gt;<br /> &lt;version&gt;2.0&lt;/version&gt;<br /> &lt;version&gt;3.0&lt;/version&gt;<br /> &lt;version&gt;3.2&lt;/version&gt;<br /> &lt;/DAP&gt;<br /> &lt;/Handlers&gt;<br /> &lt;/response&gt;<br /> &lt;/showVersion&gt;<br /> BESClient&gt; exit<br /> </pre></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_BES_Build_Instructions&diff=6253 Hyrax - BES Build Instructions 2011-02-03T18:49:39Z

<p>PatrickWest: </p> <hr /> <div>How to build the BES from either a source code distribution, or directly from our [http://subversion.tigris.org/ Subversion] repository. These instructions are for the latest release of the BES, which is version 3.7.2 used for Hyrax 1.5.2.<br /> <br /> -----<br /> =Dependancies=<br /> <br /> ==[http://directory.fsf.org/gcc.html GCC, the GNU compiler]==<br /> <br /> '''''Minimum''''': gcc 3.x<br /> <br /> '''''Recommended''''': gcc 4.x<br /> <br /> This software was developed and tested using gcc 4.0.1 on Intel OS-X. The default compiler was used, version 4.0.1. You can also obtain the compiler using such tools as [http://darwinports.opendarwin.org/ DarwinPorts] on OS-X, package managers on Unix-like machines.<br /> <br /> ==[http://directory.fsf.org/autoconf.html autoconf]==<br /> <br /> '''''Minimum''''': autoconf 2.61<br /> <br /> '''''Recommended''''': autoconf 2.63<br /> <br /> This software was developed and tested using autoconf 2.63 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/automake.html automake]==<br /> <br /> '''''Minimum''''': automake 1.10<br /> <br /> '''''Recommended''''': automake 1.11<br /> <br /> This software was developed and tested using automake 1.11 on Intel OS-X. The reason for requiring 1.10 is that there is an issue with building some of the configuration and tests for the BES.<br /> <br /> ==[http://directory.fsf.org/libtool.html libtool]==<br /> <br /> '''''Minimum''''': libtool 1.5.18<br /> <br /> '''''Recommended''''': libtool 2.2.6<br /> <br /> This software was developed and tested using libtool 2.2.6 on Intel OS-X.<br /> <br /> ==[http://directory.fsf.org/readline.html readline]==<br /> <br /> This library allows users to edit command lines as they are being typed in. This is used by the BES command line application bescmdln in the interactive mode (the default mode).<br /> <br /> '''''Minimum''''': readline 4.3<br /> <br /> '''''Recommended''''': readline 4.3<br /> <br /> This software was developed and tested using readline 4.3 on Intel OS-X.<br /> <br /> ==[http://opendap.org/download/hyrax.html libdap]==<br /> <br /> The BES does not specifically require the libdap libraries. But, to get Dap 2 responses you must have libdap. With libdap you get the DAS, DDS, DataDDS, DDX responses as well as catalog support for THREDDS.<br /> <br /> '''''Minimum''''': libdap 3.9.2<br /> <br /> '''''Recommended''''': libdap 3.9.2<br /> <br /> This software was developed and tested using libdap 3.9.2 on Intel OS-X.<br /> <br /> =Optional Dependencies=<br /> <br /> ==[http://www.openssl.org/ openssl]==<br /> <br /> Openssl is used for a secure server. If you specify BES.ServerSecure=yes (default is no) in the [[Hyrax - BES Configuration|BES Configuration]] file then a client will be required to authenticate with the BES.<br /> <br /> '''''Minimum''''': openssl 0.9.7<br /> <br /> '''''Recommended''''': openssl 0.9.7<br /> <br /> This software was developed and tested using openssl 0.9.7i on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://www.gnu.org/software/dejagnu/ dejagnu]==<br /> <br /> In order to run many of the tests in the BES you will need dejagnu. We are in the process of converting our tests to use autotest, which will remove this dependency. If you wish to run 'make check', then you will need dejagnu.<br /> <br /> '''''Minimum''''': dejagnu 1.4.4<br /> <br /> '''''Recommended''''': dejagnu 1.4.4<br /> <br /> This BES software was tested using dejagnu 1.4.4 on Intel OS-X.<br /> <br /> ----<br /> <br /> ==[http://sourceforge.net/projects/cppunit/ cppunit]==<br /> <br /> In order to run many of the tests in the BES you will need cppunit. If you wish to run all of the tests using 'make check', then you will need cppunit, although it is not required.<br /> <br /> '''''Minimum''''': cppunit 1.12.1<br /> <br /> '''''Recommended''''': cppunit 1.12.1<br /> <br /> This BES software was tested using cppunit 1.12.1 on Intel OS-X.<br /> <br /> ----<br /> <br /> =Getting The Source Code=<br /> <br /> ==As A Source Code Distribution==<br /> The most recent source code distribution will available at both the [http://opendap.org/download/hyrax.html Hyrax download page] and the [http://opendap.org/download/BES.html BES download page]. If you are attempting to build the BES in conjunction with the OLFS - for example you are building Hyrax - then get both the source distributions from the same Hyrax release on the [http://opendap.org/download/hyrax.html Hyrax download page], other wise you may experience compatibility issues. <br /> <br /> Go get the source code bundle. It will be a file named something like bes-''x.x.x''.tar.gz<br /> <br /> Unpack the bundle using the command:<br /> <br /> tar zxvf bes-''x.x.x''.tar.gz<br /> <br /> This will unpack the source code distribution into a directory called ''bes.x.x.x''<br /> <br /> <br /> ==From Subversion==<br /> * Check out the software for the BES from our [http://subversion.tigris.org/ Subversion] repository located at http://scm.opendap.org:8090/svn/trunk/bes:<br /> svn co http://scm.opendap.org:8090/svn/trunk/bes<br /> <br /> <br /> ----<br /> <br /> =Building the code=<br /> Building the BES software should be very simple.<br /> <br /> # Enter the top level directory of the bes source code tree (''bes'' if you got the source code from Subversion, ''bes-.x.x.x'' if you got it from a distribution bundle.)<br /><br /><br /> # If you checked out the latest from the Subversion repository, run the command:<br /><br /><code>autoreconf --force</code><br /><br /> This will create the configuration scripts<br /><br /><br /> # Run the command:<br /><br /><code>./configure</code><br /><br /> This will run the configuration script to make sure that you have all the required dependencies met and will also generate Makefiles and other configuration files needed to build the BES and to run the BES.<br />You may want to change some of the configuration options. Please refer to the README and INSTALL files in the top level directory for configuration options. You can also run <br /><br /><code>./configure --help</code><br /><br /> to get more options. The most commonly used option is the --prefix=<directory> option, which tells the build to install the bes into the specified directory. The default directory is /usr/local.<br /><br /><br /> # Run the command:<br /><br /><code>make</code><br /><br /> This will build the BES code<br /><br /><br /> # Run the command:<br /><br /><code>make check</code><br /><br /> This will test to make sure that you have a successful build of the BES.<br /><br /><br /> <br /> ----<br /> <br /> =Installing the BES from source code=<br /> Run the command:<br /><br /><code>make install</code><br /><br /> This will install the BES into /usr/local if no --prefix was specified on the configure line, or in the specified directory if you did use the --prefix option.<br /><br /><br /> <br /> Make sure that the BES bin directory is in your PATH. For example, if you installed the BES in the default location, the directory /usr/local/bin should be on your path. If you installed it in a non-standard location, then you will need to update your PATH environment variable. For example, if you installed the bes into /usr/local/bes, then make sure that /usr/local/bes/bin is on your PATH.<br /> <br /> ----<br /> <br /> =Testing the install=<br /> Try running the following to make sure that the install was successful.<br /> <br /> bes-config --version<br /> <br /> You should see the version of the bes that you just installed.<br /> <br /> Now, let's try to run the BES. Try typing the following on the command line:<br /> <br /> besctl start<br /> <br /> You should see something like this:<br /> <br /> BES install directory: /usr/local<br /> BES configuration file: /usr/local/etc/bes/bes.conf<br /> Starting the BES daemon<br /> Unix socket name: /tmp/opendap.socket<br /> PID: 10491 UID: 6779<br /> <br /> And then you should be able to run the bescmdln application. After running it you should see a prompt. Type in the command <code>show version;</code>, and then type in the command <code>exit</code>.<br /> <br /> <pre><br /> bescmdln -h localhost -p 10002<br /> <br /> <br /> Type 'exit' to exit the command line client and 'help' or '?' to display the help screen<br /> <br /> BESClient&gt; show version;<br /> &lt;?xml version="1.0" encoding="UTF-8"?&gt;<br /> &lt;showVersion&gt;<br /> &lt;response&gt;<br /> &lt;BES&gt;<br /> &lt;lib&gt;<br /> &lt;name&gt;bes&lt;/name&gt;<br /> &lt;version&gt;3.4.1&lt;/version&gt;<br /> &lt;/lib&gt;<br /> &lt;/BES&gt;<br /> &lt;Handlers&gt;<br /> &lt;DAP&gt;<br /> &lt;version&gt;2.0&lt;/version&gt;<br /> &lt;version&gt;3.0&lt;/version&gt;<br /> &lt;version&gt;3.2&lt;/version&gt;<br /> &lt;/DAP&gt;<br /> &lt;/Handlers&gt;<br /> &lt;/response&gt;<br /> &lt;/showVersion&gt;<br /> BESClient&gt; exit<br /> </pre></div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Frequently_Asked_Questions&diff=6252 Hyrax - Frequently Asked Questions 2011-02-03T01:12:12Z

<p>PatrickWest: /* BES Fatal; cannot open log file ./bes.log. */</p> <hr /> <div>=OLFS=<br /> The OPeNDAP Lightweight Front-end Service<br /> <br /> == Status 503 "application not currently available" ==<br /> After starting up the tomcat server with the opendap.war file placed in the webapps directory, I can see the tomcat splash page if I go to <pre>http://localhost:8080/</pre>. But when I go to <pre>http://localhost:8080/opendap/</pre> I get this error.<br /> <br /> =BES=<br /> The OPeNDAP Hyrax Back-End Server<br /> <br /> ==Regular expression matches in BES==<br /> <br /> There are two regular expression matches in the BES, both configured in the BES configuration file bes.conf.<br /> <br /> * TypeMatch<br /> <br /> The TypeMatch regular expression is used to match file names with their type. For example, a file ending with the extension .hdf would match and be handled by the hdf4 module.<br /> <br /> '''''Note: The regular expression must match the file including the path to the file.'''''<br /> <br /> The TypeMatch parameter is a list of handler/module names and a regular expression separated by a colon. If the regular expression matches an item, then the BES uses the associated handler/module. Each <handler>:<regular expression> pair is followed by a semicolon. This is used when creating containers in the BES (the 'set container' command). The example regular expression says to use the 'h4' handler for any file with an extension of 'hdf', 'HDF' or 'eos' which may also end in '.gz' or '.bz2' or '.Z'. In the latter case the file will be treated as a compressed file.<br /> <br /> Example:<br /> BES.Catalog.catalog.TypeMatch=h4.*\.(hdf|HDF|eos)(\.gz|\.bz2|\.Z)?$;<br /> <br /> * Include/Exclude directive<br /> <br /> The Include/Exclude parameters in the BES configuration file tells the BES what files and directories to include/exclude in its catalog response. Normally, when a client asks for a data catalog, all files and directories are shown. Use the following two parameters to customize this behavior. Each parameter is a list of regular expressions, each followed by a semicolon (last one must also end with a semicolon as in the example below.) First, the Include parameter is applied to the node, and then the Exclude parameter is applied. All collections of nodes are shown. In the default values below, all nodes are included (the Include parameter) except those that begin with a dot (the Exclude parameter).<br /> <br /> Example:<br /> BES.Catalog.catalog.Include=;<br /> BES.Catalog.catalog.Exclude=^\..*;<br /> <br /> * There is a binary included in the distribution that will allow you to test a regular expression. The besregtest binary is run with the type of regular expression (type, include or exclude), the regular expression enclosed in quotes, and then the list of things to match. Here is the output of the usage of besregtest:<br /> <br /> Usage: besregtest include|exclude|type <regular_expression> <string_to_match><br /> samples:<br /> besregtest include "123456;" 01234567 matches 6 of 8 characters<br /> besregtest include "^123456$;" 01234567 does not match<br /> besregtest include "^123456$;" 123456 matches all 6 of 6 characters<br /> besregtest include ".*\.nc$;" fnoc1.nc matches<br /> besregtest include ".*\.nc$;" fnoc1.ncd does not matche<br /> besregtest type "nc:.*\.nc$;nc:.*\.nc\.gz$;ff:.*\.dat$;ff:.*\.dat\.gz$;" fnoc1.nc matches type nc<br /> <br /> * Examples of the TypeMatch parameter:<br /> <br /> % besregtest type "nc:.*\.nc(\.bz2|\.gz|\.Z)?$|.*/JA2_[OIG]P[NRS]_.*$;" jason2/gdr/gdr/cycle004/JA2_GPN_2PTP004_069_20080813_105815_20080813_115428<br /> expression ".*\.nc(\.bz2|\.gz|\.Z)?$|.*JA2_[OIG]P[NRS]_.*$" matches exactly, type = nc<br /> <br /> Matches anything that ends with .nc OR anything that ends with a file that begins with JA2_; then one of O, I, G; then P; then one of N, R, S;<br /> followed by an underscore and anything else.<br /> <br /> ==BES: unable to start properly because can not determine memory keys.==<br /> <br /> In release 1.4.1 of Hyrax, which includes release 3.6.0 of the BES, we fixed a typo in the BES configuration file. Unfortunately, this produces an error when you start up the BES if the configuration still contains the typo. The error message is:<br /> <br /> BES: unable to start properly because can not determine memory keys.<br /> <br /> To correct this, please change the parameter BES.Memory.GlobalArea.Maximu'''n'''HeapSize to BES.Memory.GlobalArea.Maximu'''m'''HeapSize.<br /> <br /> ==The service dap has already been registered==<br /> <br /> Starting with Hyrax 1.6.0, the BES configuration has changed. It used to be that each handler/module would modify the bes.conf file, or the user would have to modify the bes.conf file, in order to enable that module. Now, the handler/module installs a .conf file in the directory modules under <prefix>/etc/bes/modules. The bes.conf file exists in <prefix>/etc/bes, and includes any .conf files located in the modules directory.<br /> <br /> If you've installed the new BES for Hyrax 1.6, and then modify your bes.conf file like you've done in the past, this could cause problems like duplicate modules registered, duplicate services registered, etc...<br /> <br /> So, in the new bes.conf file, there is a line:<br /> <br /> BES.modules=<br /> <br /> which is empty. Leave it empty. The .conf files in the modules directory, such as nc.conf, will contain a line that looks like this:<br /> <br /> BES.modules+=nc<br /> BES.module.nc=<prefix>/lib/bes/libnc_module.so<br /> <br /> The only items that you will need to change in the bes.conf file is:<br /> <br /> BES.ServerAdministrator=<br /> BES.User=<br /> BES.Group=<br /> BES.LogName=<path_to>/bes.log<br /> BES.CacheDir=<path_to_bes_cache_directory><br /> <br /> ==I've installed the BES, but I'm not able to access any data==<br /> <br /> The BES is a software framework that allows data providers to serve specific kinds of data in specific ways. Off the shelf the BES doesn't provide any data access or any kinds of responses, not even DAP responses like DAS, DDS, ASCII, etc...<br /> <br /> In order to return DAP responses like DAS, DDS, DDX, DataDDS you have to have a BES installed that is built against libdap. When building from source you should first install, or build and install, libdap. When installing from RPM you will not be able to install without first installing the libdap RPM.<br /> <br /> In order to have an ascii response for your data, or have the html form presented, you must install the General Purpose Handlers, which can be found on the Hyrax Download Page, as can any of the data handlers, libdap, and the fileout module. Here is the official list of BES modules.<br /> <br /> In order to serve a particular type of data you must install one of the data handlers. The current officially supported data handlers are netcdf, hdf4, hdf5 and freeform. Again, these can be downloaded from Hyrax Download Page.<br /> <br /> If building from source or from SVN, just running 'make install' isn't enough. You must also either run 'make bes-conf' or run the bes-x-data.sh script for that data handler. For example, bes-nc-data.sh. Building that target using make, or running those scripts, adds information to the BES configuration file, bes.conf. This step is important to installing the different modules and handlers.<br /> <br /> ==BES Fatal; cannot open log file ./bes.log.==<br /> <br /> Although the BES is started as root, it will be running as the user you set in the BES configuration file. So be sure that that user has permission to write to the current directory where you are starting the BES.<br /> <br /> Alternatively, edit the BES configuration file bes.conf and change the location of the bes.log file.<br /> <br /> ==How do I add configuration parameters to my .conf file==<br /> <br /> Every module has its own configuration file, x.conf. For example, the netcdf data handler module has a configuration file called nc.conf. These files are located in the intall_dir/etc/bes/modules directory. So if you install in /usr/local, it will be in /usr/local/etc/bes/modules. If you are writing a module, you can add your own parameters to your configuration file.<br /> <br /> Please go to the [http://docs.opendap.org/index.php/Hyrax_-_Extending_BES_Module#Configuring_your_Module BES module page] for more information on how to do this.</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Extending_BES_Module&diff=6251 Hyrax - Extending BES Module 2011-02-02T20:47:03Z

<p>PatrickWest: /* Configuring your Module */</p> <hr /> <div>= Writing modules and handlers =<br /> <br /> == Putting it all together into a Module ==<br /> Once you have created your extensions to the BES you need to be a able to add them to the BES. This is done in the Module class. For example, in the netcdf_handler directory you will see a NCModule class. Within this Module class there is a static function that gets called when the Module is loaded.<br /> <br /> <pre><br /> extern "C"<br /> {<br /> BESAbstractModule *maker()<br /> {<br /> return new NCModule ;<br /> }<br /> }<br /> </pre><br /> <br /> This function must exist in your shared library in order for your module to be loaded into the BES. In the above example, the netcdf_handler module NCModule is created in the maker function.<br /> <br /> The Module classes have two methods:<br /> <br /> <pre><br /> void<br /> NCModule::initialize( const string &modname )<br /> {<br /> BESDEBUG( "Initializing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Initializing NC Handlers:" << endl )<br /> }<br /> </pre><br /> and<br /> <pre><br /> void<br /> NCModule::terminate( const string &modname )<br /> {<br /> BESDEBUG( "Removing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Removing NC Handlers:" << endl )<br /> }<br /> </pre><br /> <br /> As you could probably already tell, the initialize method is where you would add all of your extensions and the terminate method is where you would remove them and do any cleanup of your module. This is not to be confused with the initialization and termination callbacks described later. The callbacks are called whenever a request is made to the BES where as the methods in the Module class are called when the module is loaded and when it is unloaded from the BES respectively.<br /> <br /> Once you have your new Module class you will need to build the new module. In order to be dynamically loaded by the BES the module needs to be built as a shared object module. The following is a snippet from a Makefile.am from an existing module for netcdf.<br /> <br /> <pre><br /> lib_besdir=$(libdir)/bes<br /> lib_bes_LTLIBRARIES = libnc_module.la<br /> <br /> libnc_module_la_SOURCES = $(MODULE_SOURCES)<br /> libnc_module_la_CPPFLAGS = $(BES_CPPFLAGS)<br /> libnc_module_la_LDFLAGS = -avoid-version -module <br /> libnc_module_la_LIBADD = $(BES_DAP_LIBS)<br /> </pre><br /> <br /> The important line to take note of is for the LDFLAGS, where we spcify <code>-avoid-version -module</code>. This will build a .so file for your library that can be loaded into the BES.<br /> <br /> ==Configuring your Module==<br /> Now that you have the shared object module built you need to have it loaded into the BES. To do this you need to create/edit your configuration file. If you used the script we've provided to create a template module, find the *.conf.in file. And find this line.<br /> <br /> <pre><br /> BES.modules+=your_module_name<br /> </pre><br /> <br /> For example, using the netcdf_handler as an example again, we would have the following:<br /> <br /> <pre><br /> BES.modules+=nc<br /> </pre><br /> <br /> And then you need to tell it where to find that <code>nc</code> module. So you'll need to add the following line:<br /> <br /> <pre><br /> BES.module.nc=/full/path/to/installation/lib/bes/libnc_module.so<br /> </pre><br /> <br /> And the next time the BES is started it will know to load your module (in this example) nc. It will know to find the nc module using the full path to the shared object module libnc_module.so.<br /> <br /> If you'd like to add new key/value pairs (parameters) to your configuration file, you can do so at the end of the file. For example, in the nc module we have:<br /> <br /> <pre><br /> NC.ShowSharedDimensions=true<br /> </pre><br /> <br /> In your code, you'll look for this parameter using the following code:<br /> <br /> <pre><br /> bool found = false ;<br /> string key = "NC.ShowSharedDimensions" ;<br /> string value ;<br /> TheBESKeys::TheKeys()->get_value( key, value, found ) ;<br /> if( found )<br /> {<br /> // Do something here with the value<br /> }<br /> </pre><br /> <br /> Parameters:<br /> * key - the parameter name that you are looking for<br /> * value - the value of the parameter if found<br /> * found - whether the parameter was found in any of the configuration files<br /> <br /> You can also have multiple values for a parameter. Notice in the BES.modules parameter, we use += to add a value to this parameter. To get these values you would use the following code:<br /> <br /> <pre><br /> bool found = false ;<br /> string key = "NC.MultiValue.Parameter" ;<br /> vector<string> values ;<br /> TheBESKeys::TheKeys()->get_values( key, values, found ) ;<br /> if( found )<br /> {<br /> // Do something here with the values<br /> }<br /> </pre><br /> <br /> Parameters:<br /> * key - the parameter name that you are looking for<br /> * values - the vector of values for the parameter if found<br /> * found - whether the parameter was found in any of the configuration files<br /> <br /> Throughout this document we will tell you how to add the particular extensions to the Module class' initialization method and how to remove your extensions in the Module class' termination method.<br /> <br /> == New services ==<br /> A service is a grouping of commands that are provided by a module and can be handled by other modules, like data handlers. For example, OPeNDAP provides the <code>dap</code> service with the commands <code>das, dds, ddx, and dods</code>. These commands can be handled by the different data handlers that can serve up OPeNDAP responses for these commands. The format for each of these commands is <code>dap2</code>.<br /> <br /> [[New Services]]<br /> <br /> == New response types. ==<br /> A response to a BES request is considered here to be a response type. Whether this response type is a simple informational response, such as a response to 'show help;' or the response type is a data response such as a DAS response in the 'get das for definition;' command. New response types can be added to the BES.<br /> <br /> [[New Response Types]]<br /> <br /> == New request/data handlers ==<br /> A request handler is a handler that knows how to fill in a particular response, such as DAS, DDS, DataDDS, help, and version responses, for example.<br /> <br /> [[New Request Handlers]]<br /> <br /> == Containers and how to store them ==<br /> New ways of storing containers, called container storage. Currently we have built in volatile storage, which means storing new containers only during the duration of your connection with the BES. We also have a way to load containers from a file that can be configured in the initialization file. You could also create user specific container storage, storing container information in a MySQL database, for example.<br /> <br /> [[BES Containers|Containers, and how to store them]]<br /> <br /> == Definitions and how to store them ==<br /> New ways of storing definitions, called definition storage. Currently we have built in volatile storage, which means storing new definitions only during the duration of your connection with the BES. You could also create user specific definition storage, saving off those oft used definitions for later use.<br /> <br /> [[BES Definitions|Definitions, and how to store them]]<br /> <br /> == Reporting ==<br /> A report mechanism. Once a request has been completed, the information is passed to any reporters that are registered with the server. This way you can save user information, usage information, data access information, and more.<br /> <br /> [[BES Reporters|Keeping Track of Requests]]<br /> <br /> == Transmitting responses ==<br /> Different ways of transmitting the data. The default is to return the data through standard output, which gets redirected back to the client. But, with the 'get' command you can specify a 'return as' command ( see above ) that could save the response to a file, transmit it via email, save it in a new data file like netcdf.<br /> <br /> [[New Transmitters]]<br /> <br /> == Initialization and Terminiation Callbacks ==<br /> Add in your own initialization and termination routines. If you have routines that need to be run when the BES starts up, you can add callback functions for initialization and termination.<br /> <br /> [[BES Initialization Termination|Creating Initialization and Termination callbacks]]<br /> <br /> == Handling Exceptions ==<br /> Add in your own exception handling methods. When an exception is thrown within the BES, the exception is handled by the BES Exception Manager. It is possible to register a routine to be called before the default handling is done. This way, if you have a specific way that you want to handle specific exceptions, you can do that here.<br /> <br /> [[BES Exception Handling|Handling Exceptions within the BES]]<br /> <br /> == Aggregating your data ==<br /> <br /> By default there are no aggregation routines in the BES. But, the BES provides a mechanism for data to be aggregated.<br /> <br /> [[BES Aggregation|Providing Aggregation Routines]]<br /> <br /> == New commands ==<br /> <br /> You've seen all the default commands available in the BES. Now what if you want to create a new command, or even modify an already existing command.<br /> <br /> [[New BES Commands|Creating your own commands]]<br /> <br /> == Adding Debugging to your code ==<br /> <br /> The BES has a simple to use debugging tool. When the BES is started up you can specify -d "<output>,<context>" on the command line of the besctl script to enable debugging. The output option is either cerr, which will dump all debug statements to standard error, or a file name, which will dump all debug information to the specified file. The context option is a comma separated list of components.<br /> <br /> [[New Debugging]]<br /> <br /> == Submitting your modules ==<br /> So you've come up with a killer module that does some pretty cool stuff. Well ... share it with the rest of us.<br /> <br /> [[Sharing BES Modules|Sharing your BES Modules]]<br /> <br /> [[Category:Extending BES|Extending BES]]<br /> [[Category:BES Modules|Extending BES Module]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Extending_BES_Module&diff=6250 Hyrax - Extending BES Module 2011-02-02T18:41:45Z

<p>PatrickWest: </p> <hr /> <div>= Writing modules and handlers =<br /> <br /> == Putting it all together into a Module ==<br /> Once you have created your extensions to the BES you need to be a able to add them to the BES. This is done in the Module class. For example, in the netcdf_handler directory you will see a NCModule class. Within this Module class there is a static function that gets called when the Module is loaded.<br /> <br /> <pre><br /> extern "C"<br /> {<br /> BESAbstractModule *maker()<br /> {<br /> return new NCModule ;<br /> }<br /> }<br /> </pre><br /> <br /> This function must exist in your shared library in order for your module to be loaded into the BES. In the above example, the netcdf_handler module NCModule is created in the maker function.<br /> <br /> The Module classes have two methods:<br /> <br /> <pre><br /> void<br /> NCModule::initialize( const string &modname )<br /> {<br /> BESDEBUG( "Initializing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Initializing NC Handlers:" << endl )<br /> }<br /> </pre><br /> and<br /> <pre><br /> void<br /> NCModule::terminate( const string &modname )<br /> {<br /> BESDEBUG( "Removing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Removing NC Handlers:" << endl )<br /> }<br /> </pre><br /> <br /> As you could probably already tell, the initialize method is where you would add all of your extensions and the terminate method is where you would remove them and do any cleanup of your module. This is not to be confused with the initialization and termination callbacks described later. The callbacks are called whenever a request is made to the BES where as the methods in the Module class are called when the module is loaded and when it is unloaded from the BES respectively.<br /> <br /> Once you have your new Module class you will need to build the new module. In order to be dynamically loaded by the BES the module needs to be built as a shared object module. The following is a snippet from a Makefile.am from an existing module for netcdf.<br /> <br /> <pre><br /> lib_besdir=$(libdir)/bes<br /> lib_bes_LTLIBRARIES = libnc_module.la<br /> <br /> libnc_module_la_SOURCES = $(MODULE_SOURCES)<br /> libnc_module_la_CPPFLAGS = $(BES_CPPFLAGS)<br /> libnc_module_la_LDFLAGS = -avoid-version -module <br /> libnc_module_la_LIBADD = $(BES_DAP_LIBS)<br /> </pre><br /> <br /> The important line to take note of is for the LDFLAGS, where we spcify <code>-avoid-version -module</code>. This will build a .so file for your library that can be loaded into the BES.<br /> <br /> ==Configuring your Module==<br /> Now that you have the shared object module built you need to have it loaded into the BES. To do this you need to create/edit your configuration file. If you used the script we've provided to create a template module, find the *.conf.in file. And find this line.<br /> <br /> <pre><br /> BES.modules+=your_module_name<br /> </pre><br /> <br /> For example, using the netcdf_handler as an example again, we would have the following:<br /> <br /> <pre><br /> BES.modules+=nc<br /> </pre><br /> <br /> And then you need to tell it where to find that <code>nc</code> module. So you'll need to add the following line:<br /> <br /> <pre><br /> BES.module.nc=/full/path/to/installation/lib/bes/libnc_module.so<br /> </pre><br /> <br /> And the next time the BES is started it will know to load your module (in this example) nc. It will know to find the nc module using the full path to the shared object module libnc_module.so.<br /> <br /> If you'd like to add new key/value pairs (parameters) to your configuration file, you can do so at the end of the file. For example, in the nc module we have:<br /> <br /> <pre><br /> NC.ShowSharedDimensions=true<br /> </pre><br /> <br /> In your code, you'll look for this parameter using the following code:<br /> <br /> <pre><br /> bool found = false ;<br /> string key = "NC.ShowSharedDimensions" ;<br /> string value ;<br /> TheBESKeys::TheKeys()->get_value( key, value, found ) ;<br /> if( found )<br /> {<br /> // Do something here with the value<br /> }<br /> </pre><br /> <br /> Parameters:<br /> * key - the parameter name that you are looking for<br /> * value - the value of the parameter if found<br /> * found - whether the parameter was found in any of the configuration files<br /> <br /> Throughout this document we will tell you how to add the particular extensions to the Module class' initialization method and how to remove your extensions in the Module class' termination method.<br /> <br /> == New services ==<br /> A service is a grouping of commands that are provided by a module and can be handled by other modules, like data handlers. For example, OPeNDAP provides the <code>dap</code> service with the commands <code>das, dds, ddx, and dods</code>. These commands can be handled by the different data handlers that can serve up OPeNDAP responses for these commands. The format for each of these commands is <code>dap2</code>.<br /> <br /> [[New Services]]<br /> <br /> == New response types. ==<br /> A response to a BES request is considered here to be a response type. Whether this response type is a simple informational response, such as a response to 'show help;' or the response type is a data response such as a DAS response in the 'get das for definition;' command. New response types can be added to the BES.<br /> <br /> [[New Response Types]]<br /> <br /> == New request/data handlers ==<br /> A request handler is a handler that knows how to fill in a particular response, such as DAS, DDS, DataDDS, help, and version responses, for example.<br /> <br /> [[New Request Handlers]]<br /> <br /> == Containers and how to store them ==<br /> New ways of storing containers, called container storage. Currently we have built in volatile storage, which means storing new containers only during the duration of your connection with the BES. We also have a way to load containers from a file that can be configured in the initialization file. You could also create user specific container storage, storing container information in a MySQL database, for example.<br /> <br /> [[BES Containers|Containers, and how to store them]]<br /> <br /> == Definitions and how to store them ==<br /> New ways of storing definitions, called definition storage. Currently we have built in volatile storage, which means storing new definitions only during the duration of your connection with the BES. You could also create user specific definition storage, saving off those oft used definitions for later use.<br /> <br /> [[BES Definitions|Definitions, and how to store them]]<br /> <br /> == Reporting ==<br /> A report mechanism. Once a request has been completed, the information is passed to any reporters that are registered with the server. This way you can save user information, usage information, data access information, and more.<br /> <br /> [[BES Reporters|Keeping Track of Requests]]<br /> <br /> == Transmitting responses ==<br /> Different ways of transmitting the data. The default is to return the data through standard output, which gets redirected back to the client. But, with the 'get' command you can specify a 'return as' command ( see above ) that could save the response to a file, transmit it via email, save it in a new data file like netcdf.<br /> <br /> [[New Transmitters]]<br /> <br /> == Initialization and Terminiation Callbacks ==<br /> Add in your own initialization and termination routines. If you have routines that need to be run when the BES starts up, you can add callback functions for initialization and termination.<br /> <br /> [[BES Initialization Termination|Creating Initialization and Termination callbacks]]<br /> <br /> == Handling Exceptions ==<br /> Add in your own exception handling methods. When an exception is thrown within the BES, the exception is handled by the BES Exception Manager. It is possible to register a routine to be called before the default handling is done. This way, if you have a specific way that you want to handle specific exceptions, you can do that here.<br /> <br /> [[BES Exception Handling|Handling Exceptions within the BES]]<br /> <br /> == Aggregating your data ==<br /> <br /> By default there are no aggregation routines in the BES. But, the BES provides a mechanism for data to be aggregated.<br /> <br /> [[BES Aggregation|Providing Aggregation Routines]]<br /> <br /> == New commands ==<br /> <br /> You've seen all the default commands available in the BES. Now what if you want to create a new command, or even modify an already existing command.<br /> <br /> [[New BES Commands|Creating your own commands]]<br /> <br /> == Adding Debugging to your code ==<br /> <br /> The BES has a simple to use debugging tool. When the BES is started up you can specify -d "<output>,<context>" on the command line of the besctl script to enable debugging. The output option is either cerr, which will dump all debug statements to standard error, or a file name, which will dump all debug information to the specified file. The context option is a comma separated list of components.<br /> <br /> [[New Debugging]]<br /> <br /> == Submitting your modules ==<br /> So you've come up with a killer module that does some pretty cool stuff. Well ... share it with the rest of us.<br /> <br /> [[Sharing BES Modules|Sharing your BES Modules]]<br /> <br /> [[Category:Extending BES|Extending BES]]<br /> [[Category:BES Modules|Extending BES Module]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Extending_BES_Module&diff=6249 Hyrax - Extending BES Module 2011-02-02T18:39:02Z

<p>PatrickWest: </p> <hr /> <div>= Writing modules and handlers =<br /> <br /> == Putting it all together into a Module ==<br /> Once you have created your extensions to the BES you need to be a able to add them to the BES. This is done in the Module class. For example, in the netcdf_handler directory you will see a NCModule class. Within this Module class there is a static function that gets called when the Module is loaded.<br /> <br /> <pre><br /> extern "C"<br /> {<br /> BESAbstractModule *maker()<br /> {<br /> return new NCModule ;<br /> }<br /> }<br /> </pre><br /> <br /> This function must exist in your shared library in order for your module to be loaded into the BES. In the above example, the netcdf_handler module NCModule is created in the maker function.<br /> <br /> The Module classes have two methods:<br /> <br /> <pre><br /> void<br /> NCModule::initialize( const string &modname )<br /> {<br /> BESDEBUG( "Initializing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Initializing NC Handlers:" << endl )<br /> }<br /> </pre><br /> and<br /> <pre><br /> void<br /> NCModule::terminate( const string &modname )<br /> {<br /> BESDEBUG( "Removing NC Handlers:" << endl )<br /> <br /> // Your code here<br /> <br /> BESDEBUG( "Done Removing NC Handlers:" << endl )<br /> }<br /> </pre><br /> <br /> As you could probably already tell, the initialize method is where you would add all of your extensions and the terminate method is where you would remove them and do any cleanup of your module. This is not to be confused with the initialization and termination callbacks described later. The callbacks are called whenever a request is made to the BES where as the methods in the Module class are called when the module is loaded and when it is unloaded from the BES respectively.<br /> <br /> Once you have your new Module class you will need to build the new module. In order to be dynamically loaded by the BES the module needs to be built as a shared object module. The following is a snippet from a Makefile.am from an existing module for netcdf.<br /> <br /> <pre><br /> lib_besdir=$(libdir)/bes<br /> lib_bes_LTLIBRARIES = libnc_module.la<br /> <br /> libnc_module_la_SOURCES = $(MODULE_SOURCES)<br /> libnc_module_la_CPPFLAGS = $(BES_CPPFLAGS)<br /> libnc_module_la_LDFLAGS = -avoid-version -module <br /> libnc_module_la_LIBADD = $(BES_DAP_LIBS)<br /> </pre><br /> <br /> The important line to take note of is for the LDFLAGS, where we spcify <code>-avoid-version -module</code>. This will build a .so file for your library that can be loaded into the BES.<br /> <br /> ===Configuring your Module===<br /> Now that you have the shared object module built you need to have it loaded into the BES. To do this you need to create/edit your configuration file. If you used the script we've provided to create a template module, find the *.conf.in file. And find this line.<br /> <br /> <pre><br /> BES.modules+=your_module_name<br /> </pre><br /> <br /> For example, using the netcdf_handler as an example again, we would have the following:<br /> <br /> <pre><br /> BES.modules+=nc<br /> </pre><br /> <br /> And then you need to tell it where to find that <code>nc</code> module. So you'll need to add the following line:<br /> <br /> <pre><br /> BES.module.nc=/full/path/to/installation/lib/bes/libnc_module.so<br /> </pre><br /> <br /> And the next time the BES is started it will know to load your module (in this example) nc. It will know to find the nc module using the full path to the shared object module libnc_module.so.<br /> <br /> If you'd like to add new key/value pairs (parameters) to your configuration file, you can do so at the end of the file. For example, in the nc module we have:<br /> <br /> <pre><br /> NC.ShowSharedDimensions=true<br /> </pre><br /> <br /> In your code, you'll look for this parameter using the following code:<br /> <br /> <pre><br /> bool found = false ;<br /> string key = "NC.ShowSharedDimensions" ;<br /> string value ;<br /> TheBESKeys::TheKeys()->get_value( key, value, found ) ;<br /> if( found )<br /> {<br /> // Do something here with the value<br /> }<br /> </pre><br /> <br /> Parameters:<br /> * key - the parameter name that you are looking for<br /> * value - the value of the parameter if found<br /> * found - whether the parameter was found in any of the configuration files<br /> <br /> Throughout this document we will tell you how to add the particular extensions to the Module class' initialization method and how to remove your extensions in the Module class' termination method.<br /> <br /> == New services ==<br /> A service is a grouping of commands that are provided by a module and can be handled by other modules, like data handlers. For example, OPeNDAP provides the <code>dap</code> service with the commands <code>das, dds, ddx, and dods</code>. These commands can be handled by the different data handlers that can serve up OPeNDAP responses for these commands. The format for each of these commands is <code>dap2</code>.<br /> <br /> [[New Services]]<br /> <br /> == New response types. ==<br /> A response to a BES request is considered here to be a response type. Whether this response type is a simple informational response, such as a response to 'show help;' or the response type is a data response such as a DAS response in the 'get das for definition;' command. New response types can be added to the BES.<br /> <br /> [[New Response Types]]<br /> <br /> == New request/data handlers ==<br /> A request handler is a handler that knows how to fill in a particular response, such as DAS, DDS, DataDDS, help, and version responses, for example.<br /> <br /> [[New Request Handlers]]<br /> <br /> == Containers and how to store them ==<br /> New ways of storing containers, called container storage. Currently we have built in volatile storage, which means storing new containers only during the duration of your connection with the BES. We also have a way to load containers from a file that can be configured in the initialization file. You could also create user specific container storage, storing container information in a MySQL database, for example.<br /> <br /> [[BES Containers|Containers, and how to store them]]<br /> <br /> == Definitions and how to store them ==<br /> New ways of storing definitions, called definition storage. Currently we have built in volatile storage, which means storing new definitions only during the duration of your connection with the BES. You could also create user specific definition storage, saving off those oft used definitions for later use.<br /> <br /> [[BES Definitions|Definitions, and how to store them]]<br /> <br /> == Reporting ==<br /> A report mechanism. Once a request has been completed, the information is passed to any reporters that are registered with the server. This way you can save user information, usage information, data access information, and more.<br /> <br /> [[BES Reporters|Keeping Track of Requests]]<br /> <br /> == Transmitting responses ==<br /> Different ways of transmitting the data. The default is to return the data through standard output, which gets redirected back to the client. But, with the 'get' command you can specify a 'return as' command ( see above ) that could save the response to a file, transmit it via email, save it in a new data file like netcdf.<br /> <br /> [[New Transmitters]]<br /> <br /> == Initialization and Terminiation Callbacks ==<br /> Add in your own initialization and termination routines. If you have routines that need to be run when the BES starts up, you can add callback functions for initialization and termination.<br /> <br /> [[BES Initialization Termination|Creating Initialization and Termination callbacks]]<br /> <br /> == Handling Exceptions ==<br /> Add in your own exception handling methods. When an exception is thrown within the BES, the exception is handled by the BES Exception Manager. It is possible to register a routine to be called before the default handling is done. This way, if you have a specific way that you want to handle specific exceptions, you can do that here.<br /> <br /> [[BES Exception Handling|Handling Exceptions within the BES]]<br /> <br /> == Aggregating your data ==<br /> <br /> By default there are no aggregation routines in the BES. But, the BES provides a mechanism for data to be aggregated.<br /> <br /> [[BES Aggregation|Providing Aggregation Routines]]<br /> <br /> == New commands ==<br /> <br /> You've seen all the default commands available in the BES. Now what if you want to create a new command, or even modify an already existing command.<br /> <br /> [[New BES Commands|Creating your own commands]]<br /> <br /> == Adding Debugging to your code ==<br /> <br /> The BES has a simple to use debugging tool. When the BES is started up you can specify -d "<output>,<context>" on the command line of the besctl script to enable debugging. The output option is either cerr, which will dump all debug statements to standard error, or a file name, which will dump all debug information to the specified file. The context option is a comma separated list of components.<br /> <br /> [[New Debugging]]<br /> <br /> == Submitting your modules ==<br /> So you've come up with a killer module that does some pretty cool stuff. Well ... share it with the rest of us.<br /> <br /> [[Sharing BES Modules|Sharing your BES Modules]]<br /> <br /> [[Category:Extending BES|Extending BES]]<br /> [[Category:BES Modules|Extending BES Module]]</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5801 Hyrax - Installation Instructions 2010-09-25T18:51:53Z

<p>PatrickWest: /* Short list */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, uuid-devel, readline-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> ## uuid<br /> # bes<br /> ## to build ssl authenticated connections, install openssl<br /> ## to use the bes command line utility, install readline-dev(el)<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> When building from source, to run tests:<br /> # dejagnu<br /> # cppunit<br /> <br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5800 Hyrax - Installation Instructions 2010-09-25T18:50:54Z

<p>PatrickWest: /* ''For the impatient'' */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, uuid-devel, readline-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> ## uuid<br /> # bes<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> When building from source, to run tests:<br /> # dejagnu<br /> # cppunit<br /> <br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5799 Hyrax - Installation Instructions 2010-09-25T18:49:52Z

<p>PatrickWest: /* Additional stuff */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, uuid-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> ## uuid<br /> # bes<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> When building from source, to run tests:<br /> # dejagnu<br /> # cppunit<br /> <br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5798 Hyrax - Installation Instructions 2010-09-25T18:49:06Z

<p>PatrickWest: /* Short list */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, uuid-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> ## uuid<br /> # bes<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5797 Hyrax - Installation Instructions 2010-09-25T18:48:41Z

<p>PatrickWest: /* ''For the impatient'' */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, uuid-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> # bes<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Hyrax_-_Installation_Instructions&diff=5796 Hyrax - Installation Instructions 2010-09-25T18:46:38Z

<p>PatrickWest: /* Short list */</p> <hr /> <div>Hyrax is really fairly easy to configure, but because there are several different components involved, it can seem complicated. Essentially you need to configure the BES to load and use one or more data handlers and to look in the right place for your data. The OLFS will work out of the box, but will need configuration to enable THREDDS functionality and Access Logging. Similarly, Tomcat needs no configuration, although you might want to change the port from 8080 to 80 if it's going to be the only web server on your host. You don't need to do that, it's just that you can if you want. Lastly, you can configure an Apache server to hand off requests to the server and, at the same time, have it emulate all your old Server3 URLs. That way catalogs, hot lists and other places where URLs to your data are stored will still work.<br /> This is not a complete manual for the BES. Instead of explaining all of the parameters and how their different settings can be used to control the BES, the discussion here is limited to what is used when the BES is run as part of Hyrax. In addition, there are number of parameters that should always be left to their default values for a Hyrax configuration and those will also be skipped. This makes it much easier to focus on those parameters that an installer really must understand.<br /> <br /> What you will need:<br /> <br /> * 30 minutes<br /> * Java 1.5 Runtime or Development Environment<br /> * Tomcat 5.5.x or newer<br /> * A web browser<br /> <br /> ==System Requirements==<br /> <br /> Hyrax is supported on Linux (our standard build platform is Fedora Core, we build our RPMs on CentOS 5.3 and we do also test on Ubuntu) and OS/X. It's certainly likely that you can get it to run on virtually any Unix-like operating system. We do not support Hyrax on Windows or Vista, although see the [[Using Virtual Machines to Serve Data]] project for a solution for Win32 platforms. Also a group at Microsoft Research has ported Hyrax 1.5.x to their Azure cloud platform.<br /> <br /> You can use either ''yum'', ''rpm'' or ''apt-get'' to get binary packages for all of the required software listed below. On many operating systems the only packages you'll need to get are those specific to data you intend to serve (e.g., hdf5). If you're building our source distributions, be sure to get the ''-devel' packages; if you're using our binary builds then the plain packages will be fine.<br /> <br /> In general, in addition to the software needed to build/run the basic framework, to serve data stored in a particular format, you need a handler that can read that format and often you need a that format's software library.<br /> <br /> === ''For the impatient'' ===<br /> <br /> On Linux system that uses RPM packages (Red Hat, CentOS, Fedora Core, ...), use ''yum'' to install libicu, netcdf, hdf4 and hdf5. Then install our packages. <br /> <br /> To build from source, install libxml2-devel, libcurl-devel, libicu-devel, netcdf-devel, hdf4-devel and hdf5-devel. Build. Your machine might have some of those packages, but RPM/yum will sort that out.<br /> <br /> On systems that use the Debian package system, use ''apt-get'' to install the same packages, except that the development packages (the ones you need to compile code) are called ''-dev'' instead of ''-devel''.<br /> <br /> === Short list ===<br /> # libdap<br /> ## libxml2 2.6.16<sup>*</sup><br /> ## libcurl 7.10.6<sup>*</sup><br /> ## pthreads (part of glibc on linux)<sup>*</sup><br /> # bes<br /> # dap-server (referred to as General purpose handlers on download page)<br /> # OLFS<br /> ## Tomcat 6.0.x<br /> ## Java 1.5<sup>*</sup> run-time (or newer, depending on the requirements of the Tomcat software).<br /> <br /> === Additional stuff ===<br /> To serve NetCDF files:<br /> # The netCDF handler<br /> ## libnetcdf 3.x<br /> <br /> To serve HDF4 files:<br /> # The HDF4 handler<br /> ## The HDF4 library (4.x) and it's required software (often libjpeg, zlib<sup>*</sup> and szip)<br /> <br /> To serve formatted ASCII using FreeForm:<br /> # The FreeForm handler<br /> <br /> To serve HDF5 files:<br /> # The HDF5 handler<br /> ## The HDF5 library (1.6.0 or 1.8.x) (and often zlib<sup>*</sup>)<br /> <br /> To provide data responses as NetCDF files:<br /> # The Fileout NetCDF handler<br /> ## libnetcdf 3.x <br /> <br /> To use NCML:<br /> # The NCML Handler<br /> ## libicu 3.6<br /> ## At least one data handler<br /> <br /> <sup>*</sup>''Stock on any recent linux or OS/X distribution''<br /> <br /> ==Secure Installation Guidelines==<br /> <br /> [[Hyrax - Secure Installation Guidelines|Making Hyrax Secure]]<br /> <br /> ==Install the BES==<br /> <br /> [[Hyrax - BES Installation|Install and configure the BES.]]<br /> <br /> ==Install the OLFS==<br /> <br /> [[Hyrax - OLFS Installation|Next You need to install and configure the OLFS inside of a Tomcat server]]<br /> <br /> ==Starting and Stopping Hyrax==<br /> <br /> To correctly start and stop Hyrax, follow these simple recipes. <br /> <br /> ===Starting===<br /> # Start the BES first using the ''besctl'' program with the ''start'' argument. See [[Hyrax_-_Starting_and_stopping_the_BES|Starting and stopping the BES]] for more information.<br /> # Start the OLFS using the Tomcat startup script. See [[Hyrax_-_OLFS_Installation#Start_Tomcat| Starting Tomcat]] for more information.<br /> <br /> ===Stopping===<br /> <br /> # Stop the OLFS first using the tomcat shutdown script.<br /> # Stop the BES using the ''besctl'' program with the ''stop'' argument.<br /> <br /> See above for stopping information.<br /> <br /> ==Test the server==<br /> <br /> To test/use the server, open a web browser and open the URL: http://localhost:8080/ and you should see the Tomcat default page. <br /> <br /> Open the URL: http://localhost:8080/opendap/ to see an HTML directory of the top of your data archive.<br /> <br /> You can get the HTML view THREDDS catalog by appending "catalog.html" to the end of any URL that returns an HTML directory, like so: http://localhost:8080/opendap/catalog.html But you may need to do some additional configuration that to work.<br /> <br /> ==Configure Your Hyrax Installation==<br /> <br /> At this point, Hyrax should be working. When you install Hyrax for the first time it is pre-configured to serve test data sets that come with each of the installed data handlers. This will allow you to test the server and make sure it is functioning correctly. After that you will want to configure so that it works with your data.<br /> <br /> ==Miscellaneous==<br /> <br /> Once opendap.war is expanded, manually copy the file favicon.ico file:<br /> $CATALINA_HOME/webapps/opendap/docs/images/favicon.ico<br /> to <br /> $CATALINA_HOME/webapps/ROOT/<br /> <br /> The favicon.ico file is mostly a convenience to keep browsers from constantly asking for it (substitute your own icon if you like!).</div>

PatrickWest https://docs.opendap.org/index.php?title=Documentation&diff=5767 Documentation 2010-07-15T18:46:38Z

<p>PatrickWest: /* Software */</p> <hr /> <div>__NOTOC__<br /> This site contains the (new) repository for OPeNDAP software documentation.<br /> <br /> =User Guides=<br /> <br /> ===[[QuickStart|Getting Started with OPeNDAP Software]]===<br /> <br /> An overview of our software<br /> <br /> ===[[UserGuide|The User's Guide]]===<br /> <br /> A comprehensive guide to sharing data with our software<br /> <br /> ===[[Server Side Processing Functions]]===<br /> <br /> A listing of functions that Hyrax provides along with their documentation.<br /> <br /> =Software=<br /> <br /> ===[[Hyrax|Hyrax - Installation and Customization (BES, OLFS, format & response handlers)]]===<br /> <br /> The OPeNDAP Data Server, Hyrax, is the latest data server from OPeNDAP. These pages contain documentation that covers server installation and customization.<br /> <br /> ===[[libdap|libdap++: C++ DAP Implementation Documentation]]===<br /> <br /> Libdap is the C++ implementation of the OPeNDAP Data Access Protocol. This pages contains links to the libdap++ reference guide and a usage guide that explains some of the ins and outs of this class library.<br /> <br /> ===[http://www.opendap.org/api/bes/html/index.html BES Implementation Documentation]===<br /> <br /> The BES (OPeNDAP Back-End Server) is a part of the OPeNDAP 4 Server, known as Hyrax. These pages contain the BES reference guide.<br /> <br /> =White Papers=<br /> <br /> ===[[Using Virtual Machines to Serve Data]]===<br /> <br /> This short guide discusses using a virtual machine and a [http://en.wikipedia.org/wiki/Hypervisor hypervisor] ([http://www.vmware.com/ VMware] Server) to serve data. In the guide we cover both serving data with Hyrax running within a VM and also using Hyrax in a workshop where the hypervisor is VMware Workstation. You cannot actually serve data to remote users with Workstation, but it's a great environment in which to learn about the server's different capabilities. In both cases the advantages of using a VM are that the server runs in Linux on the virtual machine while you run the hypervisor under any of its supported operating systems. An additional advantage is that the hypervisor can be used very effectively in the context of an overall security plan.<br /> <br /> ===[[ServerDispatchOperations|Server Dispatch Operations]]===<br /> <br /> ===[[Libdap On Leopard|Building libdap on Lepoard]] (OS/X)===<br /> <br /> =Housekeeping=<br /> <br /> ====[[Wiki Testing]]====<br /> <br /> ====[[Retired|Retired Pages]]====</div>

PatrickWest