DataDDX - Revision history

Ndp at 17:43, 29 April 2013

2013-04-29T17:43:42Z

← Older revision		Revision as of 17:43, 29 April 2013
Line 125:		Line 125:
	=== Reliable Errors in the Hyrax Response ===		=== Reliable Errors in the Hyrax Response ===

	The BES and OLFS communicate using data transmissions that are ''chunked''. That means that the BES first sends a seven-byte ASCII-HEX ''byte count'' and ''control character'' of ''d'' for data (for a total of eight bytes) followed by that number of bytes of data, zero or more times followed by a ''byte count/control character'' sequence of 0x0000000d (seven ASCII digits plus a ''d''). If the BES encounters an error it sends a ''byte count'' and ''control character'' that is out of band control information and then the error in a following data block. The chunking scheme is described on the Trac site in [~~http~~://scm.opendap.org~~:8090~~/trac/wiki/BES_Chunking BES Chunking].		The BES and OLFS communicate using data transmissions that are ''chunked''. That means that the BES first sends a seven-byte ASCII-HEX ''byte count'' and ''control character'' of ''d'' for data (for a total of eight bytes) followed by that number of bytes of data, zero or more times followed by a ''byte count/control character'' sequence of 0x0000000d (seven ASCII digits plus a ''d''). If the BES encounters an error it sends a ''byte count'' and ''control character'' that is out of band control information and then the error in a following data block. The chunking scheme is described on the Trac site in [https://scm.opendap.org/trac/wiki/BES_Chunking BES Chunking].

	What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the 0x0000000d is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.		What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the 0x0000000d is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.

Ndp: /* Example DataDDX response Sent via HTTP 1.1 */

2012-02-21T23:14:09Z

Example DataDDX response Sent via HTTP 1.1

← Older revision		Revision as of 23:14, 21 February 2012
Line 45:		Line 45:
	Note that this example shows the DataDDX being returned using HTTP/1.1. In past versions of DAP important information was encoded in the HTTP response headers. In this example the key information, that this response conforms to DAP version 3.2 is encoded both in the response header and the DDX response element using the ''dap-version'' attribute. This makes the DataDDX more friendly toward applications which use non-HTTP transport protocols.		Note that this example shows the DataDDX being returned using HTTP/1.1. In past versions of DAP important information was encoded in the HTTP response headers. In this example the key information, that this response conforms to DAP version 3.2 is encoded both in the response header and the DDX response element using the ''dap-version'' attribute. This makes the DataDDX more friendly toward applications which use non-HTTP transport protocols.

	<~~pre~~>		<font size="2">
			<source lang="xml">
	HTTP/1.1 200 OK		HTTP/1.1 200 OK
	Server: Apache-Coyote/1.1		Server: Apache-Coyote/1.1
Line 115:		Line 116:

	--mimepart_7_9651610.1145395859678--		--mimepart_7_9651610.1145395859678--
	</~~pre~~>		</source>
			</font>

	== Encoding the binary data ==		== Encoding the binary data ==

Jimg at 18:03, 15 February 2011

2011-02-15T18:03:12Z

← Older revision		Revision as of 18:03, 15 February 2011
Line 7:		Line 7:
	As an alternative, suppose we build a data response using the DDX in a multipart document and then encoded type information in the data stream as well? <s>This would provide a way to bundle attributes with variables in the data response and locate type information with the data values (for sequences mostly).</s> <font color="red">It's not possible to return the Attributes with a constrained DDX because the constraint can alter the attributes in ways that cannot be computed without semantic knowledge about the attribute.</font>		As an alternative, suppose we build a data response using the DDX in a multipart document and then encoded type information in the data stream as well? <s>This would provide a way to bundle attributes with variables in the data response and locate type information with the data values (for sequences mostly).</s> <font color="red">It's not possible to return the Attributes with a constrained DDX because the constraint can alter the attributes in ways that cannot be computed without semantic knowledge about the attribute.</font>
	</blockquote>		</blockquote>

			[[DAP_4.0_Design#DataDDX\|See also the Dap 4.0 Design for the DDX]] Note that there are some contradictions between the two documents.

	== Questions to consider ==		== Questions to consider ==

Jimg: /* Encoding Sequence Data */

2011-02-15T17:40:36Z

Encoding Sequence Data

← Older revision		Revision as of 17:40, 15 February 2011
Line 129:		Line 129:
	=== Encoding Sequence Data ===		=== Encoding Sequence Data ===

			Adopt the suggestion that type information be included in the data stream. This will duplicate some (small) amount of information, but make the software to decode the data easier to write, including particularly, nested sequences.

	[[Category:Development\|DataDDX]][[Category:DAP4\|DataDDX]]		[[Category:Development\|DataDDX]][[Category:DAP4\|DataDDX]]

Jimg: /* Reliable Errors in the Hyrax Response */

2011-02-15T17:35:13Z

Reliable Errors in the Hyrax Response

← Older revision		Revision as of 17:35, 15 February 2011
Line 125:		Line 125:
	What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the 0x0000000d is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.		What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the 0x0000000d is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.

	I would propose that for DAP 3.2 (or 3.3?) we make it so that those byte counts are passed onto the client in the data part of the DataDDX response. This will allow Hyrax to send errors and other out-of-band information to clients in a way that client can actually use.		I would propose that for DAP 3.2 (or 3.3?) we make it so that those byte counts are passed onto the client in the data part of the DataDDX response. This will allow Hyrax to send errors and other out-of-band information to clients in a way that client can actually use. See [[Hyrax_-_BES_PPT#PPT_Chunking \| Chunking]] for a description; it's fairly simple.

	=== Encoding Sequence Data ===		=== Encoding Sequence Data ===

Jimg at 17:26, 15 February 2011

2011-02-15T17:26:06Z

← Older revision		Revision as of 17:26, 15 February 2011
Line 5:		Line 5:

	<blockquote>		<blockquote>
	As an alternative, suppose we build a data response using the DDX in a multipart document and then encoded type information in the data stream as well? This would provide a way to bundle attributes with variables in the data response and locate type information with the data values (for sequences mostly).		As an alternative, suppose we build a data response using the DDX in a multipart document and then encoded type information in the data stream as well? <s>This would provide a way to bundle attributes with variables in the data response and locate type information with the data values (for sequences mostly).</s> <font color="red">It's not possible to return the Attributes with a constrained DDX because the constraint can alter the attributes in ways that cannot be computed without semantic knowledge about the attribute.</font>
	</blockquote>		</blockquote>

Jimg: /* Reliable Errors in the Hyrax Response */

2009-05-19T18:14:04Z

Reliable Errors in the Hyrax Response

← Older revision		Revision as of 18:14, 19 May 2009
Line 123:		Line 123:
	The BES and OLFS communicate using data transmissions that are ''chunked''. That means that the BES first sends a seven-byte ASCII-HEX ''byte count'' and ''control character'' of ''d'' for data (for a total of eight bytes) followed by that number of bytes of data, zero or more times followed by a ''byte count/control character'' sequence of 0x0000000d (seven ASCII digits plus a ''d''). If the BES encounters an error it sends a ''byte count'' and ''control character'' that is out of band control information and then the error in a following data block. The chunking scheme is described on the Trac site in [http://scm.opendap.org:8090/trac/wiki/BES_Chunking BES Chunking].		The BES and OLFS communicate using data transmissions that are ''chunked''. That means that the BES first sends a seven-byte ASCII-HEX ''byte count'' and ''control character'' of ''d'' for data (for a total of eight bytes) followed by that number of bytes of data, zero or more times followed by a ''byte count/control character'' sequence of 0x0000000d (seven ASCII digits plus a ''d''). If the BES encounters an error it sends a ''byte count'' and ''control character'' that is out of band control information and then the error in a following data block. The chunking scheme is described on the Trac site in [http://scm.opendap.org:8090/trac/wiki/BES_Chunking BES Chunking].

	What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the ~~0x0000000~~ is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.		What currently happens in the OLFS to suppoprt DAP2 responses is that the ''byte count'' is read, then the following block of data is read and passed onto the client, then the next ''byte count'' is read, ..., until the 0x0000000d is read signaling the end of the document. That is the chunked nature of the data transmission is stripped so that the data Hyrax now (DAP 2.0 to 3.2) sends is not chunked.

	I would propose that for DAP 3.2 (or 3.3?) we make it so that those byte counts are passed onto the client in the data part of the DataDDX response. This will allow Hyrax to send errors and other out-of-band information to clients in a way that client can actually use.		I would propose that for DAP 3.2 (or 3.3?) we make it so that those byte counts are passed onto the client in the data part of the DataDDX response. This will allow Hyrax to send errors and other out-of-band information to clients in a way that client can actually use.

Jimg at 17:51, 19 May 2009

2009-05-19T17:51:43Z

← Older revision		Revision as of 17:51, 19 May 2009
Line 1:		Line 1:
	* Use a multipart MIME document to hold the DDX and one ''data blob'' per DDX. Adopt the same use of Multipart. MIME as WCS uses.		* Use a multipart MIME document to hold the DDX and one ''data blob'' per DDX. Adopt the same use of Multipart MIME as WCS uses.

	* consider the following design idea for embedding type information within the data stream:		* consider the following design idea for embedding type information within the data stream:

Jimg: /* Questions to consider */

2009-05-06T22:47:43Z

Questions to consider

← Older revision		Revision as of 22:47, 6 May 2009
Line 10:		Line 10:
	== Questions to consider ==		== Questions to consider ==

	~~# Is it stupid?~~
	# Will we be able to implement it efficiently?		# Will we be able to implement it efficiently?
	# How can we get those pesky reliable error messages/objects in there?		# How can we get those pesky reliable error messages/objects in there?

Jimg: /* Questions to consider */

2009-04-14T18:34:24Z

Questions to consider

← Older revision		Revision as of 18:34, 14 April 2009
Line 13:		Line 13:
	# Will we be able to implement it efficiently?		# Will we be able to implement it efficiently?
	# How can we get those pesky reliable error messages/objects in there?		# How can we get those pesky reliable error messages/objects in there?
	# What will trigger ~~out~~ server to send this response?		# What will trigger our server to send this response?

	== Normative References ==		== Normative References ==