Hyrax: Asynchronous Response Implementation - Revision history

Ndp: /* Overview */

2012-04-12T16:05:14Z

Overview

← Older revision		Revision as of 16:05, 12 April 2012
Line 6:		Line 6:
	# Server side processing		# Server side processing

	I think it's important to identify the various issues in each of these use cases because much of what has to happen isn't just the asynchronous aspects of the response. I think that while the Async definition for DAP4 allows the Async response essentially forward the request to a new URL in order to access the requested item at a later date, we might want to utilize our existing architecture to make the URL the same as the original request URL. We do exactly this with uncompressed files that are cached, a we could use a similar mechanism for the async use cases...		I think it's important to identify the various issues in each of these use cases because much of what has to happen isn't just the asynchronous aspects of the response.


			;URLs
			: I think that while the Async definition for DAP4 allows the Async response essentially forward the request to a new URL in order to access the requested item at a later date, we might want to utilize our existing architecture to make the URL the same as the original request URL. We do exactly this with uncompressed files that are cached, a we could use a similar mechanism for the async use cases...

	== Near-line tape drive ==		== Near-line tape drive ==

Ndp at 15:47, 12 April 2012

2012-04-12T15:47:33Z

← Older revision		Revision as of 15:47, 12 April 2012
Line 1:		Line 1:

			== Overview ==
	The proposed use cases so far are:		The proposed use cases so far are:

Line 4:		Line 6:
	# Server side processing		# Server side processing

	I think it's important to identify the various issues in each of these use cases because much of what has to happen isn't just the asynchronous aspects of the response.		I think it's important to identify the various issues in each of these use cases because much of what has to happen isn't just the asynchronous aspects of the response. I think that while the Async definition for DAP4 allows the Async response essentially forward the request to a new URL in order to access the requested item at a later date, we might want to utilize our existing architecture to make the URL the same as the original request URL. We do exactly this with uncompressed files that are cached, a we could use a similar mechanism for the async use cases...


	== Near-line tape drive ==		== Near-line tape drive ==

Ndp at 04:37, 12 April 2012

2012-04-12T04:37:13Z

Ndp: /* Server Side Processing */

2012-04-11T21:17:36Z

Server Side Processing

← Older revision		Revision as of 21:17, 11 April 2012
Line 57:		Line 57:
	: It will be crucial that the BES be able to identify when a server side processing dataset has been requested but has not yet been cached in order to prevent multiple concurrent calls to generate the same dataset.		: It will be crucial that the BES be able to identify when a server side processing dataset has been requested but has not yet been cached in order to prevent multiple concurrent calls to generate the same dataset.

	=== Basic Flow ===		=== Basic Flow Stage 1===
	# A client asks for a catalog of holdings from a Hyrax server.		# A client asks for a catalog of holdings from a Hyrax server.
	# The client makes a request for one the datasets in the catalog, in the constraint expression the client requests that server perform a computationally intensive operation.		# The client makes a request for one the datasets in the catalog, in the constraint expression the client requests that server perform a computationally intensive operation.
Line 67:		Line 67:
	# The BES initiates the generation of the requested dataset/processing combination.		# The BES initiates the generation of the requested dataset/processing combination.

	=== ~~Alternate~~ Flow ===		=== Basic Flow Stage 2 ===
	~~# A client asks for a catalog of holdings from a Hyrax server.~~
	# The client makes a request for one the datasets in the catalog, in the constraint expression the client requests that server perform a computationally intensive operation.		# The client makes a request for one the datasets in the catalog, in the constraint expression the client requests that server perform a computationally intensive operation.
	# OLFS receives the request for a dataset and routes that request to the BES.		# OLFS receives the request for a dataset and routes that request to the BES.

Ndp: /* Basic Flow Stage 2 */

2012-04-11T21:04:19Z

Basic Flow Stage 2

← Older revision		Revision as of 21:04, 11 April 2012
Line 37:		Line 37:
	=== Basic Flow Stage 2===		=== Basic Flow Stage 2===

	~~# A client asks for a catalog of holdings from a Hyrax server.~~
	# The client makes a request for one the datasets in the catalog.		# The client makes a request for one the datasets in the catalog.
	# OLFS receives the request for a dataset and routes that request to the BES.		# OLFS receives the request for a dataset and routes that request to the BES.

Ndp: /* Near-line tape drive */

2012-04-11T21:03:01Z

Near-line tape drive

← Older revision		Revision as of 21:03, 11 April 2012
Line 23:		Line 23:
	: It will be crucial that the BES be able to identify when a near-line dataset has been requested but has not yet been cached in order to prevent multiple concurrent calls to unpack the same dataset.		: It will be crucial that the BES be able to identify when a near-line dataset has been requested but has not yet been cached in order to prevent multiple concurrent calls to unpack the same dataset.

	=== Basic Flow ===		=== Basic Flow Stage 1===

	# A client asks for a catalog of holdings from a Hyrax server.		# A client asks for a catalog of holdings from a Hyrax server.
Line 35:		Line 35:
	# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.		# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.

	=== ~~Alternate~~ Flow ===		=== Basic Flow Stage 2===

	# A client asks for a catalog of holdings from a Hyrax server.		# A client asks for a catalog of holdings from a Hyrax server.

Ndp: /* Basic Flow */

2012-04-11T21:02:24Z

Basic Flow

← Older revision		Revision as of 21:02, 11 April 2012
Line 67:		Line 67:
	# The OLFS responds to the client request with the "async" response.		# The OLFS responds to the client request with the "async" response.
	# The BES initiates the generation of the requested dataset/processing combination.		# The BES initiates the generation of the requested dataset/processing combination.
	~~# The client returns during the specified time frame and asks again for the requested dataset/processing combination.~~
	~~# The BES checks to see if the requested dataset/processing combination is in the server side processing cache.~~
	~~# The requested dataset is in the cache and the BES services the request immediately.~~


	=== Alternate Flow ===		=== Alternate Flow ===

Ndp: /* Basic Flow */

2012-04-11T21:02:08Z

Basic Flow

← Older revision		Revision as of 21:02, 11 April 2012
Line 34:		Line 34:
	# The BES checks to see if the requested dataset is in the process of being extracted from the near-line system.		# The BES checks to see if the requested dataset is in the process of being extracted from the near-line system.
	# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.		# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.
	~~# The client returns during the specified time frame and asks again for the requested dataset.~~
	~~# The BES checks to see if the requested dataset is in the near-line cache.~~
	~~# The requested dataset is in the cache the BES services the request immediately.~~

	=== Alternate Flow ===		=== Alternate Flow ===

Ndp: /* Server Side Processing */

2012-04-11T20:57:18Z

Server Side Processing

← Older revision		Revision as of 20:57, 11 April 2012
Line 66:		Line 66:
	# OLFS receives the request for a dataset and routes that request to the BES.		# OLFS receives the request for a dataset and routes that request to the BES.
	# The BES identifies that this is a request will require sometime to service.		# The BES identifies that this is a request will require sometime to service.
	# The BES checks to see if the requested result is already in the server side processing cache.		# The BES checks to see if the requested result is already in the server side processing cache, and finds that is not there.
	~~# If the requested dataset~~ is ~~in the cache the BES services the request immediately.~~		# The BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.
	~~#* Jump to end~~.		# The OLFS responds to the client request with the "async" response.
	# The ~~BES checks to see if the same requested dataset/processing combination is in the process of being generated.~~
	~~# If the the requested dataset/processing is already in the process of being generated but has not completed then the BES instructs the OLFS to return a "404" response.~~
	~~# If the requested dataset/processing combination is NOT in the server side processing cache the~~ BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.
	# The BES initiates the generation of the requested dataset/processing combination.		# The BES initiates the generation of the requested dataset/processing combination.
	~~# The OLFS responds to the client request with the "async" response.~~
	# The client returns during the specified time frame and asks again for the requested dataset/processing combination.		# The client returns during the specified time frame and asks again for the requested dataset/processing combination.
	# The BES checks to see if the requested dataset/processing combination is in the server side processing cache.		# The BES checks to see if the requested dataset/processing combination is in the server side processing cache.
	# ~~If the~~ requested dataset~~/processing combination~~ is in the cache the BES services the request immediately.		# The requested dataset is in the cache and the BES services the request immediately.
	# If the the requested dataset~~/processing combination~~ is in the ~~process of being extracted but has not completed then~~ the BES ~~instructs~~ the ~~OLFS to return a "404" response~~.

			=== Alternate Flow ===
			# A client asks for a catalog of holdings from a Hyrax server.
			# The client makes a request for one the datasets in the catalog, in the constraint expression the client requests that server perform a computationally intensive operation.
			# OLFS receives the request for a dataset and routes that request to the BES.
			# The BES identifies that this is a request will require sometime to service.
			# The BES checks to see if the requested result is already in the server side processing cache.
			# The requested dataset is in the cache and the BES services the request immediately.

Ndp: /* Basic Flow */

2012-04-11T20:54:58Z

Basic Flow

← Older revision		Revision as of 20:54, 11 April 2012
Line 36:		Line 36:
	# The client returns during the specified time frame and asks again for the requested dataset.		# The client returns during the specified time frame and asks again for the requested dataset.
	# The BES checks to see if the requested dataset is in the near-line cache.		# The BES checks to see if the requested dataset is in the near-line cache.
	# ~~If the~~ requested dataset is in the cache the BES services the request immediately.		# The requested dataset is in the cache the BES services the request immediately.
	~~# If the requested dataset is in the process of being extracted but has not completed then the BES instructs the OLFS to return a "404" response~~.

	=== Alternate Flow ===		=== Alternate Flow ===

← Older revision		Revision as of 04:37, 12 April 2012
Line 1:		Line 1:

	The proposed use cases so far are:		The proposed use cases so far are:

Line 31:		Line 30:
	# The BES checks to see if the requested dataset is in the near-line cache andy finds the requested dataset is NOT represented in the cache.		# The BES checks to see if the requested dataset is in the near-line cache andy finds the requested dataset is NOT represented in the cache.
	# The BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.		# The BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.
	# The OLFS responds to the client request with the "async" response.		# The OLFS responds to the client request with the "async" response and HTTP 201 (Accepted)
	# The BES checks to see if the requested dataset is in the process of being extracted from the near-line system.		# The BES checks to see if the requested dataset is in the process of being extracted from the near-line system.
	# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.		# If the requested dataset is not already in the process of being extracted the BES initiates the retrieval of the requested dataset from the near-line system.
Line 42:		Line 41:
	# The BES checks to see if the requested dataset is in the near-line cache.		# The BES checks to see if the requested dataset is in the near-line cache.
	# The requested dataset is in the cache the BES services the request immediately.		# The requested dataset is in the cache the BES services the request immediately.
			# The OLFS forwards the BES response and returns HTTP 200.

	== Server Side Processing ==		== Server Side Processing ==
Line 64:		Line 64:
	# The BES checks to see if the requested result is already in the server side processing cache, and finds that is not there.		# The BES checks to see if the requested result is already in the server side processing cache, and finds that is not there.
	# The BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.		# The BES returns an error/message to the OLFS indicating that the response will be delayed by a specified amount of time.
	# The OLFS responds to the client request with the "async" response.		# The OLFS responds to the client request with the "async" response and HTTP 201 (Accepted).
	# The BES initiates the generation of the requested dataset/processing combination.		# The BES initiates the generation of the requested dataset/processing combination.

Line 73:		Line 73:
	# The BES checks to see if the requested result is already in the server side processing cache.		# The BES checks to see if the requested result is already in the server side processing cache.
	# The requested dataset is in the cache and the BES services the request immediately.		# The requested dataset is in the cache and the BES services the request immediately.
			# The OLFS forwards the BES response and returns HTTP 200.