BES File Out NetCDF: Difference between revisions
No edit summary |
|||
Line 24: | Line 24: | ||
# A Sequence with a Structure or Grid should be flattened by recursively applying the flattening logic to the components. | # A Sequence with a Structure or Grid should be flattened by recursively applying the flattening logic to the components. | ||
Structures | === Structures === | ||
* Flatten | * Flatten | ||
* prepend name of structure with an underscore followed by the variable name. Keep track as there might be embedded structures, grids, etc... | * prepend name of structure with an underscore followed by the variable name. Keep track as there might be embedded structures, grids, etc... | ||
Line 32: | Line 32: | ||
[[User:Jimg|jimg]] 17:53, 18 December 2008 (PST) I would use a dot even though I know that dots in variable names are, in general, a bad idea. If we use underscores then it maybe hard for clients to form a name that can be used to access values from a server based on the information in the file. | [[User:Jimg|jimg]] 17:53, 18 December 2008 (PST) I would use a dot even though I know that dots in variable names are, in general, a bad idea. If we use underscores then it maybe hard for clients to form a name that can be used to access values from a server based on the information in the file. | ||
Grid | === Grid === | ||
* Flatten. | * Flatten. | ||
* Use the name of the grid for the array of values | * Use the name of the grid for the array of values | ||
* prepend the name of the grid plus an underscore to the names of each of the map vectors. | * prepend the name of the grid plus an underscore to the names of each of the map vectors. [[User:Jimg|jimg]] 11:31, 21 December 2008 (PST) A more sophisticated version might look at the values of two or more grids that use the same names and have the same type (e.g., Float64 lon[360]) and if they are the same, make them shared dimensions. | ||
Array | === Array === | ||
* write_array appears to be working just fine. | * write_array appears to be working just fine. | ||
* If array of complex types? | * If array of complex types? | ||
Sequences | === Sequences === | ||
* For now throw an exception | * For now throw an exception [[User:Jimg|jimg]] 11:31, 21 December 2008 (PST) Initial version should elide these I think because there are important cases where they appear as part of a dataset but not the main part. We can represent these as arrays easily in the future. | ||
Attributes | === Attributes === | ||
* Global Attributes? | * Global Attributes? | ||
** Include global attribute to include URL that generated that new file | ** Include global attribute to include URL that generated that new file |
Revision as of 19:31, 21 December 2008
How to flatten hierarchical types
For a structure such as
Structure { Int x; Int y; } Point;
represent that as
Point.x Point.y
Explicitly including the dot seems ugly and like a kludge and so on, but it means that the new variable name can be feed back into the server to get the data.
Because this is hardly a lossless, we should also add an attribute that contains the original real name of the variable - information that this is the result of a flattening operation, that the parent variable was a Structure, Sequence or Grid and its name was xyz. Given that, it should be easy to sort out how to make a future request for the data in the translated variable.
This in some way obviates the need for the dot, but I think we should use that regardless.
General questions
- Should I traverse the data structure to see if there are any sequences?
jimg 17:53, 18 December 2008 (PST) Yes. An initial version should note their presence and add an attribute noting that they have been elided. To translate a Sequence, there are several cases to consider:
- A Sequence of simple types only (which means a one-level sequence): translate to a set of arrays using a name-prefix flattening scheme.
- A nested sequence (otherwise with only simple types) should first be flattened to a one level sequence and then that should be flattened.
- A Sequence with a Structure or Grid should be flattened by recursively applying the flattening logic to the components.
Structures
- Flatten
- prepend name of structure with an underscore followed by the variable name. Keep track as there might be embedded structures, grids, etc...
jimg 17:53, 18 December 2008 (PST) Use the procedure described above in How to flatten hierarchical types.
jimg 17:53, 18 December 2008 (PST) I would use a dot even though I know that dots in variable names are, in general, a bad idea. If we use underscores then it maybe hard for clients to form a name that can be used to access values from a server based on the information in the file.
Grid
- Flatten.
- Use the name of the grid for the array of values
- prepend the name of the grid plus an underscore to the names of each of the map vectors. jimg 11:31, 21 December 2008 (PST) A more sophisticated version might look at the values of two or more grids that use the same names and have the same type (e.g., Float64 lon[360]) and if they are the same, make them shared dimensions.
Array
- write_array appears to be working just fine.
- If array of complex types?
Sequences
- For now throw an exception jimg 11:31, 21 December 2008 (PST) Initial version should elide these I think because there are important cases where they appear as part of a dataset but not the main part. We can represent these as arrays easily in the future.
Attributes
- Global Attributes?
- Include global attribute to include URL that generated that new file
- Variable Attributes
- What about attributes for structures? Should these attributes be created for each of the variables in the structure? So, if there is a structure a with variables v1 and v2 then the attributes for a will be attributes for a_v1 and a_v2? Or are there attributes for each of the variables in the structure? Or both.
- For multi-dimensional datasets there will be a structure for each container, and each of these containers will have global attributes.