Loading and Saving Datasets

Introduction

DAVE has built-in read/write support for several file formats and more will be added in future as needed or as requested by the user community. Currently, the following file formats are supported.

Reading/Loading Data from File

Loading ASCII Column files

To initiate this action, select the
Data Input/Ouput => Read Dataset From => ASCII(txt,spe,...)
menu item from the main application window. A file selection dialog window will be displayed for you to select the file to be loaded. Navigate to the appropriate directory and select the ASCII file to load. If your chosen file contains multicolum data, you will be taken through a 3 step wizard that guides you though the remainder of the process.

ASCII column wizard
Wizard for reading ASCII column data: step 1 of 3

The first dialog shows the data columns read from the file with options to specify how the columns are separated, comments to ignore and the line number where the real data starts. The first line in the example data file contains comments/axes labels so we want to start reading data from line 2. So enter 2 in the Data Starts at Line: field - the comment line should disappear from the data shown. Click on the Next>> button at the bottom right of the dialog to continue to step 2.

ASCII column wizard
Wizard for reading ASCII column data: step 2 of 3

The step 2 dialog allows you to indicate the number of valid data columns and the delimiter used between the columns. The default values for both are correct so click on the Next>> button to proceed.

ASCII column wizard ASCII column wizard
Wizard for reading ASCII column data: step 3 of 3

In step 3, you should assign sensible labels to the data columns which will be used later in plotting. Also you can alter the data type for any column that appears incorrect on inspection. The default column names are shown on the left screedshot. Use the mouse to select each of the 3 fields from the top left of the dialog. For each field, enter a column name in the Name field located slightly to the right and press the Enter when complete. Note that non-alphanumeric and special characters (space, comma, brackets etc) are not allowed to be used in column names!. When done, the three fields should have the labels shown in the right screenshot. Click on the Finish button to complete the wizard. An entry for the dataset will be created in the Data Browser tree in the main application window.

Loading ASCII Group or SPE files

To initiate this action, select the
Data Input/Ouput => Read Dataset From => ASCII(txt,spe,...)
menu item from the main application window. A file selection dialog window will be displayed for you to select the file to be loaded. Navigate to the appropriate directory and select the ASCII file to load. If your chosen file contains ASCII GRP or SPE format data, then it would be read without any errors. However, since these files do not contain sufficient metadata information, a dialog will be launched to allow you the opportunity to specify additional details.

ASCII SPE metadata dialog
Dialog for entering metadata for ASCII SPE or GRP format data

Enter metadata as appropriate in the fields indicated. Values entered in the NAme column will be used to identify the data entries when viewed from the Data Browser while the Text Labels and Units will be used in the axes labels in visualizations created subsequently from the dataset. Click on the OK button to accept the changes you have made or on the Cancel button to discard the changes. An entry for the dataset with the same name as the filename will be created in the Data Browser.

Loading DAVE files

To initiate this action, select the
Data Input/Ouput => Read Dataset From => DAVE
menu item from the main application window. A file selection dialog window will be displayed for you to select the file to be loaded. Navigate to the appropriate directory and select the file to load. If your chosen file contains DAVE format data, then it would be read without any errors. DAVE format files include basic metadata information required for plotting so no additional dialogs are displayed during the loading process. If the file is successfully read, an entry for it will be created in the Data Browser section of the main application window.

Writing/Saving Data to File

Datasets located in the Data Browser section of the application window can be written to a file on disk. Only the entire dataset can be saved and not a subsection - for example, it is not possible to simply save the dependent data field of a dataset.
To save one or more dataset(s), simply select the top-level container(s) of the dataset(s) from the Data Browser tree. Then, depending on the output format you wish to save or export your dataset as, select one of the following menu items:

Menu to SelectOutput File Format
Data Input/Ouput => Write Dataset as => DAVEDAVE
Data Input/Ouput => Write Dataset as => ASCII ColumnASCII Column
Data Input/Ouput => Write Dataset as => ASCII GroupASCII Group
Data Input/Ouput => Write Dataset as => ASCII SPEASCII SPE

You should be able to save a dataset using any of the supported data formats irrespective of the original format of the dataset. However, it should be noted that ASCII Group and ASCII SPE formats are suitable only for storing 2D datasets i.e. datasets containing two independent variables.

Description of Supported File Formats

ASCII Column

The ASCII column format is the basic space-delimited multicolum data format that is suitable for reading into any standard spreadsheet application. A header section is optional and must be located at the top of the file occupying as many lines as necessary. An example ASCII Column file is included here for inspection.

ASCII Group

An ASCII group file is designed to store a 2-dimensional (i.e. two independent variables) dataset. An example file will look like this:

#Number of x-values
5
# Number of y-values
3
# xvalues:
-2.0000000
-1.0000000
0.00000000
1.0000000
2.0000000
# yvalues:
0.250000
1.00000
1.75000
# Group: 0
229.00000 15.132746
451.00000 21.236761
4380.0000 66.181568
455.00000 21.330729
209.00000 14.456832
# Group: 1
200.00000 14.142136
426.00000 20.639767
3260.0000 57.096410
393.00000 19.824228
219.00000 14.798649
# Group: 2
203.00000 14.247807
320.00000 17.888544
2103.0000 45.858478
333.00000 18.248288
232.00000 15.231546


The ASCII group format is very strict in the sense that the comments lines (those beginning with a #) are required and must occupy a single line only. The first couple of lines consist of a comment and the size (number of points) of the first independent variable - which is 5 in the above example. The next couple of lines specify the same information for the second independent variable. This is followed by another comment line and the actual values for the first independent variable. The values for the second independent are then added. Note that the comments can be used to specify sensible labels (names + data units) for the independent axes. The remainder of the file defines the dependent data and error in the data.
Essentially, the data and errors are defined as follows. The second independent variable is assumed to defined data groups - in the above example, there are therefore 3 groups. For each group, the data and corresponding error are defined as a function of the first independent variable - the first column is the data and the second the errors. This is then repeated until all the groups have been defined.
Another example ASCII Group file is included here for your inspection.

ASCII SPE

The ASCII SPE file format is also meant for storing 2D datasets. Take a look at the included example ASCII SPE file. It is also a very simple format that can store both point or histogram data.
The first line contains two integers that define the size of the second (y) and first (x) independent variables respectively. The second line is a comment which can be used to specify a label for the y variable. This is followed by a data block defining the y values - the number of values in each line is expected to be 8 and the number of lines is determined by the number of y values specified in the first line of the file. Next the x variable is defined in a similar way - a comment line followed by a data block containing the x values.
The remainder of the file conatins the dependent data and error in the data. The data are arranged in blocks called groups - each group consists of one data and one error block and the number of groups is defined by the size of the y variable. A data block consists of a comment line followed by a section of data, with no more that 8 data points per line. Note that the data are NOT space delimited but are formated using a G10.4 format code - that is each number occupies a fixed width of up to 10 characters with 4 significant digits displayed. For example the floating point number -1234567.12 will be formated as -1.235E+06.

DAVE

The DAVE data format is a binary file format that is stored on disk as an IDL sav file. It is fairly complicated and is fully described in DAVE1x_dataformat.pdf. The document also describes DAVE library functions that can be used to create and manipulate a DAVE format data structure in memory.