DataSet class and factory functions.
A container for one complete measurement loop.
May contain many individual arrays with potentially different sizes and dimensionalities.
Normally a DataSet should not be instantiated directly, but through
new_data or load_data.
location (Union[str,bool]) – A location in the io manager, or False
for an only-in-memory temporary DataSet.
Note that the full path to or physical location of the data is a
combination of io + location. the default DiskIO sets the base
directory, which this location is a relative path inside.
io (Optional[io_manager]) – base physical location of the DataSet.
Default DataSet.default_io is initially DiskIO('.') which
says the root data directory is the current working directory, ie
where you started the python session.
arrays (Optional[List[qcodes.data.data_array.DataArray]) – arrays to add
to the DataSet. Can be added later with self.add_array(array).
formatter (Optional[Formatter]) – sets the file format/structure to
write (and read) with. Default DataSet.default_formatter which
is initially GNUPlotFormat().
write_period (Optional[float]) – Only if mode=LOCAL, seconds
between saves to disk. If not LOCAL, the DataServer handles
this and generally writes more often. Use None to disable writing
from calls to self.store. Default 5.
Add one DataArray to this DataSet, and mark it as part of this DataSet.
Note: DO NOT just set data_set.arrays[id] = data_array, because
this will not check if we are overwriting another array, nor set the
reference back to this DataSet, nor that the array_id in the array
matches how you’re storing it here.
data_array (DataArray) – the new array to add
ValueError – if there is already an array with this id here.
Update DataSet.metadata with additional data.
new_metadata (dict) – new data to be deep updated into the existing metadata
The value fn is a callable accepting no
arguments, and key is a name to identify the function and help
you attach and remove it.
In DataSet.complete we call each of these periodically, in the
order that they were attached.
Note that because this is a class attribute, the functions will apply to every DataSet. If you want specific functions for one DataSet you can override this with an instance attribute.
Periodically sync the DataSet and display percent complete status.
Also, each period, execute functions stored in (class attribute)
self.background_functions. If a function fails, we log its
traceback and continue on. If any one function fails twice in
a row, it gets removed.
delay (float) – seconds between iterations. Default 1.5
Return default parameter array
paramname (str) – Name to match to parameter name. Defaults to ‘amplitude’
array corresponding to the default parameter
See also
default_parameter_name
Return name of default parameter for plotting
The default parameter is determined by looking into metdata[‘default_parameter_name’]. If this variable is not present, then the closest match to the argument paramname is tried.
paramname – Name to match to parameter name
Name of the default parameter, or None if no parameter is found
A list of names (strings) of dictionaries
which are (or will be) attributes of self, whose keys should
be treated as attributes of self.
Mark the DataSet complete and write any remaining modifications.
Also closes the data file(s), if the Formatter we’re using
supports that.
filename (Optional[str]) – The file name (minus extension) to write to. The location of the file is the usual one.
write_metadata (bool) – Whether to save a snapshot. For e.g. dumping raw data inside a loop, a snapshot is not wanted.
Get the fraction of this DataSet which has data in it.
fraction_complete() values, independent of the individual
array sizes. If there are no measured arrays, returns zero.
float
Convert the dataset to an xarray DataSet
Get the metadata for a single contained DataArray.
array_id (str) – the array to get metadata for.
metadata for this array.
dict
Find changes since the last sync of this DataSet.
synced_indices (dict) – {array_id: synced_index} where
synced_index is the last flat index which has already
been synced, for any (usually all) arrays in the DataSet.
array_id for each array with changes,values are dicts as returned by DataArray.get_changes
and required as kwargs to DataArray.apply_changes.
Note that not all arrays in synced_indices need be
present in the return, only those with changes.
Dict[dict]
Read the whole DataSet from storage, overwriting the local data.
Read the metadata from storage, overwriting the local data.
Remove an array from a dataset
Throws an exception when the array specified is refereced by other arrays in the dataset.
array_id (str) – array_id of array to be removed
Evaluate and save the DataSet’s metadata.
JSON state of the DataSet.
Insert data into one or more of our DataArrays.
loop_indices (tuple) – the indices within whatever loops we are inside. May have fewer dimensions than some of the arrays we are inserting into, if the corresponding value makes up the remaining dimensionality.
values (Dict[Union[float, Sequence]]) – a dict whose keys are array_ids, and values are single numbers or entire slices to insert into that array.
Synchronize this DataSet with the DataServer or storage.
If this DataSet is on the server, asks the server for changes. If not, reads the entire DataSet from disk.
True if this DataSet is live on the server
bool
Convert the dataset to an xarray Dataset
Writes updates to the DataSet to storage. N.B. it is recommended to call data_set.finalize() when a DataSet is no longer expected to change to ensure files get closed
write_metadata (bool) – write the metadata to disk
only_complete (bool) – passed on to the match_save_range inside self.formatter.write. Used to ensure that all new data gets saved even when some columns are strange.
filename (Optional[str]) – The filename (minus extension) to use. The file gets saved in the usual location.
Write a new complete copy of this DataSet to storage.
path (Optional[str]) – An absolute path on this system to write to.
If you specify this, you may not include either io_manager
or location.
io_manager (Optional[io_manager]) – A new io_manager to use with
either the DataSet’s same or a new location.
location (Optional[str]) – A new location to write to, using
either this DataSet’s same or a new io_manager.
Convert QcodesDataSet to dictionary.
data_set – The data to convert.
include_data – If True then include the ndarray field.
include_metadata – If True then include the metadata.
Dictionary containing the serialized data.
Load an existing DataSet.
location (Optional[str]) – the location to load from. Default is the
current live DataSet.
Note that the full path to or physical location of the data is a
combination of io + location. the default DiskIO sets the base
directory, which this location is a relative path inside.
formatter (Optional[Formatter]) – sets the file format/structure to
read with. Default DataSet.default_formatter which
is initially GNUPlotFormat().
io (Optional[io_manager]) – base physical location of the DataSet.
Default DataSet.default_io is initially DiskIO('.') which
says the root data directory is the current working directory, ie
where you started the python session.
A new DataSet object loaded with pre-existing data.
Create a new DataSet.
location (Optional[Union[str,Callable, Bool]]) –
If you provide a string, it must be an unused location in the io manager. Can also be:
a Callable location provider with one required parameter
(the io manager), and one optional (record dict),
which returns a location string when called
False - denotes an only-in-memory temporary DataSet.
Note that the full path to or physical location of the data is a
combination of io + location. the default DiskIO sets the base
directory, which this location is a relative path inside.
Default DataSet.location_provider which is initially
FormatLocation()
loc_record (Optional[dict]) – If location is a callable, this will be
passed to it as record
name (Optional[str]) – overrides the name key in the loc_record.
overwrite (bool) – Are we allowed to overwrite an existing location? Default False.
io (Optional[io_manager]) – base physical location of the DataSet.
Default DataSet.default_io is initially DiskIO('.') which
says the root data directory is the current working directory, ie
where you started the python session.
arrays (Optional[List[qcodes.data.data_array.DataArray]) – arrays to add
to the DataSet. Can be added later with self.add_array(array).
formatter (Optional[Formatter]) – sets the file format/structure to
write (and read) with. Default DataSet.default_formatter which
is initially GNUPlotFormat().
write_period (Optional[float]) – seconds between saves to disk.
A new DataSet object ready for storing new data in.
Convert QCoDeS gridded dataset to xarray dataset