Developer information¶
Database structure¶
The user has two configurations to operate in,
local database
local database + server side database (network connection needed)
The serverside configuration offers the advantage that you can access the data of your experiments from any device. The serverside is combined with a local database, in order to be able to keep operating when the network is down.
When operating only locally, no features are lost. Mainly you don’t have back up and you can’t access the data with a device that is not your measurement computer.
To accommodate a structure that can dynamically switch between both modes, the following tables are made :
Client side¶
global_measurement_overview : contains all the local measurements performed. Used for queries when the server is unavailable (e.g. no network).
Server side¶
global_measurement_overview : same as for client side, only much bigger (this will have consequences, see further) (todo update …)
Both server and client have a identical database scheme.
Storage information¶
Measurement identification¶
For identifying measurements two types of identifiers are generated:
exp id
uuid
exp id¶
This is a id that is associated with the measurement project, set up and sample. This id always starts from 1 and counts up as you make more measurements on your sample and is unique in this space. Once you switch the sample, the counter resets, and will start back from 1.
The exp id is designed to be used when you want to quickly access data of a single of single sample.
uuid¶
This is a 64bit identifier that is designed to be unique among all measurements in the database. The goal of the ID is to have a relatively easily type-able id that can be used over all the measurements.
The unique ID’s are generated by concatenating the current time (epoch time in ms) + ~ the last 27 bits of your mac address. This results in a 15 digit number which should be unique for every measurement
raw data storage in the database¶
The raw data of the measurements is stored in so called large objects (postgres type). This are large binary object that are directly saved to your harddrive (much like your files on your computer). A pointer to the location of these files is kept in the database.
When the dataset is constructed a buffer (large object), is made for each setpoint and measured variable. When performing a measurement, the buffers are filled in the RAM. To enable real time plotting, these buffers are synchronized every 200ms to the local database. Only the data that has been modified in the buffer will be written. The same counts for the read buffer. The synchronization to the server is slower to ensure low overhead. This is typically done every few seconds (= slower online liveplotting).
The size of a dataset can be arbitrary, where the main limitation is the amount of RAM in the measurement computer and the amount of hard disk space available.
Managing space on measurements computers¶
The software can make an estimation of the amount of data that has been written to the hard disk of the measurement computer. Since it is recommended to use SSD’s for fast performance, the space might be limited. Therefore the user has the option to set a soft limit on the amount of space that can be used to save data.
The software will regularly check the amount of data that has been used. When this goes over the limit, it will trash the oldest datasets that are present on the system. Only datasets that are synchronized to the server can be trashed. When asking for the data of the removed measurements, a call will be made to the server instead of the computer.
By default no limit is applied.