# Register an example patch function
@register_func(EXAMPLE_PATCHES, key="new_das_patch")
def create_example_patch(argument_1='default_value'):
...
# Register an example spool function
@register_func(EXAMPLE_SPOOLS, key="new_das_spool")
def create_example_spool(another_value=None):
...
# Register an example inventory function
@register_func(EXAMPLE_INVENTORIES, key="new_das_inventory")
def create_example_inventory():
...Adding Test Data
There are a few different way to add test data to dascore. The key, however, is to ensure test files and generated patches are small (a few mb at most) so the documentation and test suite still run quickly.
Adding functions which create example data
The examples module contains several functions for creating example Patch, Spool, and Inventory instances. You can add a new function in that module which creates one, then just register the function so it can be called from dc.get_example_patch, dc.get_example_spool, or dc.get_example_inventory. These should be simple objects which can be generated within python. If you need to download a file see adding a data file.
Note
All example functions should have either no arguments or keyword arguments with default values.
The new example patches/spools can then be created via
import dascore as dc
patch_example = dc.get_example_patch("new_das_patch", argument_1="bob")
spool_example = dc.get_example_spool("new_das_spool")
inventory_example = dc.get_example_inventory("new_das_inventory")If, in the test code, the example object is used only once, just call the get_example function in the test. If it is needed multiple times, consider putting it in a fixture. See testing for more on fixtures.
Adding a data file
Of course, not all data can easily be generated in python. For example, testing support for new file formats typically requires a test file.
All test-suite data files must be hosted in dasdae’s data repo. Simply clone the repo, add your file, and push back to master or open a PR on a separate branch and someone will merge it. Files for other purposes (e.g. documentation examples) may be hosted elsewhere and still be listed in the data registry, but CI won’t cache them, so the test suite shouldn’t depend on them (add them to SKIP_DATA_FILES in tests/test_io/test_common_io.py).
Next, add your file to dascore’s data registry (dascore/data_registry.txt). You will have to get the sha256 hash of your test file, for that you can simply use Pooch’s hash_file function, and you can create the proper download url using the other entries as examples.
Note
CI doesn’t download registry files one by one; it checks out the whole test_data repo once and builds a single test-data cache shared by all operating systems (see .github/actions/prime-test-data-cache). The cache invalidates automatically whenever data_registry.txt or DATA_VERSION changes. If it ever needs a manual reset, increment the cache number in that action.
The name, hash, and url might look something like this:
jingle_test_file.jgl
12e087d2c1cd08c9afd18334e17e21787be0b646151b39802541ee11a516976a
https://raw.githubusercontent.com/dasdae/test_data/master/das/jingle_test_file.jgl
Important
Use raw.githubusercontent.com URLs, not github.com/<owner>/<repo>/raw/.... The latter is a redirect, and the redirect response carries an empty Access-Control-Allow-Origin header, so browsers refuse it. That breaks fetching data from DASCore running in WebAssembly (Pyodide, JupyterLite). tests/test_utils/test_downloader.py enforces this.
from dascore.utils.downloader import fetch
path = fetch("jingle_test_file.jgl")If you need to create a smaller version of an existing hdf5 file you can use the modify_h5_file.py in DASCore’s scripts directory. It will require some modifications, but shows how to copy and modify datasets and attributes.