Profiling and Benchmarks

Benchmarks

DASCore uses Airspeed Velocity (asv) to create and run a simple benchmark suite. The benchmarks are found in the benchmark folder at the top level of the repository.

The asv documentation is great, and you can do all sorts of things with the benchmarks. For the most common use case, that is to compare changes in performance on a development branch to the main branch, you can simply use two scripts in the script directory.

To run the benchmarks:

python scripts/run_benchmarks.py

And to get a simple (text-based) visualization of the differences:

python scripts/visualize_benchmarks.py

If you notice any significant degradations, it is probably worth looking into via profiling.

Profiling

You can profile the problematic benchmark(s) to see why their performance degraded. This is done with the asv profile command.

For example, suppose you notice a large increase in runtime for the time_pass_filter benchmark in the patch_benchmarks’ ProcessingSuite. You can load a nice interactive snakeviz gui (assuming you installed snakeviz pip install snakeviz) like so:

asv profile patch_benchmarks.ProcessingSuite.time_pass_filter -E existing --gui snakeviz

You can then click through the call stack and see which functions can be improved. After tweaking them, re-run the profiling and see if the overall time improves.