Development environment

Setting up

In order to prepare the development environment, please follow the steps below:

  1. Install the Python 3.11 interpreter and pip package manager.

    • Optionally create a Python virtual environment with python3 -m venv venv in the project directory and activate it using generated script: . venv/bin/activate.

  2. Install all required libraries with pip3 install -r requirements-dev.txt.

  3. Install riscv64-unknown-elf binutils using your favourite package manager. On Debian-based distros the package is called binutils-riscv64-unknown-elf, on Arch-based - riscv64-unknown-elf-binutils.

  4. Optionally, install all precommit hooks with pre-commit install. This will automatically run the linter before commits.

Using scripts

The development environment contains a number of scripts which are run in CI, but are also intended for local use. They are:

run_tests.py

Runs the unit tests. By default, every available test is run. Tests from a specific file can be run using the following call (test_transactions is used as an example):

scripts/run_tests.py test_transactions

One can even run a specific test class from a file:

scripts/run_tests.py test_transactions.TestScheduler

Or a specific test method:

scripts/run_tests.py test_transactions.TestScheduler.test_single

The argument to run_tests.py is actually used to search within the full names of tests. The script runs all the tests which match the query. Thanks to this, if a given test class name is unique, just the class name can be used as an argument.

The run_tests.py script has the following options:

  • -l, --list – lists available tests. This option is helpful, e.g., to find a name of a test generated using the parameterized package.

  • -t, --trace – generates waveforms in the vcd format and gtkw files for the gtkwave tool. The files are saved in the test/__traces__/ directory. Useful for debugging and test-driven development.

  • -p, --profile – generates Transactron execution profile information, which can then be read by the script tprof.py. The files are saved in the test/__profile__/ directory. Useful for analyzing performance.

  • -v, --verbose – makes the test runner more verbose. It will, for example, print the names of all the tests being run.

lint.sh

Checks the code formatting and typing. It should be run as follows:

scripts/lint.sh subcommand [filename...]

The following main subcommands are available:

  • format – reformats the code using black.

  • check_format – verifies code formatting using black and flake8.

  • check_types – verifies typing using pyright.

  • verify – runs all checks. The same set of checks is run in CI.

When confronted with would reformat [filename] message from black you may run:

black --diff [filename]

This way you may display the changes black would apply to [filename] if you chose the format option for lint.sh script. This may help you locate the formatting issues.

core_graph.py

Visualizes the core architecture as a graph. The script outputs a file in one of supported graph formats, which need to be passed to an appropriate tool to get a graph.

The core_graph.py script has the following options:

  • -p, --prune – removes disconnected nodes from the output graph.

  • -f FORMAT, --format FORMAT – selects the output format. Supported formats are elk (for Eclipse Layout Kernel), dot (for Graphviz), mermaid (for Mermaid).

build_docs.sh

Generates local documentation using Sphinx. The generated HTML files are located in build/html.

tprof.py

Processes Transactron profile files and presents them in a readable way. To generate a profile file, the run_tests.py script should be used with the --profile option. The tprof.py can then be run as follows:

scripts/tprof.py test/__profile__/profile_file.json

This displays the profile information about transactions by default. For method profiles, one should use the --mode=methods option.

The columns have the following meaning:

  • name – the name of the transaction or method in question. The method names are displayed together with the containing module name to differentiate between identically named methods in different modules.

  • source location – the file and line where the transaction or method was declared. Used to further disambiguate transaction/methods.

  • locked – for methods, shows the number of cycles the method was locked by the caller (called with a false condition). For transactions, shows the number of cycles the transaction could run, but was forced to wait by another, conflicting, transaction.

  • run – shows the number of cycles the given method/transaction was running.

To display information about method calls, one can use the --call-graph option. When displaying transaction profiles, this option produces a call graph. For each transaction, there is a tree of methods which are called by this transaction. Counters presented in the tree shows information about the calls from the transaction in the root of the tree: if a method is also called by a different transaction, these calls are not counted. When displaying method profiles, an inverted call graph is produced: the transactions are in the leaves, and the children nodes are the callers of the method in question. In this mode, the locked field in the tree shows how many cycles a given method or transaction was responsible for locking the method in the root.

Other options of tprof.py are:

  • --sort – selects which column is used for sorting rows.

  • --filter-name – filters rows by name. Regular expressions can be used.

  • --filter-loc – filters rows by source locations. Regular expressions can be used.