Writing tests for the code is a good practice in any programming language. It allows to write and debug code quickly with covering many possible cases ranging from the most common to edge cases. Moreover, code covered by tests can be modified without worrying to break something in other places where it is used.
Also, tests serve as excellent documentation of what is expected at outputs with specific parameters at inputs.
In XOD node testing implemented as tabular tests (tabtests for short). A test suite looks like a table. The first row contains pin labels of a tested node. All the next rows are test cases: a row is one test case and one transaction of the XOD program. In cells, you put input values and their expected outputs for the given pins.
Here’s how a typical tabtest looks like:
Input values and expected outputs defined by tabtest literals, which generally is the same as literals in Inspector, but have some special literals. See the full list of tabtest literals in the reference.
When you run tests, they either pass or fail with an error describing the difference between actual and expected outputs.
Nodes with such primitive types as strings, numbers, and booleans are simplest to test. Let’s imagine that there is no
to-percent node in the XOD standard library, and we implement it by ourselves.
It’s a node that converts some number into a string with the percent character concatenated. However, what’s in the output if it takes a
NaN on the input, or
First of all, place a new marker node:
xod/patch-nodes/tabtest. Now double-click the
tabtest node. An embedded tabtest editor opens up which looks like a table. Let’s write test cases:
|// Test rounding|
|// Negative values|
|// Special values|
If you save the project as a multi-file project, you can find out
patch.test.tsv file in the directory of your patch. So you can write it manually if you like. Pay attention that columns are separated by tabs (
All right, we just wrote the tests. It’s time to run them.
Press the “Run tests” button in the panel above and wait for a little. It can take a few seconds because these tests are transpiling to the C++ code, then compile in the cloud, and then execute. You can watch for the progress in the Deployment pane below: you’ll see a progress bar. Expand the pane to see more details.
Tests failed! Let’s inspect details in the “Tests” tab in deployment pane. Here are the lines that are important to us:
./to-percent.catch.inl:21: FAILED: REQUIRE( probe_PERC.state.lastValue == xod::XStringCString("255%") ) with expansion: "2550%" == "255%"
It tells us that we get value
"2550%" in the output instead of expected
"255%". Oh, it looks like we make a mistake in the test case. Let’s fix it and rerun tests.
Now you see a green notification message about passed tests. Congratulations!
You can write tabtests for the nodes with C++ implementation in the same way, just place
xod/patch-nodes/tabtest marker node and fill the table.
We just tested a pure node. That is a node which depends only on its input values when producing output. But how to deal with impure nodes, like
count, which also depend on the internal state and past?
All right, we did not tighten the nuts so that we can test these nodes too, but we have to keep it in mind when we are writing tests for them. Each subsequence test case does not reset the state of the previous test case, so if you send a pulse to the
INC pin of the
count node twice, it outputs
2 in the second case. If you want to start over — add a new test case and send
pulse to the
|// And so on...|
Do not forget to test edge cases like pulses in both pulse pins
Inf as steps in counter and so on. It helps to catch bugs, make the behavior of the node more obvious, and provide better documentation.
Some impure nodes are dependent on the time flow. For example,
delay. To test them we can mock the time.
Make a column with the magic name
__time(ms) and specify timestamps in milliseconds in this column. Such way you can travel through time.
However, also you have to know three things:
isTimedoutfunction inside or contain nodes like
xod/core/clock, are marked as timed out only at the next millisecond. For example, if we set a delay for 500 ms, it pulses
DONEon the 501 ms.
defernodes. However, when you use the
__time(ms)column you have to update it manually. So if you want to test something with at the same time point at each row, you probably do not need this column at all.
You can checkout tabtests in these nodes to understand how it looks and works:
xod/core/system-time. For example, here a part of tabtest for
|// And so on...|
We recommend placing
__time(ms) column at first place to keep it clear and see what time is it. However, you can place it in any place of the table.
To test nodes with variadics, generics or custom types do the following:
For example, you can make
test-select-4-number, which tests selecting one of the four numbers by pulses,
test-add-vectors, which makes two custom types
vectorfrom primitive input-numbers, then add one to another and then unfold it back to primitives.
Let’s look at the first example in more detail.
And some test cases:
Now you know how to write and run tests. It will help you to write more stable programs.
Moreover, this is the time to tell you about test-driven development (known as TDD). It’s a development process when a developer writes tests before writing code. It brings productivity, quality, and confidence to the development process. Also, when you write test cases, some things can become evident for you, like “it should be two small nodes instead of the huge one” or “oh, it can return invalid values and how I should deal with it?”
Also, it is a lot faster to run tests than upload code to the board after each change and test all possible cases manually.
So we recommend using TDD when you’re making some nodes that are not about hardware. Detach all the logic from the hardware to test it quickly especially as XOD does not allow to test the nodes working with hardware. For this reason, for tabtest not provided literals for pins of the port type.