Status: proposed convention for new and rewritten test cases under tests/robot/. Independent of DESIGN.md/SIM_INTERFACES.md (applies to any test case, not only ones using the new message interface) but the worked example below uses that interface to show both together.
1. The template
Description:
- Tested requirement(s): <REQ-ID(s) this test case exercises - see
docs/requirements/SRS.md (safeAPIFreamwork), same convention
production code comments already use>
- Overall description: <one paragraph - what real-world scenario this
test case represents and why it matters>
- Test case:
> TestStep <n>: <what this step does>
TestExpectation: <what must be true afterward for the step to
have succeeded>
> TestStep <n+1>: ...
TestExpectation: ...
(one pair per step, listed here as a short spec/summary a reader
can review without reading the implementation)
Preconditions:
<state the test assumes already holds before TestStep 1 runs, and how
it gets there - e.g. "train already connected and has a valid MA of
500m (see rbc_scenario/ma_tests/il_grants_a_movement_authority.robot,
numeric-prefix-ordered before this one)" or "stack freshly brought up,
no route granted yet">
TestStep 1: <action>
TestExpectation 1: <assertion>
TestStep 2: <action>
TestExpectation 2: <assertion>
...
Cleanup:
<state restored so the NEXT test case is not affected - e.g. "route
set back to initial state", "train disconnected", "no cleanup needed,
next test's own numeric ordering already assumes this one ran">
2. Mapping the template onto real .robot syntax
Robot Framework has no built-in "TestStep N / TestExpectation N" construct - this template maps onto standard, already-idiomatic RF building blocks, not a new mechanism:
| Template section | RF construct | Why |
| Description | [Documentation] | RF's own designated place for exactly this - requirement/rationale prose that is not itself executable, shown in the log/report without being test logic. Keep the TestStep n / TestExpectation n list here even though the executable steps below repeat the split - this copy is the reviewable spec; the steps below are the implementation of it. |
| Preconditions | [Setup] (a keyword call, e.g. [Setup] Verify Train Has Valid MA ${TRAIN_1} 500) | [Setup] failing is reported as a distinct status from the test body failing - exactly the right distinction between "the state this test assumes was never true" and "this test found a real defect." If the precondition is guaranteed by test ordering instead (numeric-prefix files, rbc_scenario/'s own existing convention), say so in prose here rather than re-asserting it - don't duplicate a check another file already owns. |
| TestStep n | One keyword call (ideally one well-named custom keyword whose own name states the action) | Prefer a keyword whose name IS the step, e.g. Command Train To Report Position ${TRAIN_1} 500, over a generic call with a comment on top - the log/report then reads as the step list without needing comments to carry the meaning. |
| TestExpectation n | The assertion inside/immediately after that same step's keyword | In RF, a step "has an expectation" simply by being a keyword that fails (via Should Be Equal, Verify Packet, etc.) when the expectation is not met - there is no separate mechanism needed. Where the expectation is a distinct, separately-nameable fact (most cases), give it as its OWN following keyword call rather than folding it silently into the action's own keyword, so a failure log line names which of the two (action vs expectation) actually failed. |
| Cleanup | [Teardown] | Already the established convention in this exact codebase - fault_injection/*.robot's own [Teardown] Wait Until Stack Healthy is precisely this. [Teardown] runs even if the test body failed, which is the whole point of a cleanup step - use it for exactly that, not a plain trailing keyword call at the bottom of the test body (which would be skipped on failure). |
3. Worked example
Rewriting rbc_scenario/ma_tests/il_extends_the_movement_authority.robot (today's actual log-scraping version) in this template, using the new message-interface keywords from DESIGN.md/SIM_INTERFACES.md.
3.1 Filled-in template (the spec, reviewable without reading code)
Description:
- Tested requirement(s): REQ-RBC-007 (`granted_length`/`ma_seq` are
part of the cross-compared session state this behavior updates -
docs/requirements/SRS.md). NOTE: checked against the actual SRS
while writing this example - there is currently no REQ-ID
specifically for "a second ROUTE_ADD is a running sum, not a fresh
grant" or "CTC sees CTC_MA_EXTENDED, not a second CTC_MA_GRANTED"
(channel_ab.c's own `post_execute()` implements both, ADR-029
section 2.3, but neither ever got its own SRS line). This is a real
example of what this template is *for*: writing the test case
forced noticing the gap rather than citing a plausible-looking but
wrong REQ-ID - if this test case is actually added, add the missing
REQ-ID to the SRS first (CLAUDE.md: "update the spec first when a
requirement's wording changes, then the code comment"), then cite
it here for real.
- Overall description: A train that already has a granted Movement
Authority receives a SECOND commanded route from its IL. The RBC
must add the new route's length to the EXISTING granted length
(not replace it), send the train an updated M3 reflecting the new
total, and indicate CTC_MA_EXTENDED (not CTC_MA_GRANTED again) to
CTC.
- Test case:
> TestStep 1: Command il-west to add a second route for train 1
TestExpectation: il-west's control port accepts the command
> TestStep 2: Wait for train 1's own M3
TestExpectation: ma_length is the running sum (1000m, not a
fresh 500m) and ma_seq has advanced (2, not still 1)
> TestStep 3: Wait for CTC's own indication for train 1
TestExpectation: CTC received CTC_MA_EXTENDED (not
CTC_MA_GRANTED) with the same 1000m/seq 2
Preconditions:
Train 1 already connected and already holds a granted MA of 500m
(ma_seq=1) - guaranteed by this file's own numeric ordering, running
after il_grants_a_movement_authority.robot in the same stack
lifetime (see that file's own header for why a numeric prefix, not
filename wording, is what actually guarantees this).
TestStep 1: Command il-west to add a second route for train 1
TestExpectation 1: il-west's own control port accepts the command (no
ERR reply)
TestStep 2: Wait for train 1's own M3
TestExpectation 2: ma_length = 1000, ma_seq = 2
TestStep 3: Wait for CTC's own indication for train 1
TestExpectation 3: CTC_MA_EXTENDED received (not CTC_MA_GRANTED) with
ma_length = 1000, ma_seq = 2
Cleanup:
None required - 04 (if/when it exists) would assume this file's own
1000m/seq 2 state, same numeric-ordering convention as today.
3.2 The corresponding .robot file
*** Settings ***
Documentation REQ-RBC-007 (granted_length/ma_seq are part of the
... cross-compared session state this behavior updates -
... docs/requirements/SRS.md). No REQ-ID currently exists
... specifically for "a second ROUTE_ADD is a running
... sum" or "CTC sees CTC_MA_EXTENDED, not a second
... CTC_MA_GRANTED" (channel_ab.c's post_execute(),
... ADR-029 section 2.3) - add one to the SRS before this
... test case is actually added, then cite it here.
...
... A train that already has a granted Movement Authority
... receives a SECOND commanded route from its IL. The RBC
... must add the new route's length to the EXISTING
... granted length (not replace it), send the train an
... updated M3 reflecting the new total, and indicate
... CTC_MA_EXTENDED (not CTC_MA_GRANTED again) to CTC.
...
... Test case:
... > TestStep 1: Command il-west to add a second route
... for train 1
... TestExpectation: il-west's control port accepts the
... command
... > TestStep 2: Wait for train 1's own M3
... TestExpectation: ma_length is the running sum
... (1000m, not a fresh 500m) and ma_seq has advanced
... (2, not still 1)
... > TestStep 3: Wait for CTC's own indication for
... train 1
... TestExpectation: CTC received CTC_MA_EXTENDED (not
... CTC_MA_GRANTED) with the same 1000m/seq 2
...
... Preconditions: train 1 already connected and already
... holds a granted MA of 500m (ma_seq=1) - guaranteed by
... this file's own numeric ordering, running after
... il_grants_a_movement_authority.robot in the same
... stack lifetime.
Resource ../supportFunctions/rbc_message.resource
*** Test Cases ***
IL Extends The Movement Authority
[Teardown] No Cleanup Needed
Log TestStep 1: Command il-west to add a second route for train 1
Send IL Message ${TRAIN_1_NID_ENGINE}
Log TestExpectation 1: il-west's own control port accepted the command
Log TestStep 2: Wait for train 1's own M3
Wait Train Message ${TRAIN_1_NID_ENGINE} M3
Log TestExpectation 2: ma_length=1000, ma_seq=2
Verify Packet ${TRAIN_1_NID_ENGINE} M3 M3 ma_length 1000
Verify Packet ${TRAIN_1_NID_ENGINE} M3 M3 ma_seq 2
Log TestStep 3: Wait for CTC's own indication for train 1
Wait Train Message ${TRAIN_1_NID_ENGINE} CTC_MA_EXTENDED
Log TestExpectation 3: CTC_MA_EXTENDED (not CTC_MA_GRANTED) with ma_length=1000, ma_seq=2
Verify Packet ${TRAIN_1_NID_ENGINE} CTC_MA_EXTENDED CTC_MA_EXTENDED ma_length 1000
Verify Packet ${TRAIN_1_NID_ENGINE} CTC_MA_EXTENDED CTC_MA_EXTENDED ma_seq 2
*** Keywords ***
No Cleanup Needed
Log Cleanup: none required - a hypothetical 04 file would assume this file's own 1000m/seq 2 state, same numeric-ordering convention as today.
(Send IL Message here takes just ${TRAIN_1_NID_ENGINE} because route_len is left unset - SIM_INTERFACES.md's own note that it defaults to SAFEAPI_EXAMPLE_MA_ROUTE_LENGTH_M, matching ADD_ROUTE's existing default behavior today.)
3.3 What this buys over today's version
- ma_seq becomes checkable at all (today's log-line assertion never exposed it - channel_ab.c's own session table always tracked it, but nothing surfaced it to a test).
- The distinction "CTC_MA_EXTENDED, not a second CTC_MA_GRANTED" is asserted by which message was received, not by hoping a specific log wording ("extended" vs "granted") never gets refactored.
- A test-log reader (log.html) sees the numbered steps and their expectations directly via the Log lines, without needing to read the .robot source to know what each block of keyword calls was supposed to prove.
4. Cleanup/[Teardown] when there genuinely is state to reset
Most of rbc_scenario/ relies on numeric-prefix ordering and deliberately does not reset state between files (02's grant is intentionally still there for 03 to extend) - for those, [Teardown] should say so explicitly (as in the worked example above) rather than be silently absent, so a reader does not have to wonder whether cleanup was forgotten or was never needed.
Where a test genuinely must not leak state (most of fault_injection/, which already does this - [Teardown] Wait Until Stack Healthy), use [Teardown] for the real reset action, e.g.:
[Teardown] Run Keywords
... Disconnect Train ${TRAIN_1_NID_ENGINE}
... AND Reset Routes To Initial State ${TRAIN_1_NID_ENGINE}