ERTMS Solutions is happy to announce that we have successfully applied 30% of the Subset-076 test sequences (210 sequences) against the ERTMSFormalSpecs model. We have continued applying the process described here, and this article provides an update of our previous findings according to our recent work.
The updated statistics are the following. Over the 210 implemented tests (out of 689 in-scope):
- Thanks to our toolchain, we have fully translated, fixed, and executed 165 test sequences from Subset-076. This work is not painless though.
- We have found 563 non-blocking issues in Subset-076. These are errors or inconsistencies expressed in the tests for which we could find a clear solution.
- We have also identified 78 blocking issues in Subset-076, preventing the execution of 45 sequences. This means that, even with goodwill, the corresponding Subset-076 test sequence cannot be applied, hence, cannot prove technical conformity and functionality of the ETCS on board subsystem in its current state.
Since the last update, the issues encountered have largely been consistent with our findings detailed in our previous reports.
ERTMSFormalSpecs being a formal model, our semi-automated testing process highlights issues of precision and completeness of information. Notably, using placeholder variable names instead of using specific values, not specifying the required values for driver inputs, or lacking description of the train speed (such as putting the train in standstill when required), have required a lot of effort.
New and noteworthy, there have consistently been errors in the End of Mission, and the Start of Mission, when the train is switched off and on again. And even situations where the train is not switched off (as is should be) before switching it on again.
This work also allowed us to detect missing or conflicting information in Subset-026, Subset-027, and DMI specifications. For instance, we have encountered undefined stop conditions for System Status Messages, but also Subset-076 tests which suggested more precision for some Subset-026 requirements. In that case, we considered that Subset-076 completes Subset-026, updated the model accordingly, and we ensured traceability of these cases between the test sequences and the relevant requirements.
As already stated, ERTMSFormalSpecs is a great tool to check conformity between Subset-026 and Subset-076, but our experience also indicates that it could be a great tool for reviewing and updating test sequences to reach conformity with the newer baselines of Subset-026.
- This document presents a description of the process we apply to analyze the Subset-076 test sequences
- This document provides the results of our analysis of 20% of Subset-076 test sequences.
ERTMSFormalSpecs is a software tool, specifically designed by ERTMS Solutions to formally model and test ERTMS requirements, both for trainborne and trackside systems. The tool includes a railway signaling domain specific software language, a tool chain and a test environment.
Part of our costs on this activity are covered by a 50% grant from INEA under the Connecting Europe Facility