Is continuous integration worth the price? Yes, I’m sure.
Software development teams from different industries recognize continuous integration as a valuable practice. Some treat it as a "must-have" piece of an effective life cycle. However, in safety-critical areas and in the aerospace domain, in particular, there is still strong resistance to implementing continuous integration. Some people may think that continuous integration cannot formally fit into DO-178B\C. Some people may think that those agile things bring no practical value into strict and rigorous development process.
These fears are groundless and I can show you that a DO—178C project can benefit from continuous integration.
Continuous integration aims to pull testing activities inside the development cycle rather than saving them for last. This goal breaks down to the following tasks:
Integrate all software pieces to see how they fit each other;
Check for any regression possibly introduced by recent changes;
Check the functionality of newly implemented changes (if applicable).
These tasks should be done frequently enough to allow removal of any bugs before moving to the next stage of the life cycle.
"Any DO—178C project can benefit
from continuous integration." You will need several prerequisites to implement continuous integration:
A version control system to store and retrieve project artifacts coupled with a build tool;
Automated tests and a test execution framework;
Means to analyze test results.
Luckily, DO-178 features of heavyweight development approach provide a solid base for all three requirements.
DO-178 mandates comprehensive configuration control and change management. Every single change request and the resulting changes in artifacts should be segregated and tracked. Usually it’s only a matter of setting up a configuration management tool to automate the build process and see how each subsequent change fits into the previous build.
Requirements-based testing is one of the main ideas of DO-178 standard. Usually different groups develop code and test cases concurrently. If test strategy is established and test tools and environment are set up, nothing prevents the test group from developing test procedures along with test cases. This means that when a code change is deployed, everything is ready for continuous integration and, more important, for continuous testing.
DO-178 requires establishing thorough traceability. Usually every single function is traced to corresponding requirements. Test cases and procedures are traced to requirements as well. Trace matrices are stored in a well-structured way. Everything is in place to select an appropriate set of tests and run them against a code change. Trace matrix will also aid analysis: you can identify the broken parts almost instantly.
"DO-178 features of heavyweight
development approach provide
a solid base for continuous integration." Of course, this approach works well if testing is automated or at least semi-automated. Modern software testing toolsets from VectorCast, LDRA, Rational, and others provide a solid base for test automation. Model-based development environments open new horizons for continuous integration by software-in-the-loop and hardware-in-the-loop concepts. Even old school manual visual tests can be adjusted for continuous integration. A way to set inputs automatically almost always exists. Output from a screen can be recorded for further analysis. Thus, all tests, with few exceptions, can fit into the strategy: "Run automatically, when a change is ready, analyze results when needed."
Compliance is one of the most important questions when you develop airborne software. Some people believe that every tool should be qualified for continuous integration. It is not true. The DO-178C standard states: "Tools that are used to eliminate, reduce, or automate software life cycle process activities, and whose outputs are not verified, need be qualified." You may choose different strategies to fulfill the DO-178 requirements such as:
Automation and pass\fail selection functionality may be qualified'
Tools' outputs may be reviewed;
A combination of qualification and reviews may be used.
"It is not true that
every tool should be qualified
for continuous integration" You may also go a different way — separate continuous integration and certification tasks. You may conduct formal verification (a k a run-for-score) once at the end without using automation tools. If continuous integration is applied throughout the project, you would be sure that your code is in good shape and your tests are correct and complete. There is minimal risk of getting some sudden costly updates.
Since continuous integration is not mandated by certification authorities some may seek to avoid the additional cost. It’s your choice, but make it wisely. Time and resources spent during earlier stages of your project to introduce test automation and continuous integration will save money later. You’ll be rewarded with fewer bugs,, avoiding integration hell and decreasing the cost of changes in later stages of project (Beck's curve). Or defer testing and pay the price closer to the project deadline as you fix all those cascaded bugs. Each single change will be a challenge under such conditions (Boehm's curve). Look at those two curves and decide which one you would rather see in your project.
