Software verification evidence
Structural-coverage evidence review for airborne software
A structural-coverage evidence review checks that an article's coverage analysis is the right kind, at the right strength, and honestly closed: that the coverage criterion matches the assigned software level, that the analysis was produced from requirements-based test execution, and that every coverage gap has a defensible disposition. It is used by avionics and equipment suppliers verifying airborne software against DO-178C. You receive a reviewed coverage package and a list of gaps that are unexplained, mislabeled, or resolved without sufficient rationale.
When this review is needed
- Coverage results are near closure and the team needs the gaps and their dispositions read independently.
- The software level requires a stronger coverage criterion than the analysis currently demonstrates.
- Dead, deactivated, or unreachable code has been found and the disposition rationale needs scrutiny.
- The analysis was measured from tests that were not driven by requirements and the basis is in question.
The problem
Structural coverage is meant to expose code that no requirements-based test exercises, but it is easy to satisfy on paper and hard to satisfy in substance. Coverage harvested from ad hoc tests rather than requirements-based execution measures the wrong thing, and uncovered code is too often dispositioned with a one-line note that names the gap without justifying it. The strength of the criterion, decision versus modified condition decision, is also frequently below what the level demands.
What gets reviewed
- The coverage criterion applied against the criterion the software level requires
- Whether coverage was obtained from requirements-based test execution
- Dispositions for uncovered, dead, deactivated, and unreachable code
- Traceability from coverage gaps back to the tests and requirements involved
- Tooling used to measure coverage and its qualification basis where credit is claimed
- Whether the analysis was run against source or object code consistent with the level
What gets validated
- The coverage criterion matches the criterion required at the assigned software level
- Reported coverage was produced by executing requirements-based tests, not incidental ones
- Each uncovered structure has a disposition with a rationale a reviewer can follow
- Dead and deactivated code is identified and handled per the applicable objective
- Measurement tool credit is supported by a qualification basis where claimed
- The analysis was measured at the level of representation the objective expects for the software level
Evidence normally required
- The structural coverage analysis and its supporting reports
- The requirements-based test cases, procedures, and execution results
- The software level assignment and the verification plan
- Any tool qualification data for the coverage measurement environment
Common discrepancies
- A coverage criterion below the strength the software level requires
- Credit taken from tests that were not driven by requirements
- Uncovered code dispositioned with a label but no justifying rationale
- Deactivated code present with no evidence it cannot execute in the target configuration
- Measurement tool credit claimed without qualification support
What is at stake
Weak coverage evidence means the verification argument has holes the authority can see. Reopening it late forces new test development against requirements, re-execution, and re-analysis, all on the critical path where the program can least absorb it.
Move from findings to resolution
Identify gaps against the means of compliance.
How the work runs
Check the criterion
Confirm the coverage criterion applied matches the strength the assigned software level requires.
Trace the source
Establish that reported coverage came from executing requirements-based tests rather than incidental runs.
Scrutinize the gaps
Examine each uncovered, dead, deactivated, or unreachable structure for a disposition a reviewer can follow.
Order the closure
Return the reviewed package with weak dispositions flagged and open items ordered by verification risk.
What the buyer receives
- A reviewed coverage package with each gap class confirmed or flagged
- A list of dispositions that lack sufficient rationale or mislabel the gap
- A closure note ordering the open items by verification risk
Who uses the output
- Verification leads closing the coverage analysis
- Software engineering leads adding requirements-based tests for the open structures
- Certification leads carrying the coverage argument into the summary
How the work fits into the transaction or program
The review sits downstream of the test evidence it depends on, checking that coverage was harvested from requirements-based execution rather than incidental runs. It feeds the accomplishment summary, where the coverage argument is reported as part of the objectives claimed.
Start with a single asset
Confirm requirements trace through verification.
Regulatory limits
The review checks the applicant's coverage analysis and dispositions for strength and substance. It does not make compliance findings, qualify the measurement tools, or guarantee the authority will accept the coverage argument.
What this review does not cover
- Making compliance findings on the authority's behalf
- Developing the missing tests or re-running coverage measurement
- Qualifying the coverage measurement tool itself
Specific to this review
- Coverage credited from tests not derived from requirements measures execution, not compliance, and is a frequent finding at the higher software levels.
- The required coverage criterion strengthens with the software level, so a criterion that was fine at one level can be insufficient at another.
- An uncovered structure is only closed when its disposition carries a rationale a reviewer can follow, not merely a label naming it.
- Deactivated code needs evidence it cannot execute in the target configuration; a label asserting it is deactivated does not by itself close the gap.
Sources
RTCA. Objectives and lifecycle data for airborne software assurance, by design assurance level (DAL A-E).
U.S. Government (eCFR). Type certificates, STCs (Subpart E), TSO authorizations (Subpart O), PMA (Subpart K), and export airworthiness approvals (Subpart L).
Frequently asked questions
Does the coverage criterion change with the software level?
Yes. The required criterion strengthens as the level rises, so coverage that satisfied one level can fall short at another. The review checks the criterion against the level the article was assigned.
What makes a coverage gap closed?
A gap is closed when its disposition carries a rationale a reviewer can follow, supported by evidence for dead or deactivated code where the objective requires it, rather than a one-line label naming the gap.
Relevant glossary terms
Related pages
Where this fits
Talk to an engineer who has done this work
We will walk through your current state, the records or evidence involved, and a scoped first engagement.
Walk through your situation with an engineer who has done this work.