Evidence review
Certification evidence review before authority submittal
A certification evidence review reads the substantiating data behind a submission and reports where it does not hold up. It is used by avionics and equipment teams who want an independent read of the package before an examiner sees it. It examines the qualification test reports, the airborne software and hardware lifecycle data, and the verification evidence against the objectives the standards set. You receive findings on the evidence that is missing, internally inconsistent, or short of the assurance level claimed, with a closure list ordered by review risk.
When this review is needed
- A package is close to submittal and the team wants a reviewer who did not build it to read the evidence.
- A program is reusing evidence from a prior effort and needs to know whether it still substantiates the current claim.
- The assigned assurance level changed and the lifecycle data has to be checked against the new objectives.
- Findings from a prior submission point at the evidence and the team needs to know which artifacts fall short.
The problem
The evidence behind a submission is produced by the people closest to the design, and they read their own data the way they intended it. Test reports cover the categories the team expected rather than the ones the installation imposes, software lifecycle data carries open problem reports that were never dispositioned, and structural coverage analysis stops short of the objective for the assigned level. An examiner reading cold sees the gaps the authors are conditioned not to.
What gets reviewed
- Qualification test reports against the environmental categories the installation requires
- Airborne software lifecycle data against the objectives of the assigned software level
- Airborne electronic hardware lifecycle data against the design assurance for the complexity involved
- Requirements-based test coverage and structural coverage to the objective for the level
- Problem reports and their disposition across the lifecycle data
- Internal consistency between requirements, design, test, and verification status
Scope this review
Tell us the asset, the event, and the evidence in scope, and we will outline a focused first engagement.
Identify what is missing against the means of compliance.
What gets validated
- Qualification reports cover every environmental category the installation environment imposes
- Software lifecycle data satisfies the objectives of the assigned software level
- Structural coverage meets the objective the level requires, with justified exceptions documented
- Hardware lifecycle data matches the design assurance expected for the hardware's complexity
- Open problem reports are dispositioned and their effect on the claim is assessed
- Reused evidence is shown to apply to the current configuration and claim
- Requirements, design, and verification artifacts tell a consistent story
Evidence normally required
- The qualification, software, and hardware lifecycle data assembled to date
- The requirements set and the verification evidence behind it
- The assigned software and hardware assurance levels and the objectives they invoke
- Open problem reports and the disposition record
- Any reused or prior-program evidence offered for credit
Common discrepancies
- Qualification gaps against the environmental categories the installation actually imposes
- Software lifecycle data short of the objectives for the assigned level
- Structural coverage that stops below the objective without a documented justification
- Open problem reports left undispositioned in the package
- Reused evidence offered for credit without showing it applies to the current configuration
What is at stake
Evidence that does not survive a cold read turns review into a sequence of clarification requests, and each request reopens an artifact the team had closed. Schedule erodes against the open items, and an article waiting on its authorization cannot ship while the data is reworked.
How the work runs
Fix the objectives
Establish the assigned software and hardware levels and the environmental categories the installation imposes.
Read the artifacts cold
Examine the qualification, software, and hardware data against those objectives as an examiner would.
Test consistency
Trace requirements through verification and check problem-report disposition and reuse arguments.
Rank the closures
Deliver findings and a closure list ordered by review risk and the effort each fix takes.
What the buyer receives
- Findings on evidence that is missing, inconsistent, or short of the assigned level
- A traceability view from requirements through verification with the gaps marked
- An assessment of any reused evidence against the current claim
- A closure list ordered by review risk and effort
Who uses the output
- Certification leads deciding whether the package is ready to submit
- Engineering teams closing the flagged artifacts
- Quality functions confirming objectives are met before release
How the work fits into the transaction or program
The review is an independent read of the substantiating data, deeper than a matrix check and focused on the artifacts themselves. It pairs with the data-support work that builds a package and with a matrix review that checks coverage.
Start with a single asset
Reduce finding cycles by checking the package first.
Regulatory limits
The review checks the applicant's evidence against the applicable standards. It does not make compliance findings on the authority's behalf, assign assurance levels, or guarantee that the evidence will be accepted.
What this review does not cover
- Making official compliance findings or assigning assurance levels
- Generating the missing qualification, software, or hardware data
- Issuing any approval or authorization
Specific to this review
- The value of an independent evidence read is the cold reviewer: the authors of substantiating data are conditioned to read it as intended, which is how undispositioned problem reports and short coverage survive an internal check.
- Structural coverage analysis under DO-178C is level-driven, so the same software can be complete at one level and short at another once the objectives change.
- Reused evidence is only as good as its applicability argument; data carried from a prior program is a frequent finding when nothing shows it still fits the current configuration.
Sources
RTCA. Environmental qualification test categories and procedures referenced by TSO and equipment qualification.
RTCA. Objectives and lifecycle data for airborne software assurance, by design assurance level (DAL A-E).
RTCA. Design assurance objectives and lifecycle data for airborne electronic hardware (FPGA/ASIC/PLD).
U.S. Government (eCFR). Type certificates, STCs (Subpart E), TSO authorizations (Subpart O), PMA (Subpart K), and export airworthiness approvals (Subpart L).
Frequently asked questions
How is this different from a compliance matrix review?
A matrix review checks that each requirement maps to a means of compliance and to current evidence. An evidence review goes into the artifacts themselves, reading the qualification, software, and hardware data against the objectives the standards set.
Relevant glossary terms
Related pages
Where this fits
Talk to an engineer who has done this work
We will walk through your current state, the records or evidence involved, and a scoped first engagement.
Walk through your situation with an engineer who has done this work.