Technical Report: Systemic Assurance

Report Number: Technical Report SERC-2015-TR-019-1

Report Name:  Atlas 0.6: Expanding on the Theory of Effective Systems Engineers

Publication Date: July 31, 2015

Technical Report RT-119: Systemic Assurance


Systems cannot be deployed until customer organizations judge them fit for use in the mission environment. These assurance judgments must be based on evidence that a system manifests the necessary functionality and does so at a level of quality and security appropriate to the operating environment. Achieving this goal has two benefits. The first is direct: Cost-effective and rapid recertification is essential to support the development of systems that must adapt to changes in both the mission environment and the infrastructure environment. The second benefit is indirect: The prerequisites for progress towards this goal are the same as those that will, in the more general case, improve the level of assurance and the efficacy and efficiency of the processes through which it is achieved.

Assurance is a human judgment of fitness for use. This judgment is thus contextualized by the mission definition, the features of the normative operating environment, the threat landscape, and characteristics of the evolving infrastructure environment. The judgment must be based on evidence, and there are many different kinds of evidence that can be produced and managed in a large-scale systems engineering activity. The judgment must also address diverse quality attributes and ilities.

The project takes a multi-faceted approach, focusing on combines technical analysis of system artifacts and requirements with architecture techniques to promote assurance and resiliency. An important goal is to apply these techniques not just in anomaly detection, but also to support stronger possibilities for positive assurance, guaranteeing an absence of defects of certain specific kinds, for greater scalability to large systems, building on technical approaches to composability, and for more rapid execution, building on design experience. These have been successfully applied in such areas as the High Level Architecture analysis for networked DoD models and simulations, cyber-physical robotic systems, and extremely large commercial Java programs.

An important goal is to develop incrementally compostable combinations of models, practices, and tools for obtaining the most cost- and schedule-effective combinations for the assurance of necessary system properties. One analogy to exploit is with successful techniques in other domains, such as building codes. Building codes provide engineering guidance and constraint. But they are also continuously evolving, and successfully doing so under the influence of diverse and conflicting stakeholder interests. Another analogy to exploit is the idea of chains of evidence — semantic dependency modeling — to support ongoing reevaluation for rapidly evolving systems both in development and in sustainment/modernization. A third analogy is with the use of metaphors in language — how, in the engineering and evaluation of systems, do we choose to express the key design concepts and engineering abstractions? And how do these choices influence the kinds of models and analytics and the extent to which assurance judgments can be reached and at what scale of system and complexity? The abstractions come in many forms, including for example language extensions for assurance assertions or context metadata.