Verification, Validation And Accreditation Shortfalls For Models And Simulations


Principal Investigator: Dr. Sue O’Brien, University of Alabama Huntsville (obriens@uah.edu)

Timeframe:  September 2010 to February 2011

Category:  Systems Engineering and Systems Management Transformation


Description

Objectives: As systems become more software intensive and complex, managing their development and implementation also becomes more complex. Models in development today are isolated, domain-specific artifacts that are created throughout the design lifecycle. A mechanism is needed to integrate the design models with simulation environments in order to rapidly see the impacts as the design matures. The ability to perform verification, validation, and accreditation (VV&A) early in the modeling process and throughout the lifecycle could greatly improve the model and its contribution. To do that, a precise language would be required to model these systems in an integrated fashion to remove ambiguity and the segmented developmental lifecycle.
Objectives of this research included exploring the unique capabilities of Architectural Analysis and Design Language (AADL) for developing high confidence (verified and validated) models as part of a system development lifecycle and to determine the maturity of the AADL tools for VV&A model refinement. To show how AADL could be used to embed the Verification and Validation of architectural models into the development process, an architectural model of the Army’s Systems Integration and Test Laboratory (STIL) was developed and used as a test bed.

Approach: The following process was followed:

  • Develop a conceptual architectural model that includes the logical functional blocks of the architecture, relevant performance requirements, and traceability to the system specification.
  • Use automated tools to verify model semantics and completeness.
  •  Develop a runtime architectural model that includes the software and execution platform components of the system, relevant properties for analysis, and traceability to the conceptual architecture model and to the sources used to derive values in the model.
  • Verify the runtime architectural model for semantics, completeness, and whether the modeled architecture fulfills performance requirements specified in the conceptual architecture.
  •  Validate analyses performed on the architectural models.
  • Continually engage the stakeholders in an iterative manner to refine the research.

Significant Research Findings & Products: The benefits of using AADL were seen including the ability to perform incremental VV&A on system architectures, perform trade studies on crucial components of the system and to discern deep architectural issues. Long-term benefits include the ability to reduce overall costs as found in the SAVI costs benefits study.

Deliverables

Major Deliverables:

  • A test bed built and provided to explore how development times could be reduced by integrating verification and validation into the model development process by using the AADL
  • A demonstration of the impacts and benefits of using AADL in the verification and validation process including the ability to use the tools on DoD designs and determining the ability to migrate toolsets to the user community (learning curve, challenges, etc.)
  • Technical Report SERC-2011-TR-018-1, May 3, 2011

Publications

Publications:  none to date

Research Team

Researchers

  • Sue O’Brien, University of Alabama Huntsville
  • Russell Peak, Georgia Institute of Technology
  • Julie Fortune, University of Alabama in Huntsville
  • Philip Alldredge, University of Alabama in Huntsville
  • Lance Warden, University of Alabama in Huntsville
  • Selcuk Cimtalay, Georgia Institute of Technology
  • Andy Scott, Georgia Institute of Technology
  • Miyako Wilson, Georgia Institute of Technology
  • Brian Aikens, Georgia Institute of Technology
  • Drew Martin, Georgia Institute of Technology

 

Collaborating Institutions

Project Researchers