Objective: To develop, prototype and pilot practical and relevant risk early warning methods, procedures and tools for acquisition programs in the Technology Development and Engineering and Manufacturing Development stages, using evidence extracted from standard program and system development contract data and reports. Risk early warning combines cost, schedule and system development data to assess integrated risk exposure. Risk exposure refers to conditions such that unforeseen future events, unrecognized past events, and normal variances due to uncertainty can have amplified adverse consequences. Risk exposure warning is based on understanding the causal chains from root causes to adverse outcomes, and how the effects manifest in standard program and system development data and reports. The secondary objective is to identify sources of risk that are injected prior to contract award, and evidence-based indicators of risk exposure.
- Review the literature on system development leading indicators, on risk leading indicators, and on root causes of adverse acquisition outcomes
- Examine in depth the Request for Proposal packages original source material across several major ground vehicle acquisition programs to answer the following questions:
- How are risks considered, what risks are considered, and what risk indicators are used during proposal evaluation and program execution, and in making tradeoffs among cost, schedule, and technical performance?
- What program management and system development data and report deliverables could provide data and evidence to compute Risk Leading Indicators and to calibrate Risk Estimating Relationships?
- Integrate the results of (a) and (b) to define a set of practical and relevant Risk Leading Indicators than can be computed from standard proposal and contract artifacts
- Review and coordinate the findings with TARDEC System Engineering risk team for practicality and relevance, and for transition to the Integrated Systems Engineering Framework suite of tools
- Initiate a pilot study application to a ground vehicle acquisition program to “stress test” the methods, processes, and tools, and to demonstrate value
- Identify potential risk sources and leading indicators addressing risks injected into the program prior to contract award
- Identify candidate outlier detection and regression modeling approaches suitable for Risk Estimating Relationships
- Technical Report SERC-2013-TR 040-1 QuantitativeRisk – Phase 1, May 29, 2013
- Technical Report SERC-2013-TR 040-2 Quantitative Risk – Phase 1 Final, September 3, 2013
- Technical Report SERC-2013-TR 040-3, Quantitative Risk – Phases 1 & 2, November 12, 2013
- Technical Report SERC-2015-TR 040-4, Quantitative Risk, February 24, 2015
- Technical Report SERC-2016-TR-117, Quantitative Risk – Phase 4, April 15, 2016
- Prototype risk leading indicator calculation “spreadsheet” tools
Transition: The results of this research will be applied initially by US Army TARDEC in coordination with Program Executive Office Ground Combat Systems, and by the Office of Major Program Reviews in the Office of the Deputy Assistant Secretary of Defense for Systems Engineering.
Witus, G., Bryzik, W., Umpfenbach, E., Addis, R., Tzau, J., Rizk, K., “Risk Leading Indicators for DoD Acquisition Programs”, NDIA Ground Vehicle Systems Engineering and Technology Symposium. Livonia, MI., Aug 12-14 2014
► Research Team
Current Research Team:
- Dr. Gary Witus, Wayne State University
- Dr. Mark Blackburn, Stevens Institute of Technology
Past Research Team Members:
- Dr. Roshanak Nilchiani, Stevens Institute of Technology
- Dr. Ali Mostashari, Stevens Institute of Technology
- Dr. Stan Rifkin, Stevens Institute of Technology
- Dr. Walt Bryzik, Wayne State University