Report Number: Technical Report SERC-2015-TR-107
Report Name: Virtual Collaborative Environment for Conducting Project Design and Tests
Publication Date: 20 October, 2015
Multiple Systems Engineering (SE) toolsets are being developed by the military services to enable collaboration, requirements analysis, configuration management of data, and more extensive tradespace analyses. Two toolsets of note are USMC’s Framework for Assessing Cost and Technology (FACT) and Army Research Labs’ (ARL) Executable Architecture Systems Engineering (EASE). FACT is a framework which enables the definition of a design architecture using SysML and the ability to integrate various models; these are generally fast running models. EASE integrates various simulation tools together, and abstracts the input and configuration parameters so that non-subject matter experts can execute the simulation tools. EASE offers executions which are available simulation setups that can include one to many tools; EASE handles the data dependencies between integrated models and simulations within an execution. Standalone, EASE provides a web-based interface for running executions.
Previous work developed a REST API on top of EASE to enable the remote manipulation of the available executions.  The REST API has been consumed by FACT, including a custom interface for tying the architecture definition within FACT to the inputs/outputs of an EASE execution. The current interface only allows for a single execution at a time and although users can retrieve the EASE output artifacts, the data loop is not complete, meaning output from an EASE simulation cannot be sent into another model by FACT.
An overarching goal of RT-111 is to identify the best strategy for integrating analysis occurring within FACT with a constructive model. Initial tasking under this research effort explored the use of NAVAIR’s Architecture Management Integration Environment (AMIE) and NUWC’s Virtual World. Review of these technologies determined that they were not appropriate for realizing this goal at this time. However, the FACT-EASE interface offers an existing baseline capability for manipulating a constructive simulation from the FACT environment. This interface, however, needs to be expanded in order to become the preferred mechanism for stimulating constructive models. MARCORSYSCOM is interested in advancing the usefulness of FACT into other stages of the design cycle to include variants created in FACT within Live, Virtual, Constructive (LVC) simulations.
For this specific effort, MARCORSYSCOM and SERC will collaborate with ARL to expose OneSAF1 , CACCTUS2 and/or VBS23 and expand the link between FACT and EASE. Specific areas for expanding the link between FACT and EASE include: (1) allowing for trade studies, as opposed to just single point designs, to be submitted to EASE via the API and updating FACT’s consumption of the API to leverage this capability; (2) developing a Design of Experiment (DoE) generation tool within FACT which would inform what points need to be executed on a long-running simulation offered by EASE to adequately represent the design space; (3) providing semi-automated tools for the DoE capability developed in (2) and the analysis of the results to create fast-running surrogate models hosted within FACT.
Over the four years of FACT development, significant interest from stakeholders has been identified regarding the surrogate model generation process. Generally, this is a more manual process conducted by a subject matter expert who runs a simulation tool with a specific design from a formal DoE, conducts regression analysis, and iterates until the full domain of interest is covered and the surrogate model is producing results that are within the desired accuracy (generally 95+% accurate when compared to the base simulation model). Research and development in the area of (semi-)automating the surrogate model development process could offer a valuable capability to the FACT tool suite, but the product could also more generally be a standalone tool and documented process for creating accurate surrogate models that is available to the SE community