Dynamic Assurance

The problem of dynamic assurance is related to developing mechanisms that can estimate the risk of operations for the CPS in a given environment and scenario. Dynamic assurance methods take the current system state, the output of the assurance monitors, the estimates of fault diagnosers in the system and the estimation of the environmental state. This dynamic assurance approach often utilizes the static design time assurance cases and provides a runtime analysis on wether the design time guarantees are still valid operationally in the given environment context.

Our work in this area has led to the development of a framework called Resonate. It computes the likelihood of unsafe conditions or system failures considering the safety requirements, assumptions made at design time, past failures in a given operating context, and the likelihood of system component failures. The system has been demonstrated in simulations using two separate autonomous system simulations: CARLA and an unmanned underwater vehicle. Please see the following publications for details.

Publications

1. C. Hartsell, S. Ramakrishna, A. Dubey, D. Stojcsics, N. Mahadevan, and G. Karsai, ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems, in 16th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2021, 2021.
   Bibtex Abstract

   @inproceedings{resonate2021,
     author = {Hartsell, Charles and Ramakrishna, Shreyas and Dubey, Abhishek and Stojcsics, Daniel and Mahadevan, Nag and Karsai, Gabor},
     title = {ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems},
     booktitle = {16th {International} Symposium on Software Engineering for Adaptive and Self-Managing Systems, {SEAMS} 2021},
     year = {2021},
     tag = {da}
   }
   

   Autonomous Cyber-Physical Systems (CPSs) are often required to handle uncertainties and self-manage the system operation in response to problems and increasing risk in the operating paradigm. This risk may arise due to distribution shifts, environmental context, or failure of software or hardware components. Traditional techniques for risk assessment focus on design-time techniques such as hazard analysis, risk reduction, and assurance cases among others. However, these static, design time techniques do not consider the dynamic contexts and failures the systems face at runtime. We hypothesize that this requires a dynamic assurance approach that computes the likelihood of unsafe conditions or system failures considering the safety requirements, assumptions made at design time, past failures in a given operating context, and the likelihood of system component failures. We introduce the ReSonAte dynamic risk estimation framework for autonomous systems. ReSonAte reasons over Bow-Tie Diagrams (BTDs), which capture information about hazard propagation paths and control strategies. Our innovation is the extension of the BTD formalism with attributes for modeling the conditional relationships with the state of the system and environment. We also describe a technique for estimating these conditional relationships and equations for estimating risk-based on the state of the system and environment. To help with this process, we provide a scenario modeling procedure that can use the prior distributions of the scenes and threat conditions to generate the data required for estimating the conditional relationships. To improve scalability and reduce the amount of data required, this process considers each control strategy in isolation and composes several single-variate distributions into one complete multi-variate distribution for the control strategy in question. Lastly, we describe the effectiveness of our approach using two separate autonomous system simulations: CARLA and an unmanned underwater vehicle.

2. S. Ramakrishna, C. Hartsell, A. Dubey, P. Pal, and G. Karsai, A Methodology for Automating Assurance Case Generation, in Thirteenth International Tools and Methods of Competitive Engineering Symposium (TMCE 2020), 2020.
   Bibtex Abstract PREPRINT

   @inproceedings{ramakrishna2020methodology,
     author = {Ramakrishna, Shreyas and Hartsell, Charles and Dubey, Abhishek and Pal, Partha and Karsai, Gabor},
     title = {A Methodology for Automating Assurance Case Generation},
     booktitle = {Thirteenth International Tools and Methods of Competitive Engineering Symposium (TMCE 2020)},
     year = {2020},
     tag = {da},
     archiveprefix = {arXiv},
     eprint = {2003.05388},
     preprint = {https://arxiv.org/abs/2003.05388},
     primaryclass = {cs.RO}
   }
   

   Safety Case has become an integral component for safety-certification in various Cyber Physical System domains including automotive, aviation, medical devices, and military. The certification processes for these systems are stringent and require robust safety assurance arguments and substantial evidence backing. Despite the strict requirements, current practices still rely on manual methods that are brittle, do not have a systematic approach or thorough consideration of sound arguments. In addition, stringent certification requirements and ever-increasing system complexity make ad-hoc, manual assurance case generation (ACG) inefficient, time consuming, and expensive. To improve the current state of practice, we introduce a structured ACG tool which uses system design artifacts, accumulated evidence, and developer expertise to construct a safety case and evaluate it in an automated manner. We also illustrate the applicability of the ACG tool on a remote-control car testbed case study.

3. S. Ramakrishna, C. Harstell, M. P. Burruss, G. Karsai, and A. Dubey, Dynamic-weighted simplex strategy for learning enabled cyber physical systems, Journal of Systems Architecture, vol. 111, p. 101760, 2020.
   Bibtex Abstract DOI

   @article{ramakrishna2020dynamic,
     title = {Dynamic-weighted simplex strategy for learning enabled cyber physical systems},
     journal = {Journal of Systems Architecture},
     volume = {111},
     pages = {101760},
     year = {2020},
     tag = {da,tool},
     issn = {1383-7621},
     doi = {https://doi.org/10.1016/j.sysarc.2020.101760},
     url = {https://www.sciencedirect.com/science/article/pii/S1383762120300540},
     author = {Ramakrishna, Shreyas and Harstell, Charles and Burruss, Matthew P. and Karsai, Gabor and Dubey, Abhishek},
     keywords = {Convolutional Neural Networks, Learning Enabled Components, Reinforcement Learning, Simplex Architecture}
   }
   

   Cyber Physical Systems (CPS) have increasingly started using Learning Enabled Components (LECs) for performing perception-based control tasks. The simple design approach, and their capability to continuously learn has led to their widespread use in different autonomous applications. Despite their simplicity and impressive capabilities, these components are difficult to assure, which makes their use challenging. The problem of assuring CPS with untrusted controllers has been achieved using the Simplex Architecture. This architecture integrates the system to be assured with a safe controller and provides a decision logic to switch between the decisions of these controllers. However, the key challenges in using the Simplex Architecture are: (1) designing an effective decision logic, and (2) sudden transitions between controller decisions lead to inconsistent system performance. To address these research challenges, we make three key contributions: (1) dynamic-weighted simplex strategy – we introduce “weighted simplex strategy” as the weighted ensemble extension of the classical Simplex Architecture. We then provide a reinforcement learning based mechanism to find dynamic ensemble weights, (2) middleware framework – we design a framework that allows the use of the dynamic-weighted simplex strategy, and provides a resource manager to monitor the computational resources, and (3) hardware testbed – we design a remote-controlled car testbed called DeepNNCar to test and demonstrate the aforementioned key concepts. Using the hardware, we show that the dynamic-weighted simplex strategy has 60% fewer out-of-track occurrences (soft constraint violations), while demonstrating higher optimized speed (performance) of 0.4 m/s during indoor driving than the original LEC driven system.