Matching Items (3)
Filtering by
- All Subjects: Verification
- Creators: Ben Amor, Heni
- Member of: Theses and Dissertations
Description
Automated driving systems are in an intensive research and development stage, and the companies developing these systems are targeting to deploy them on public roads in a very near future. Guaranteeing safe operation of these systems is crucial as they are planned to carry passengers and share the road with other vehicles and pedestrians. Yet, there is no agreed-upon approach on how and in what detail those systems should be tested. Different organizations have different testing approaches, and one common approach is to combine simulation-based testing with real-world driving.
One of the expectations from fully-automated vehicles is never to cause an accident. However, an automated vehicle may not be able to avoid all collisions, e.g., the collisions caused by other road occupants. Hence, it is important for the system designers to understand the boundary case scenarios where an autonomous vehicle can no longer avoid a collision. Besides safety, there are other expectations from automated vehicles such as comfortable driving and minimal fuel consumption. All safety and functional expectations from an automated driving system should be captured with a set of system requirements. It is challenging to create requirements that are unambiguous and usable for the design, testing, and evaluation of automated driving systems. Another challenge is to define useful metrics for assessing the testing quality because in general, it is impossible to test every possible scenario.
The goal of this dissertation is to formalize the theory for testing automated vehicles. Various methods for automatic test generation for automated-driving systems in simulation environments are presented and compared. The contributions presented in this dissertation include (i) new metrics that can be used to discover the boundary cases between safe and unsafe driving conditions, (ii) a new approach that combines combinatorial testing and optimization-guided test generation methods, (iii) approaches that utilize global optimization methods and random exploration to generate critical vehicle and pedestrian trajectories for testing purposes, (iv) a publicly-available simulation-based automated vehicle testing framework that enables application of the existing testing approaches in the literature, including the new approaches presented in this dissertation.
One of the expectations from fully-automated vehicles is never to cause an accident. However, an automated vehicle may not be able to avoid all collisions, e.g., the collisions caused by other road occupants. Hence, it is important for the system designers to understand the boundary case scenarios where an autonomous vehicle can no longer avoid a collision. Besides safety, there are other expectations from automated vehicles such as comfortable driving and minimal fuel consumption. All safety and functional expectations from an automated driving system should be captured with a set of system requirements. It is challenging to create requirements that are unambiguous and usable for the design, testing, and evaluation of automated driving systems. Another challenge is to define useful metrics for assessing the testing quality because in general, it is impossible to test every possible scenario.
The goal of this dissertation is to formalize the theory for testing automated vehicles. Various methods for automatic test generation for automated-driving systems in simulation environments are presented and compared. The contributions presented in this dissertation include (i) new metrics that can be used to discover the boundary cases between safe and unsafe driving conditions, (ii) a new approach that combines combinatorial testing and optimization-guided test generation methods, (iii) approaches that utilize global optimization methods and random exploration to generate critical vehicle and pedestrian trajectories for testing purposes, (iv) a publicly-available simulation-based automated vehicle testing framework that enables application of the existing testing approaches in the literature, including the new approaches presented in this dissertation.
ContributorsTuncali, Cumhur Erkan (Author) / Fainekos, Georgios (Thesis advisor) / Ben Amor, Heni (Committee member) / Kapinski, James (Committee member) / Shrivastava, Aviral (Committee member) / Arizona State University (Publisher)
Created2019
Description
Testing and Verification of Cyber-Physical Systems (CPS) is a challenging problem. The challenge arises as a result of the complex interactions between the components of these systems: the digital control, and the physical environment. Furthermore, the software complexity that governs the high-level control logic in these systems is increasing day by day. As a result, in recent years, both the academic community and the industry have been heavily invested in developing tools and methodologies for the development of safety-critical systems. One scalable approach in testing and verification of these systems is through guided system simulation using stochastic optimization techniques. The goal of the stochastic optimizer is to find system behavior that does not meet the intended specifications.
In this dissertation, three methods that facilitate the testing and verification process for CPS are presented:
1. A graphical formalism and tool which enables the elicitation of formal requirements. To evaluate the performance of the tool, a usability study is conducted.
2. A parameter mining method to infer, analyze, and visually represent falsifying ranges for parametrized system specifications.
3. A notion of conformance between a CPS model and implementation along with a testing framework.
The methods are evaluated over high-fidelity case studies from the industry.
In this dissertation, three methods that facilitate the testing and verification process for CPS are presented:
1. A graphical formalism and tool which enables the elicitation of formal requirements. To evaluate the performance of the tool, a usability study is conducted.
2. A parameter mining method to infer, analyze, and visually represent falsifying ranges for parametrized system specifications.
3. A notion of conformance between a CPS model and implementation along with a testing framework.
The methods are evaluated over high-fidelity case studies from the industry.
ContributorsHoxha, Bardh (Author) / Fainekos, Georgios (Thesis advisor) / Sarjoughian, Hessam S. (Committee member) / Maciejewski, Ross (Committee member) / Ben Amor, Heni (Committee member) / Arizona State University (Publisher)
Created2017
Description
There are more than 20 active missions exploring planets and small bodies beyond Earth in our solar system today. Many more have completed their journeys or will soon begin. Each spacecraft has a suite of instruments and sensors that provide a treasure trove of data that scientists use to advance our understanding of the past, present, and future of the solar system and universe. As more missions come online and the volume of data increases, it becomes more difficult for scientists to analyze these complex data at the desired pace. There is a need for systems that can rapidly and intelligently extract information from planetary instrument datasets and prioritize the most promising, novel, or relevant observations for scientific analysis. Machine learning methods can serve this need in a variety of ways: by uncovering patterns or features of interest in large, complex datasets that are difficult for humans to analyze; by inspiring new hypotheses based on structure and patterns revealed in data; or by automating tedious or time-consuming tasks. In this dissertation, I present machine learning solutions to enhance the tactical planning process for the Mars Science Laboratory Curiosity rover and future tactically-planned missions, as well as the science analysis process for archived and ongoing orbital imaging investigations such as the High Resolution Imaging Science Experiment (HiRISE) at Mars. These include detecting novel geology in multispectral images and active nuclear spectroscopy data, analyzing the intrinsic variability in active nuclear spectroscopy data with respect to elemental geochemistry, automating tedious image review processes, and monitoring changes in surface features such as impact craters in orbital remote sensing images. Collectively, this dissertation shows how machine learning can be a powerful tool for facilitating scientific discovery during active exploration missions and in retrospective analysis of archived data.
ContributorsKerner, Hannah Rae (Author) / Bell, James F. (Thesis advisor) / Ben Amor, Heni (Thesis advisor) / Wagstaff, Kiri L (Committee member) / Hardgrove, Craig J (Committee member) / Shirzaei, Manoochehr (Committee member) / Arizona State University (Publisher)
Created2019