Author: Donald “Bucket” Costello
Date/Time: Oct 29, 2020 07:00 PM Eastern Time (US and Canada)
- Asst. Prof. Huan Xu, Chair/Advisor
- Prof. Adam Porter, Dean’s Representative
- Prof. Jeﬀrey Herrmann
- Prof. Miao Yu
- Asst. Prof. Sarah Bergbreiter
Abstract: When naval certiﬁcation oﬃcials issue a safety of ﬂight clearance, they are certifying that when the vehicle is used by a qualiﬁed pilot they can safety accomplish their mission. The pilot is ultimately responsible for the vehicle. While the naval safety of ﬂight clearance process is an engineering based risk mitigation process, the qualiﬁcation process for military pilots is largely a trust process. When a commanding oﬃcer designates a pilot as being fully qualiﬁed, they are placing their trust in the pilot’s decision making abilities during oﬀ nominal conditions. The advent of autonomous systems will shift this established paradigm as there will no longer be a human in the loop who is responsible for the vehicle. Yet, a method for certifying an autonomous vehicle to make decisions currently reserved for qualiﬁed pilots does not exist. We propose and exercise a methodology for certifying an autonomous system to complete tasks currently reserved for qualiﬁed pilots. First, we decompose the steps currently taken by qualiﬁed pilots to the basic requirements. We then develop a speciﬁcation which deﬁnes the envelope where a system can exhibit autonomous behavior. Following a formal methods approach to analyzing the speciﬁcation, we developed a protocol that software developers can use to ensure the vehicle will remain within the clearance envelope when operating autonomously. Second, we analyze ﬂight test data of an autonomous system completing a task currently re-served for qualiﬁed pilots while focusing on legacy test and evaluation methods to determine suitability for obtaining a certiﬁcation. We found that the system could complete the task under controlled conditions. However, when faced with conditions that were not anticipated (situations where a pilot uses their judgment) the vehicle was unable to complete the task. Third, we highlight an issue with the use of onboard sensors to build the situational awareness of an autonomous system. As those sensors degrade, a point exists where the situational awareness provided is insuﬃcient for sound aeronautical decisions. We demonstrate (through modeling and simulation) an objective measure for adequate situational awareness (subjective end) to complete a task currently reserved for qualiﬁed pilots.