Trust and Artificial Intelligence - NIST.gov Draft

The artificial intelligence (AI) revolution is upon us, with the promise of advances such as driverless cars, smart buildings, automated health diagnostics and improved security monitoring. In fact, many people already have AI in their lives as “personal” assistants that allow them to search the internet, make phone calls, and create reminder lists through voice commands. Whether consumers know that those systems are AI is unclear. However, reliance on those systems implies that they are deemed trustworthy to some degree. Many current efforts are aimed to assess AI system trustworthiness through measurements of Accuracy, Reliability, and Explainability, among other system characteristics. While these characteristics are necessary, determining that the AI system is trustworthy because it meets its system requirements won’t ensure widespread adoption of AI. It is the user, the human affected by the AI, who ultimately places their trust in the system.

The study of trust in automated systems has been a topic of psychological study previously. However, artificial intelligence systems pose unique challenges for user trust. AI systems operate using patterns in massive amounts of data. No longer are we asking automation to do human tasks, we are asking it to do tasks that we can’t. Moreover, AI has been built to dynamically update its set of beliefs (i.e. "learn"), a process that is not easily understood even by its designers. Because of this complexity and unpredictability, the AI user has to trust the AI, changing the dynamic between user and system into a relationship. Alongside research toward building trustworthy systems, understanding user trust in AI will be necessary in order to achieve the benefits and minimize the risks of this new technology.