Artificial Intelligence algorithms (AI) and Machine Learning (ML) are used across a widening array of application domains. To trust the decisions made by these algorithms it is important for people to understand how they arrive at them.
Invention Summary:
Researchers at Rutgers University have created a novel solution that uses Bayesian Teaching for explaining the decisions made by AI algorithms. Bayesian Teaching is a model-agnostic system that samples data subsets to explain model inferences to a domain (but not necessarily technical) expert.
This solution allows a user to ask questions of a ML model (used by an AI Algorithm to make a decision) and in response it provides (e.g., displays) examples to explain the reasons for the decisions. For a use case, consider a user who wants to know why an AI algorithm in a self-driving automobile made a specific decision; our solution provides responses such as: “in similar situations it was found that the reason for this decision were: “example reason 1”, “example reason 2”, etc. The examples might be in the form of displayed images (see above) or some other form relevant to the type of application/use.
Advantages:
- Use of bayesian teaching ensures that the best sets of examples are generated according to a model.
- Supports any type of machine learning model that has probabilistic interpretation (e.g., supervised, unsupervised and reinforcement learning (including deep learning)) models.
Applications:
This invention can be used in any functional area where explanations of decisions of AI/ML algorithms are desired or necessary. Examples are:
- Enables the enforcement of the requirements of European Union General Data Protection Regulation (GDPR) compliance.
- Provides the ability to determine whether machine learning models cause decisions that discriminate based on race and ethnicity (bias detection). For example, to determine if there is bias in AI/ML applications that perform bank loan and mortgage processing, resume processing for matching candidates/positions, and targeted advertising.
Intellectual Property & Development Status:
The technology is patent pending and is currently available for licensing.
