Understanding and influencing decisions
Decision testing
Description
People can test how an AI model output changes by trying different input values before committing to the result. For example, when buying car insurance, someone could test how parking their car in different places changes the cost of the policy before they purchase it. IF thinks this pattern is useful because it exposes how automated systems make decisions without overwhelming someone with technical information. This pattern works best when combined with ways to give feedback or get support if the result doesn’t look right.
Advantages
- A low-friction way to help users understand more about how automated decisions are made that doesn’t require lots of previous knowledge.
- Helps someone test the output before they commit to a course of action.
- Allows people to test a model without using real data about themselves.
Limitations
- People might create their own theories for why outcomes change, unless it’s combined with other explanation methods. This could work against the intent of the pattern by undermining understanding and explainability.
- Could be gamed by people and lead to an increase in the number of false inputs.
Examples
-
Flock, a drone insurance company, uses this pattern in their app to help pilots understand the risk number generated by Flock’s algorithm, and act on it by using it to select different insurance plans.
-
ImageNet Roulette was an experiment published by the AINow Institute that used an algorithm trained using images of people from ImageNet to classify photos uploaded by users. The authors said:“...ImageNet contains a number of problematic, offensive, and bizarre categories. Hence, the results ImageNet Roulette returns often draw upon those categories. That is by design we want to shed light on what happens when technical systems are trained using problematic training data.”
-
Google Flights allows users to view the price for the same, or comparative flight, in a data grid view. This lets users see whether the cost is more or less, at any date in the future.
Related patterns
Was this pattern useful?