Understanding and influencing decisions
Adapt to content and context
Description
Analyse people’s text, speech or behaviour to tailor experiences in ways that better meet their needs and keep them safe. Always tell them when a service is analysing what they do, keep analysis neutral and users in control to ensure outcomes are fair. Treat outputs as tentative signals that can be corrected by the user.
For example, a user’s message might indicate specific needs or sensitive circumstances that mean a case should be automatically escalated to a human. These can be respected and validated through follow up questions and clear next steps.
IF thinks this pattern is useful for creating tailored experiences that provide care and dignity for people using a product or service. The pattern can also help uphold legal compliance.
Advantages
- Carefully analysing language-based signals has been proven to effectively prioritise people’s access to services
- Contextual cues such as location, time of day and others can be used to protect people from harm
- People can use their own language to communicate with services in an accessible and inclusive way
- Established guidance for natural language processing and automated decision making ensures experiences can be consistent in regulated sectors
Limitations
- Without guardrails there is room for misinterpretation of the words people use or introduction of bias
- False positives may occur which negatively impact user experience and trust
- Inferring vulnerability or mental state by analysing user interactions should be avoided, particularly in regulated sectors, and the resulting information should not be used for any other purposes
Examples
-
Analyses tone and sentiment to automatically triage customer service calls
-
Makes late-night riders take a reaction test, aimed at deterring drunk driving
-
Prevents fraud by showing a contextual message in-app that confirms whether they are calling you if you're actively on an voice call
Related patterns
Was this pattern useful?