Guidelines for Responsible AI
What you will learn
In this domain, you learn the guidelines for developing and operating AI systems in a responsible manner toward society and users. The goal is to understand the six perspectives of "responsible AI" that AWS advocates (fairness, explainability, privacy, security, robustness, governance) and the implementation tools such as SageMaker Clarify and SageMaker Model Monitor.
Key points
- The six perspectives of responsible AI - fairness, explainability, privacy, safety / security, robustness, governance
- Bias - the problem of a model making unfair predictions for particular attributes (gender, race, etc.) due to skew in the training data or design
- Explainability - the ability to show the basis for a model's predictions in a form humans can understand. It relates to the persuasiveness of decisions
- SageMaker Clarify - a tool that detects bias in data and models and provides explainability through feature importance
- SageMaker Model Monitor - monitors whether a model in production exhibits behavior different from training time (data drift)
- Privacy - it is important to design so that personal information is neither mixed into the training data nor output at inference time
- AWS AI Service Cards - cards that document the intended uses and limitations of each AWS AI service
Terms and concepts
The six perspectives of responsible AI
AWS organizes responsible AI into six perspectives: fairness (eliminating discrimination), explainability (explaining the basis for decisions), privacy (protecting personal information), safety / security (preventing malicious use), robustness (resilience to abnormal inputs), and governance (organizational management). These are guidelines to keep in mind across all AI development.
Bias and fairness
When there is skew in the training data, "bias" occurs in which the model makes unfair predictions for particular attributes (gender, race, age, etc.). For example, if past hiring data has a high proportion of men, the model may evaluate men favorably. It is prevented by rebalancing the data, evaluating with fairness metrics, and reviewing the training data.
Explainability
The ability to show, in a form humans can understand, why a model made a given prediction. In loan screening or medical diagnosis, decisions cannot be accepted without being able to explain "why it was rejected / why that diagnosis." It is often explained with SHAP values or feature importance, and SageMaker Clarify is the primary AWS tool.
SageMaker Clarify and Model Monitor
Clarify provides pre-training data bias detection, post-training model bias detection, and explainability through feature importance. Model Monitor continuously monitors whether a model in production exhibits behavior different from training time (data drift or prediction drift). Both are core tools for implementing responsible AI.
AWS AI Service Cards
AWS publishes an "AI Service Card" for each AI service, describing its intended uses, limitations, and guidelines for responsible use. For example, Rekognition's face comparison feature explicitly states limitations such as "not recommended for law enforcement use." These are important documents to review before adopting a service.
Check your understanding
Check what you have learned with 5 questions