Guidelines for Responsible AI practice questions

Worked examples for Guidelines for Responsible AI

1. Question 1

   A company has already configured an Amazon Bedrock guardrail that encodes its approved safety policy: a list of denied topics and filters for harmful content. It now wants to apply that same policy to two additional workloads that do not call an Amazon Bedrock foundation model — a support assistant running a self-managed open-weight model on Amazon EC2, and a workflow that sends user text to a third-party model endpoint. In both cases the team wants to evaluate the user's text and the generated response against the existing guardrail at inference time, before the text is used. Which approach correctly meets this requirement?

   1. A. This is not achievable: a guardrail only takes effect when it is associated with an inference request to a foundation model hosted in Amazon Bedrock, so both workloads must first be migrated to Bedrock-hosted models.

      Reflects the misconception that a guardrail is inseparable from a Bedrock model invocation. Guardrails are an independent resource that can also be invoked on their own to assess arbitrary text, so migrating the models is not a prerequisite.

   2. B. Fine-tune the self-managed model on examples of the prohibited topics so that refusal behaviour is trained into the model weights, because a guardrail cannot evaluate text unless a Bedrock model generates it.

      Substitutes model training for an inference-time safeguard and repeats the false claim that guardrails cannot evaluate text standalone. Fine-tuning changes the model's tendencies but provides no deterministic, centrally maintained filter, and it would have to be redone per model rather than reusing the approved policy.

   3. C. Configure Amazon SageMaker Model Monitor to scan the assistant's responses on a schedule and block any response that mentions a denied topic.

      Confuses scheduled monitoring with an inline content filter. Model Monitor runs scheduled jobs that compare captured data and prediction quality against a baseline and emit alerts; it does not sit in the request path and cannot block an individual response for content-policy reasons.

   4. D. Call the ApplyGuardrail API to evaluate the input text and the generated output against the existing guardrail independently, without invoking an Amazon Bedrock foundation model, and act on the guardrail's assessment in the application.Correct answer

      Correct. Amazon Bedrock Guardrails expose an API that assesses any text against a configured guardrail without a model invocation, which lets the same approved policy be reused for self-managed and third-party models while remaining maintained in one place.

   Explanation

   Amazon Bedrock Guardrails are a standalone resource, not merely a setting on a model call: alongside associating a guardrail with a Bedrock inference request, the service offers an API for evaluating arbitrary text against a configured guardrail without invoking any foundation model, which is exactly what lets one approved safety policy be reused for models hosted outside Amazon Bedrock. Claiming the workloads must be migrated to Bedrock-hosted models misreads guardrails as inseparable from model invocation. Fine-tuning a model to refuse topics changes only the model's learned tendencies and gives no deterministic, centrally governed filter that can be applied consistently across applications. SageMaker Model Monitor addresses a different lifecycle stage entirely — scheduled comparison of production data and prediction quality against a baseline — and is not an inline content-safety control.

2. Question 2

   An organization defines its entire responsible AI process as a single pre-deployment sign-off: before any model is released, reviewers confirm that the training data was documented and that the model met its fairness thresholds. Once that sign-off is granted, the model is considered "responsibly built" for the remainder of its production life, and no further responsible-AI activity is scheduled. Which statement best describes the gap in this approach?

   1. A. Responsible AI is a lifecycle practice, so dimensions such as fairness, safety, and veracity must continue to be addressed after deployment — real-world data and model behavior can shift, which is why deployed systems need ongoing monitoring and controls.Correct answer

      Correct. AWS presents responsible AI as core dimensions applied across the whole AI lifecycle — data, development, deployment, and ongoing operation — supported by tools that evaluate models before release and monitor and guard them in production.

   2. B. There is no gap, provided an IAM policy restricts which users and applications may invoke the deployed model, because access control is what enforces responsible AI in production.

      Confuses access control with responsible AI. IAM decides WHO may call an AWS API; it says nothing about whether the model's outputs remain fair, safe, or truthful for the users who are authorized to call it.

   3. C. The gap is that fairness and data quality can only be meaningfully assessed after a model is serving live traffic, so the pre-deployment data documentation and fairness review contribute nothing.

      Inverts the practice by dismissing pre-deployment work. Analyzing datasets and models before training and deployment is an established part of responsible AI; the flaw in the scenario is that the process stops there, not that it started there.

   4. D. The gap is that the reviewers measured fairness bias when they should have measured statistical bias, since a high-bias model is by definition an unfair model.

      Conflates two different meanings of "bias." Statistical bias in the bias–variance sense describes underfitting by an overly simple model; fairness bias describes unfair skew against a group. A high-bias (underfitting) model is not automatically an unfair one, and measuring one does not substitute for the other.

   Explanation

   AWS frames responsible AI as a set of core dimensions — including fairness, safety, robustness, and veracity — that are applied continuously across the AI lifecycle rather than certified once, which is why the tooling spans pre-deployment analysis, inference-time safeguards, and production monitoring. Treating access control as the responsible-AI control mistakes authorization for output quality: restricting who may invoke a model does not make its responses fair or truthful. Dismissing pre-deployment review inverts the practice, because dataset and model assessment before release is a genuine part of it. Substituting statistical bias for fairness bias confuses underfitting with unfair group-level outcomes, which are unrelated measurements.

3. Question 3

   An insurance company fine-tunes and hosts its OWN claims-triage model on Amazon SageMaker. Its governance board wants two things before launch: (1) a single durable record of the model's intended uses, risk rating, training details, and evaluation results that auditors can review over the model's lifetime, and (2) the ability to show a reviewer which input features contributed most to an individual claim's score. Which TWO actions address these needs? (Select TWO.)

   1. A. Request an AWS AI Service Card from AWS that documents this fine-tuned claims-triage model.

      Misunderstands who authors AI Service Cards and what they cover: AWS publishes them for AWS's own AI services, describing those services' intended use cases, design choices, and limitations. They are not authored for a customer's self-trained or fine-tuned model.

   2. B. Create an Amazon SageMaker Model Card for the model that records its intended uses, risk rating, training details, and evaluation results as an auditable artifact.Correct answer

      Correct for the documentation need. SageMaker Model Cards are the customer-authored transparency artifact for a model you train or tune yourself, capturing intended uses, risk rating, training and evaluation details in one record that travels with the model for governance and audit.

   3. C. Encrypt the model artifacts at rest and tighten IAM policies on the inference endpoint.

      Confuses security and privacy controls with transparency and explainability. Encryption and access control protect the model and its data but reveal nothing about intended use, limitations, or why a given claim was scored the way it was.

   4. D. Generate feature attribution explainability reports with Amazon SageMaker Clarify so reviewers can see each input feature's relative contribution to an individual prediction.Correct answer

      Correct for the per-decision need. Feature attribution is the explainability tooling that shows how much each input feature pushed a specific prediction, which documentation alone cannot supply.

   5. E. Increase the model's size and layer count, since larger and more complex models are inherently easier to explain.

      Inverts the interpretability tradeoff. Added capacity may raise accuracy, but larger and more complex models are generally harder to interpret, not easier, which is why explainability tooling is needed in the first place.

   Explanation

   Two different responsible-AI needs are in play, and each has its own artifact. For a model the customer trains or tunes, the transparency record — intended uses, risk rating, training details, evaluation results — belongs in a SageMaker Model Card; AWS AI Service Cards are AWS-authored documentation for AWS's own AI services and do not cover customer models. For explaining one prediction, feature attribution reporting shows the relative contribution of each input feature, which no static document provides. Encryption and IAM address security rather than transparency or explainability, and scaling a model up makes it harder, not easier, to interpret.

4. Question 4

   During a model review, two people use the word "bias" to mean different things. A data scientist reports that the model has "high bias" and is performing poorly on both the training and validation sets. A responsible AI reviewer, separately, says the model shows "bias" because it approves applications from one demographic group at a much lower rate than others. Which statement correctly distinguishes these two uses of the term?

   1. A. They describe the same underlying problem, so raising the model's capacity until training and validation error drop will also remove the disparity between demographic groups.

      Conflates statistical bias with fairness bias. Fitting the data better can just as easily reproduce a historical disparity more faithfully; accuracy improvements carry no guarantee about group-level outcome differences.

   2. B. Statistical bias in the bias–variance sense means the model is too simple and underfits, which is an accuracy concern; fairness bias means the system systematically disadvantages a group and is assessed with dedicated bias metrics such as those SageMaker Clarify computes.Correct answer

      Correct. High bias in the bias–variance decomposition is underfitting, evidenced by poor performance on training and validation data alike; fairness bias is an imbalance in outcomes across groups, which SageMaker Clarify measures with pre-training and post-training bias metrics.

   3. C. In the bias–variance sense, high bias means the model is overly sensitive to the training data and memorizes it, while fairness bias means the model is too simple to represent minority groups.

      Swaps bias and variance. Over-sensitivity to training data and memorization describe high variance (overfitting); high bias is underfitting. It also misdefines fairness bias, which is about disparate outcomes rather than model capacity.

   4. D. Fairness bias originates only in the training data, so once the dataset has been balanced before training, no further fairness assessment of the trained model or its predictions is required.

      Frames responsible AI as a one-time data checklist. Bias can also be introduced by feature choice, labels, proxies, and the learning process itself, which is why SageMaker Clarify computes post-training bias metrics on model predictions in addition to pre-training data metrics.

   Explanation

   The two senses of "bias" are unrelated diagnostics. In the bias–variance decomposition, high bias means the model is too simple for the problem and underfits, which shows up as weak performance on training and validation data together; the opposite failure, over-sensitivity to the training set, is high variance. Fairness bias instead means the system produces systematically worse outcomes for a group, and it is quantified separately using bias metrics on both the dataset and the model's predictions, as Amazon SageMaker Clarify does. Treating the two as one problem is wrong because a better-fitting model can reproduce historical disparities just as faithfully; swapping the definitions of bias and variance inverts the underfitting/overfitting distinction; and assuming a balanced dataset ends the obligation ignores that bias can enter through features, labels, proxies, and the learning process, which is why post-training bias assessment exists.

5. Question 5

   A team is mapping its generative AI review checklist onto the core dimensions of responsible AI that AWS publishes. During testing, the team records two related problems with a deployed assistant: it sometimes fabricates citations that cannot be traced to any source document, and it starts producing garbled or off-policy answers when users submit malformed or deliberately adversarial prompts. Which single AWS responsible AI dimension is defined as achieving correct system outputs, even with unexpected or adversarial inputs, and therefore covers both of these problems?

   1. A. Safety

      Represents the misconception that any undesirable model behaviour is a safety issue. In the AWS dimension list, safety is about avoiding harmful system output and misuse; an untraceable citation is factually wrong rather than harmful, and resilience to malformed input is not what the safety dimension names.

   2. B. Veracity and robustnessCorrect answer

      Correct. AWS lists this as one combined dimension, defined as achieving correct system outputs even with unexpected or adversarial inputs. Fabricated citations are a failure of output correctness and degradation under malformed or adversarial prompts is a failure under unexpected input — both fall inside this one dimension.

   3. C. Controllability

      Confuses oversight mechanisms with output quality. Controllability is about having mechanisms to monitor and steer AI system behaviour; it describes the operator's ability to intervene, not whether the generated answer is factually correct or holds up under adversarial prompts.

   4. D. Explainability

      Confuses understanding a system's outputs with the outputs being correct. Explainability concerns understanding and evaluating how a system arrives at its outputs; a model can be fully explainable while still hallucinating citations and breaking down on adversarial input.

   Explanation

   AWS describes responsible AI along a set of core dimensions, and veracity and robustness is a single combined dimension defined as achieving correct system outputs even with unexpected or adversarial inputs. Both symptoms described — hallucinated, ungrounded content and degraded behaviour on malformed or hostile prompts — are instances of that one dimension rather than of separate ones. Safety in this list targets harmful output and misuse, controllability targets the mechanisms to monitor and steer behaviour, and explainability targets understanding and evaluating outputs; none of those is about whether the output is correct under difficult input.

6. Question 6

   A logistics company's ML platform team trains and hosts dozens of its own models on Amazon SageMaker. Models are retrained on a regular cadence, and their evaluation results and known limitations change over time. The governance board asks the team how model documentation should be handled operationally so that it remains useful for internal risk review and external audit. Which practice BEST reflects AWS guidance on model documentation for models an organization builds itself?

   1. A. Create a model card when the model is designed and maintain it as a living record — intended uses, training and evaluation details, risk rating, and known limitations — updating it as the model is retrained or its risk profile changes.Correct answer

      Correct. Model documentation is meant to be a single, durable source of truth maintained across the model's lifecycle, so reviewers and auditors always see the intended use, evaluation evidence, and limitations of the version actually in production (AWS Responsible AI — transparency and governance dimensions).

   2. B. Wait until the model is approved for production, then write the documentation once as a compliance sign-off artifact and archive it unchanged.

      Treats model documentation as a one-time launch gate. Because retraining changes training data, evaluation results, and limitations, a frozen document quickly misrepresents the deployed model — the opposite of transparency, which depends on stakeholders receiving accurate, current information about the system.

   3. C. Rely on AWS to publish documentation for these models, since AWS produces the responsible-AI documentation for models running on its infrastructure.

      Confuses AI Service Cards with customer-owned model documentation. AWS authors AI Service Cards for AWS-developed AI services; when a customer trains and hosts its own model, documenting that model's intended use, data, evaluation, and limitations is the customer's responsibility.

   4. D. Replace the model documentation with the production endpoint's live latency, throughput, and error-rate dashboards, since operational metrics are the most current description of model behavior.

      Confuses operational monitoring with transparency documentation. Runtime metrics describe service health, not intended use, training data, evaluation results, or known limitations — the information stakeholders need to judge whether the model is appropriate for a use case.

   Explanation

   Transparency in AWS's responsible AI guidance means giving stakeholders accurate information about how a system was built, what it is intended for, and where its limits lie, so they can make informed decisions about using it. For models an organization trains itself, that information belongs in model documentation created early and kept current across retraining and risk-profile changes, rather than frozen at launch, delegated to AWS (which authors service documentation only for its own AI services), or substituted with operational health dashboards that say nothing about intended use or limitations.

7. Question 7

   A healthcare startup is assembling a training dataset for a triage-assistance model. Reviewers note that the collected records come almost entirely from three large urban hospitals, so several patient populations the product will serve are barely represented. Which responsible-AI dimension does this specific problem most directly represent, and what is the appropriate response?

   1. A. Robustness — the model is underfitting because the algorithm is too simple, so a higher-capacity architecture should be selected

      Conflates fairness bias with statistical bias in the bias–variance sense. High statistical bias means an over-simple model underfits; here the problem is who is present in the data, and a larger model trained on the same skewed records reproduces the same gaps.

   2. B. Veracity — the model may produce factually unsupported statements, so output responses should be grounded against source documents

      Confuses veracity (truthfulness of generated content relative to sources) with representational coverage of the training population. Grounding outputs does nothing about patient groups missing from the data.

   3. C. Safety — the deployment risk should be contained by applying identity-based access policies that limit which clinicians can query the model

      Confuses access control with a data-representativeness problem. Restricting who may call the model does not change the fact that the model was trained on data that under-represents parts of its intended population.

   4. D. Inclusivity and fairness — the dataset is unrepresentative, so coverage of under-represented groups should be broadened and outcome disparities across those groups measured before trainingCorrect answer

      Correct. Responsible datasets must reflect the populations the system serves; skewed collection is a fairness/inclusivity defect addressed by broadening data coverage and measuring pre-training imbalance across groups, which is what SageMaker Clarify's pre-training bias metrics are designed to quantify.

   Explanation

   Fairness and inclusivity in responsible AI begin with the dataset: training data that omits populations the system will serve produces systematically worse outcomes for those groups, and no downstream tuning fully repairs it. The remedy is to broaden data collection so the dataset reflects the served population and to measure outcome imbalance across groups before training — the pre-training bias metrics that Amazon SageMaker Clarify computes on a dataset. A too-simple model causing underfitting is statistical bias in the bias–variance sense, a distinct problem from unfair skew; veracity concerns whether generated statements are supported by their sources rather than who appears in the data; and identity-based access policies decide who may invoke the model, leaving the representational gap untouched.

8. Question 8

   A company is writing its internal standard for how responsible AI practices apply to machine learning projects. One proposal is that a project should complete a single responsible AI sign-off immediately before launch, after which the model is considered compliant for the rest of its production life. A second proposal is that responsible AI work is confined to the data-collection phase, because any unfairness originates in the data. The company's chief architect asks which view correctly reflects AWS guidance on developing responsible AI systems. Which statement is the most accurate?

   1. A. Responsible AI is a one-time pre-launch sign-off; once a model passes its fairness and accuracy review, no further responsible AI activity is required unless the model is retrained.

      This is the 'responsible AI is a checklist' misconception. A model that was fair and accurate at launch can drift as real-world data changes, so a single pre-launch gate cannot keep it fair or accurate; continuous monitoring of deployed models is part of the practice.

   2. B. Responsible AI is solely a data-collection concern; if the training dataset is balanced and representative, the deployed system cannot behave unfairly or unsafely.

      This is the 'fix the data and you are done' misconception. Representative data reduces one source of unfairness, but modelling choices, threshold selection, adversarial or unexpected inputs, and post-deployment drift can all introduce unfair or unsafe behaviour that data curation alone never catches.

   3. C. Responsible AI is enforced entirely by IAM policies and encryption controls, so a project satisfies it by restricting who can invoke the model and encrypting the data at rest and in transit.

      This confuses access control and data protection with responsible AI. IAM decides who may call an AWS API and encryption protects data confidentiality; neither measures fairness, detects bias drift, filters harmful model output, or explains a prediction.

   4. D. Responsible AI applies across the lifecycle — data preparation, model development and evaluation, and ongoing operation after deployment — with different capabilities suited to each stage, such as pre-training data bias analysis, post-training bias and explainability reports, inference-time safeguards, and continuous monitoring of deployed models.Correct answer

      Correct. AWS positions responsible AI as spanning the ML lifecycle, and SageMaker Clarify explicitly supports bias detection at data-preparation time, after training, and on the deployed model over time, while Guardrails and Model Monitor cover the operational stage.

   Explanation

   AWS treats responsible AI as an ongoing practice that spans the machine learning lifecycle rather than a single gate: SageMaker Clarify can analyse bias in the raw dataset before training, produce bias metrics and feature-attribution explanations for a trained model, and — integrated with Model Monitor — track bias and explanation drift on a deployed endpoint, while Amazon Bedrock Guardrails apply safeguards at inference time. Treating the pre-launch review as final ignores that real-world data shifts after deployment and can degrade both fairness and accuracy. Treating data collection as sufficient ignores unfairness introduced by modelling and threshold choices and by adversarial or unexpected inputs. Access control and encryption protect who reaches the model and how data is stored, which is a different problem from measuring fairness or filtering harmful output.