Fundamentals of AI and ML practice questions

Worked examples for Fundamentals of AI and ML

1. Question 1

   A team trains a model to predict energy consumption from a rich set of sensor readings that are known to interact in complex, non-linear ways. They start with a small, heavily regularized linear model. At evaluation, the model scores 0.63 on the training data and 0.61 on the validation data — the two scores are close, but both fall far short of the business target. Which diagnosis and remedy best fit this result?

   1. A. The model is overfitting, because the training score and the validation score are nearly identical; the team should add stronger regularization and reduce the number of input features.

      Reverses the symptoms of overfitting. Overfitting shows up as a large gap — high training performance with markedly lower validation performance — not as two poor, similar scores, and adding regularization to an already-too-simple model pushes performance further down.

   2. B. The model exhibits fairness bias against an under-represented group of sensors, which is what the term 'high bias' means; the team should rebalance the dataset so every sensor is equally represented.

      Conflates statistical bias in the bias–variance sense with fairness bias. High bias here means the model's functional form is too rigid to capture the underlying relationship; it is a modeling-capacity problem, not an unfair skew against a group, and rebalancing would not fix it.

   3. C. Because the training and validation scores agree so closely, the model is generalizing correctly and is as good as this problem allows; the team should deploy it and rely on post-deployment monitoring to improve it.

      Mistakes a small train–validation gap for adequacy. Consistency only shows the model performs the same on seen and unseen data; it says nothing about whether that shared level of performance is acceptable, and monitoring detects drift rather than raising a model's baseline capability.

   4. D. The model is underfitting — it has high bias and is too simple to capture the non-linear relationships in the data; the team should increase model capacity, engineer more expressive features, or relax the regularization, then re-evaluate.Correct answer

      Correct: poor performance on both the training and validation sets is the signature of underfitting (high bias), where the chosen hypothesis is too constrained for the underlying pattern. The remedy is more capacity, richer features, or less regularization, verified by re-running evaluation (AWS — What is Artificial Intelligence?, machine learning model training concepts).

   Explanation

   Underfitting and overfitting are distinguished by which scores are poor, not merely by the size of the gap between them. When a model performs badly on the data it was fitted to and equally badly on held-out data, it never captured the pattern in the first place — the classic high-bias case, expected when a heavily regularized linear form is applied to non-linear interactions; the fix is greater capacity, more expressive features, or weaker regularization. Overfitting is the opposite signature (strong training performance, weak validation performance), so adding regularization here would worsen the problem, and a small train–validation gap alone is no evidence that the achieved level of performance is good enough. Statistical bias in this sense also must not be read as fairness bias, which concerns unfair skew against a group rather than insufficient model capacity.

2. Question 2

   A retail team is building a churn-prediction model. Working from a raw table of customer transactions, they derive a "days since last purchase" column, bucket ages into ranges, one-hot encode the store-region column, and compute a rolling 90-day spend average — all with the goal of giving the algorithm stronger, more learnable signals. Which stage of the ML development lifecycle does this work belong to?

   1. A. Exploratory data analysis (EDA), because the team is inspecting the transaction data to understand it

      Confuses understanding data with transforming it. EDA profiles and visualizes data — distributions, correlations, missing values, outliers — to inform decisions; it does not produce new input variables for the algorithm to train on.

   2. B. Hyperparameter tuning, because encoding schemes and bucket widths are hyperparameters of the model

      Misclassifies data-representation choices as hyperparameters. Hyperparameters (learning rate, tree depth, number of epochs) configure the training algorithm itself and are searched over during tuning; how a column is encoded changes the input data, not the algorithm's configuration.

   3. C. Feature engineering, because the team is creating and transforming input variables that better represent the predictive signalCorrect answer

      Correct. Deriving new columns, aggregating over time windows, bucketing, and encoding categoricals into model-consumable form is exactly feature engineering — the stage that turns prepared data into the feature set a model trains on.

   4. D. Data collection, because new columns such as "days since last purchase" are being added to the dataset

      Confuses deriving values from data already on hand with acquiring new data. Data collection is about sourcing and ingesting raw records; every value here is computed from transactions the team already has, so no new data is being gathered.

   Explanation

   Feature engineering is the lifecycle stage where prepared data is turned into the input variables a model actually learns from — creating derived attributes, aggregating over time windows, bucketing continuous values, and encoding categoricals numerically. Understanding and profiling the data beforehand is exploratory data analysis, a separate earlier stage that informs these choices rather than performing them. Searching over learning rates, depths, or epochs configures the training algorithm and belongs to tuning, not to reshaping the inputs. Sourcing and ingesting new raw records is data collection, whereas every value here is computed from records the team already holds. Amazon SageMaker AI provides dedicated capabilities for this stage — data preparation and transformation tooling plus a feature store for sharing and reusing engineered features.

3. Question 3

   During the evaluation stage of the ML lifecycle, a team assesses a binary classifier that flags fraudulent credit-card transactions. Only about 0.5% of transactions in the holdout set are actually fraudulent, and the model reports 99.5% accuracy — but analysts report that almost no real fraud is being caught. Which statement best describes what is happening and which evaluation metrics the team should rely on?

   1. A. Accuracy is misleading on this imbalanced dataset because always predicting 'not fraud' scores about 99.5%; the team should evaluate recall (the share of actual fraud the model finds) alongside precision (the share of flagged transactions that truly are fraud), summarizing the trade-off with F1 or AUC.Correct answer

      Correct. Model evaluation must match the business problem and the class distribution: recall exposes missed fraud, precision exposes false alarms, and F1 (their harmonic mean) or AUC summarizes the trade-off across thresholds. This is the standard evaluation practice described for the ML lifecycle in the Amazon SageMaker AI Developer Guide and AWS's AI overview.

   2. B. Accuracy of 99.5% is a sound measure of quality on any classification problem, so the model is performing well and the analysts' complaint reflects a data-collection issue rather than a modeling issue.

      This is the 'a single metric such as accuracy is always sufficient' misconception. On a severely imbalanced dataset a model that predicts the majority class for every record still scores near-perfect accuracy while detecting nothing, so accuracy alone cannot validate the model.

   3. C. The team should optimize precision alone, because precision measures the fraction of all genuinely fraudulent transactions that the model successfully identified.

      This swaps the definitions of precision and recall. Precision is the share of predicted positives that are correct; the share of actual positives that were found is recall — which is exactly the quantity collapsing here.

   4. D. The gap between the high accuracy score and the analysts' observation is the classic signature of overfitting, so the fix is to train the model for more epochs on the same training data.

      This confuses majority-class bias under class imbalance with overfitting. Overfitting shows up as high training performance paired with poor holdout performance, not as a high holdout accuracy driven by a dominant class — and training longer on the same data would tend to worsen overfitting, not cure it.

   Explanation

   Evaluation is the lifecycle stage where the metric must be chosen to fit the problem and the class distribution. With roughly 0.5% positives, a degenerate model that never predicts fraud already achieves about 99.5% accuracy, so accuracy carries almost no information; recall reveals how much actual fraud is missed, precision reveals how many alerts are false, and F1 or AUC summarizes the balance between them. Treating accuracy as sufficient ignores this imbalance effect, defining precision as the fraction of real fraud found reverses precision and recall, and labeling the symptom as overfitting mistakes majority-class dominance for memorization of the training set — additional training epochs would not address it. See the model-evaluation guidance in the Amazon SageMaker AI Developer Guide (https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html).

4. Question 4

   A retailer asks a data science team to reduce the cost of unsold stock. The team has no dataset gathered yet and no agreed success metric, and it works through the **ML development lifecycle** up to the point where a model first learns from the data. Arrange these lifecycle activities in the order AWS lists them, from first to last.

   1. A. Derive and select the input features the model will train on

      Feature engineering is part of the data processing AWS puts after exploration: the source says to inspect and understand the data before processing it. The features it selects are the inputs the fitting step consumes.

   2. B. Explore the collected data to understand its distributions, gaps and anomalies

      Exploration examines the gathered examples, so it needs them to exist, and AWS states the link to what follows: inspect and understand the data before processing it.

   3. C. Fit the model on the training data so it learns the underlying patterns

      Fitting consumes the selected input features, so it depends on them being chosen. It is where the model first learns from the data, which is where this scenario stops.

   4. D. Frame the business problem: decide what the model should predict and which metric it must optimize

      Framing decides what is being predicted and which metric counts, which is what makes it possible to say which data would even be relevant to gather. The retailer has asked for an outcome, not a model.

   5. E. Gather the labeled ground-truth data the model will learn from

      Collection supplies the examples everything else works on. It answers the question framing posed, and nothing can be inspected, cleaned or learned from until these examples exist.

   Explanation

   The lifecycle's early stages are ordered by what each one needs before it can start: a question to answer, then examples, then an understanding of those examples, then inputs a model can learn from. AWS states the sharpest of these links outright — inspect and understand the data before processing it (AWS, What is Machine Learning?; Amazon SageMaker AI Developer Guide).

5. Question 5

   A team is labeling a new dataset so that a supervised learning model can be trained on it. A stakeholder asks what a "label" actually is in this context and why supervised learning depends on labels. Which statement best describes labeled training data in supervised learning?

   1. A. A label is a reward signal that the model receives after each action it takes in an environment, which it uses to adjust its future behavior.

      Describes reinforcement learning, not supervised learning. Rewards are earned through trial-and-error interaction with an environment, whereas supervised labels are known correct answers attached to examples before training begins.

   2. B. A label is the known correct output paired with each training example, which the model learns to map inputs to so it can predict outputs for new, unseen inputs.Correct answer

      Correct. In supervised learning each training example carries a known answer — a class or a numeric value — and training adjusts the model so it maps inputs to those answers and generalizes the mapping to new data.

   3. C. A label is the cluster assignment that the algorithm itself discovers during training, so no human or pre-existing annotation is needed before training begins.

      Confuses supervised labels with the groupings produced by unsupervised clustering. Cluster assignments are an output the algorithm infers from unlabeled data; supervised labels must already exist in the data as ground truth.

   4. D. A label is a configuration setting, such as learning rate or number of layers, that the team chooses before training to control how the model learns.

      Confuses labels with hyperparameters. Hyperparameters govern the training process itself; labels are ground-truth values that belong to the data and define what the model is being taught to predict.

   Explanation

   Supervised learning trains on data in which every example is paired with its known correct output — the label — so the algorithm can learn the mapping from input features to that output and then apply it to inputs it has never seen. This ground truth must be present in the data beforehand, which is why labeling is a distinct and often costly step. Groupings that an algorithm discovers on its own come from unsupervised learning on unlabeled data; reward signals earned by acting in an environment belong to reinforcement learning; and settings such as learning rate or layer count are hyperparameters that control training rather than values the model is taught to predict.

6. Question 6

   A team trains a binary classifier to flag fraudulent card transactions. In the holdout set, 0.3% of transactions are fraudulent. A trivial baseline that labels every transaction "not fraud" scores 99.7% accuracy. The business goal is to catch as large a share of the genuinely fraudulent transactions as possible, while keeping the number of legitimate transactions sent to human investigators at a level the fraud team can absorb. Which statement correctly describes the evaluation metrics the team should rely on?

   1. A. Recall measures the share of genuinely fraudulent transactions the model flags, precision measures the share of flagged transactions that are truly fraudulent, and F1 — their harmonic mean — summarizes the trade-off between catching fraud and the investigator workload created by false positives.Correct answer

      This states both metrics correctly and maps them to the stated business constraints: recall (over actual positives) tracks fraud caught, precision (over predicted positives) tracks how much of the investigators' queue is real fraud, and F1 as their harmonic mean penalizes a model that sacrifices one for the other — which is exactly why these are preferred over accuracy on imbalanced data.

   2. B. Accuracy is the appropriate primary metric here, because it aggregates both classes into a single number and the holdout set is large enough for that number to be statistically reliable.

      This is the 'accuracy is always sufficient' misconception. On a severely imbalanced dataset accuracy is dominated by the majority class — the do-nothing baseline already reaches 99.7% while catching zero fraud — so a large sample size does not rescue it. Sample size fixes variance, not the fact that the metric ignores the rare class the business cares about.

   3. C. Precision is the share of the genuinely fraudulent transactions that the model successfully flags, and recall is the share of flagged transactions that turn out to be fraudulent, so precision should be maximized to catch more fraud.

      This swaps the definitions of precision and recall. Precision is computed over predicted positives (of the transactions flagged, how many were truly fraud); recall is computed over actual positives (of the truly fraudulent transactions, how many were flagged). With the definitions inverted, the stated optimization target is also inverted.

   4. D. AUC should be the sole metric, because it reports the model's accuracy at the default 0.5 decision threshold and therefore already accounts for the class imbalance.

      This misdescribes AUC. AUC is the area under the ROC curve and summarizes ranking quality across all decision thresholds — it is explicitly not the accuracy at any single threshold, including 0.5. It is also threshold-free, so it alone cannot tell the team what investigator workload a chosen operating point produces.

   Explanation

   On a heavily imbalanced dataset, a metric computed over all predictions is dominated by the majority class, which is why the ML lifecycle's evaluation stage relies on class-specific metrics: recall is computed over the actual positives (how much of the real fraud was found), precision over the predicted positives (how much of the flagged queue is real fraud), and F1 is their harmonic mean, capturing the trade-off between the two in one number. Treating overall accuracy as the primary metric fails because a model that never predicts the rare class already scores 99.7%, and a larger sample does not change that. Reversing the definitions of precision and recall reverses which quantity the team would be optimizing. AUC is the area under the ROC curve, summarizing ranking quality across all thresholds rather than accuracy at any one threshold, so it cannot by itself express the investigator-workload constraint.

7. Question 7

   A logistics company wants a model that predicts whether a shipment will arrive late. The company has three years of historical shipments, and every past shipment record is tagged with whether it actually arrived late or on time. Which TWO statements about this situation are correct? (Select TWO.)

   1. A. This is supervised learning, because the training data includes the known outcome for each historical shipment.Correct answer

      Supervised learning is defined by training on labeled examples in which each input is paired with the correct output. The recorded late/on-time outcome is exactly that label, and the model learns to map shipment features to it.

   2. B. The late/on-time tag on each historical record serves as the label that the model learns to predict.Correct answer

      A label is the known target value attached to a training example. Here the historical arrival status is the target the algorithm fits against, and prediction quality is measured by how well the model reproduces it on unseen data.

   3. C. This is reinforcement learning, because the model is being trained on data collected over three years.

      Confuses a long historical data collection period with reinforcement learning. Reinforcement learning has no fixed labeled dataset — an agent takes actions in an environment and learns from reward or penalty signals through trial and error.

   4. D. This is unsupervised learning, because the goal is to make a prediction rather than to describe the data.

      Misstates what makes learning unsupervised. Unsupervised learning is defined by the absence of labels and is used to find hidden structure such as clusters; the presence of a known target here rules it out, and prediction is characteristic of supervised learning.

   5. E. The historical outcomes must be discarded before training so that the model is not biased by past results.

      Reflects the misconception that learning from past outcomes is a form of leakage or bias. Learning from historical outcomes is the entire mechanism of supervised training; without the labels there is nothing for the algorithm to learn to predict.

   Explanation

   Supervised learning trains an algorithm on a labeled dataset, where each training example pairs input features with the known correct output, so the model can predict that output for new inputs. Historical shipments tagged late or on time are labeled examples, and the arrival status is the label the model is fit against. Reinforcement learning is ruled out because it learns from rewards earned through trial-and-error interaction with an environment rather than from a labeled historical dataset, and unsupervised learning is ruled out because it is defined by having no labels at all — it discovers structure such as groupings rather than predicting a known target. Removing the recorded outcomes would leave nothing to supervise the training, since those outcomes are what the model learns to reproduce.

8. Question 8

   A national retail chain wants to publish a mobile app feature with three parts, using only AWS managed AI services and no model training of its own: 1. A shopper photographs a store shelf; the app must identify the products and objects visible in the image. 2. The app must read aloud, in a lifelike voice, the product description that the shopper selects. 3. The app must scan customer product reviews written as free-form text and report the sentiment of each review along with the key phrases it contains. Which AWS managed AI services correctly provide capabilities 1, 2, and 3, in that order?

   1. A. Amazon Textract, Amazon Transcribe, Amazon Kendra

      Confuses document extraction with general image analysis, and speech-to-text with text-to-speech. Textract extracts text and data from scanned documents rather than identifying arbitrary objects in a photo, Transcribe converts speech audio into text (the opposite of reading text aloud), and Kendra provides intelligent search over document repositories, not sentiment and key-phrase analysis.

   2. B. Amazon Rekognition, Amazon Polly, Amazon ComprehendCorrect answer

      Correct. Rekognition performs image and video analysis including object and scene detection, Polly is the text-to-speech service that turns written text into lifelike spoken audio, and Comprehend is the natural-language-processing service that surfaces insights from text such as sentiment and key phrases.

   3. C. Amazon Rekognition, Amazon Transcribe, Amazon Translate

      Gets the vision service right but inverts the speech direction and mistakes translation for text analytics. Transcribe converts spoken audio into written text, whereas the requirement is to speak written text (Polly), and Translate converts text between languages rather than reporting sentiment or key phrases.

   4. D. Amazon Personalize, Amazon Polly, Amazon Lex

      Mistakes a recommendation engine for image analysis and a bot-building service for text analytics. Personalize generates per-user recommendations from interaction data and cannot analyze image content, and Lex builds conversational interfaces with intent recognition rather than returning sentiment and key phrases for a body of review text.

   Explanation

   Each AWS managed AI service occupies a distinct role, and mapping a requirement correctly means matching the input and output modality. Detecting objects and scenes in a photograph is computer vision, handled by the image and video analysis service; converting written text into lifelike spoken audio is text-to-speech; and extracting sentiment and key phrases from free-form text is natural-language processing. The incorrect groupings swap speech-to-text for text-to-speech, substitute document data extraction or recommendations for general image analysis, or offer translation, intelligent search, or conversational bot building in place of text analytics — each of which serves a different purpose entirely.