Deep Learning Foundations practice questions

Worked examples for Deep Learning Foundations

1. Question 1

   Which statement best describes how feature extraction differs between deep learning and classical machine learning?

   1. A. Deep learning learns useful features automatically from raw data, whereas classical machine learning usually relies on features engineered by humans.Correct answer

      Correct. Successive layers of a deep network build their own representations from raw input, while classical algorithms typically depend on a practitioner selecting and constructing informative features up front.

   2. B. Classical machine learning discovers features automatically, while deep learning requires features to be hand-engineered first.

      Exactly reverses the two approaches. Manual feature engineering is characteristic of classical machine learning; automatic feature learning is the hallmark of deep learning.

   3. C. Neither approach uses features; both operate directly on raw bytes without any internal representation.

      Denies the role of representations entirely. Both approaches predict from features — the difference is only whether a human or the model produces them.

   4. D. Both approaches require an identical, manually specified feature set before training can begin.

      Treats the two as interchangeable in workflow. Deep learning's main practical advantage on unstructured data is precisely that it removes the manual feature-specification step.

   Explanation

   A defining advantage of deep learning is representation learning: successive layers derive their own increasingly abstract features from raw input such as pixels or text, so no human feature-engineering step is required. Classical machine learning generally depends on a practitioner choosing and constructing the input features. Claiming the reverse, claiming both need the same manual feature set, or claiming neither uses features all miss where the automation actually sits.

2. Question 2

   Why does stacking several hidden layers help a deep neural network handle complex inputs such as photographs?

   1. A. Each successive layer builds a more abstract representation from the previous layer's output, moving from simple patterns to higher-level concepts.Correct answer

      Correct. Hierarchical representation learning is the core benefit of depth: early layers capture simple structure such as edges, and later layers combine those into higher-level concepts.

   2. B. Each additional layer stores a copy of the training data so the network can look up similar examples at prediction time.

      Confuses a parametric network with instance-based lookup. Layers hold learned weights, not stored training examples to be retrieved later.

   3. C. Additional layers remove the need for training data by supplying the missing information themselves.

      Claims architecture substitutes for data. Adding layers increases the number of parameters to fit, which generally raises rather than lowers the amount of training data required.

   4. D. Additional layers guarantee that the network cannot overfit, regardless of how much data is available.

      Inverts the effect of capacity on overfitting. More layers add parameters and, on limited data, make overfitting more likely rather than impossible.

   Explanation

   Depth gives a network a hierarchy of representations: each hidden layer transforms the output of the one before it, so simple low-level patterns are progressively combined into higher-level, more abstract concepts — the mechanism behind deep learning's success on complex inputs like images. Layers hold learned weights rather than copies of training examples, extra layers increase rather than remove the need for data, and added capacity makes overfitting more of a risk, not less.

3. Question 3

   What computation does a single artificial neuron perform on the values it receives?

   1. A. It selects the single largest input value and forwards it unchanged to the next layer.

      Confuses a neuron with a pooling operation. Taking a maximum is a layer-level operation in some architectures; it is not how a neuron combines its weighted inputs.

   2. B. It compares its inputs against stored training examples and returns the closest match.

      Describes instance-based (nearest-neighbour) methods. A neuron stores learned weights, not training examples, and computes rather than retrieves.

   3. C. It sorts its inputs and passes the median value forward to reduce noise.

      Confuses a neuron with a statistical filtering step. Neurons combine all inputs through learned weights instead of discarding them in favour of one order statistic.

   4. D. It computes a weighted sum of its inputs plus a bias, then passes that value through an activation function to produce its output.Correct answer

      Correct. This is the standard neuron computation: scale each input by its connection weight, add them together with a bias term, and transform the result with an activation function.

   Explanation

   The basic unit of a neural network multiplies each incoming value by the weight on that connection, sums the results together with a bias, and applies an activation function to the total; that output becomes an input to neurons in the next layer. Selecting a maximum, retrieving the nearest stored example, or taking a median all describe other techniques — pooling, instance-based learning, and statistical filtering — none of which is what a neuron does.

4. Question 4

   During the training of a neural network, what actually changes as the model learns from its errors?

   1. A. The choice of activation function in each layer is rewritten automatically to fit the data.

      Treats a fixed architectural choice as a learned parameter. The activation function is selected as part of the design; training does not rewrite it.

   2. B. The training examples themselves are edited so they agree with the model's current predictions.

      Inverts the direction of learning. Training data is ground truth held fixed; the model is adjusted to match it, never the other way round.

   3. C. The weights and biases are adjusted so that the network's predictions move closer to the expected outputs.Correct answer

      Correct. Training compares predictions against expected values and propagates the error backwards to update weights and biases, repeating until the error is acceptably small.

   4. D. The number of layers in the network is increased automatically until the error reaches zero.

      Confuses architecture design with parameter fitting. Depth is a design decision made before training; a standard training run does not add layers.

   Explanation

   Learning in a neural network means parameter fitting: the network makes a prediction, the error against the expected output is measured, and that error is propagated backwards so weights and biases are nudged in the direction that reduces it, over many passes through the data. Activation functions and the number of layers are architectural choices fixed before training rather than things a training run rewrites, and the training data is the ground truth being fitted, never something the model edits to suit itself.

5. Question 5

   Which TWO statements correctly describe how CNNs and RNNs relate to the kind of data they process? (Select TWO.)

   1. A. A CNN applies learned filters across grid-structured input, which makes it well suited to images.Correct answer

      Correct. Convolution over a spatial grid detects local patterns anywhere in the input, matching the structure of image data.

   2. B. An RNN carries information from earlier steps forward as it processes input, which makes it well suited to sequential data such as text or time series.Correct answer

      Correct. The recurrent connection gives the network memory of prior steps, which is what sequential data requires.

   3. C. An RNN is the standard choice for classifying a single static image.

      Misapplies a sequence model to a spatial task. A single image has no meaningful step ordering, so recurrence provides no advantage over convolution.

   4. D. CNNs and RNNs are interchangeable, so either can be substituted for the other without affecting suitability.

      The interchangeability misconception. The two architectures encode different assumptions — spatial locality versus sequential dependence — and swapping them degrades results.

   5. E. Because RNNs process data step by step, they can be trained without any labeled examples.

      Confuses processing order with supervision. Step-by-step processing says nothing about labels; supervised RNN training still requires labeled data.

   Explanation

   CNNs and RNNs encode different assumptions about data: convolutional filters exploit spatial locality in grid-structured input such as images, while recurrent connections carry information forward across steps to model sequential data such as text and time series. Choosing a recurrent model for a single static image ignores that distinction, the two architectures are not interchangeable, and processing data step by step has no bearing on whether labeled training examples are needed.

6. Question 6

   A retailer wants to forecast next month's demand from several years of daily sales history, where each day's value depends on recent days and seasonal trends. Which characteristic of the chosen model matters most for this task?

   1. A. The ability to model dependencies across ordered time steps so earlier observations inform later predictions.Correct answer

      Correct. Sales history is sequential data, so the model must carry information across ordered time steps — the capability that recurrent and other sequence models provide.

   2. B. The ability to apply convolutional filters that detect edges within a two-dimensional pixel grid.

      Applies image-specific machinery to a time series. Edge detection over a pixel grid addresses spatial structure, which is not what makes daily sales history predictable.

   3. C. The ability to treat every day's sales figure as fully independent of the others.

      Discards the temporal dependency that is the whole signal. Assuming independence throws away the recent-history and seasonal patterns the forecast relies on.

   4. D. The ability to run without any training phase, since forecasting is a rules-based calculation.

      Denies that the model is learned from data. Forecasting patterns must be fit from the historical series; there is no training-free shortcut that discovers them.

   Explanation

   Daily sales history is sequential data: order matters and each value depends on what came before, so the decisive model capability is representing dependencies across ordered time steps — the property recurrent and other sequence models are built around. Convolutional edge detection targets spatial image structure, treating each day as independent throws away the very signal being forecast, and a forecast model must still be trained on the historical series rather than applying fixed rules.

7. Question 7

   How does deep learning differ from classical machine learning with respect to feature engineering?

   1. A. Deep learning requires more manual feature engineering than classical machine learning because it has more parameters to configure.

      Inverts the relationship. Having many parameters is what enables a deep network to learn its own features; it does not create additional manual feature engineering work.

   2. B. Neither approach uses features; both learn directly from raw data with no representation of the input.

      Denies that features exist at all. Both approaches operate on features — the difference is whether a human specifies them or the network derives them from raw input.

   3. C. Deep learning learns useful feature representations automatically from raw data, whereas classical machine learning typically relies on features engineered by humans.Correct answer

      Correct. Successive layers of a deep network build increasingly abstract representations from raw input, which is why deep learning is favoured for unstructured data where good features are hard to hand-craft.

   4. D. Classical machine learning learns features automatically, while deep learning requires the practitioner to specify every feature in advance.

      Reverses the two approaches. Automatic representation learning is the deep learning property; classical algorithms are the ones that generally depend on features prepared by a practitioner.

   Explanation

   A key distinction is where features come from: a deep network's layers derive progressively more abstract representations directly from raw input, while classical algorithms generally depend on features a practitioner designs and extracts beforehand. Deep learning therefore reduces rather than increases manual feature engineering, both approaches do operate on features in some form, and it is deep learning — not classical ML — that performs the automatic representation learning.

8. Question 8

   In a deep neural network, what is the role of the hidden layers that sit between the input layer and the output layer?

   1. A. They store a copy of the training dataset so it can be replayed during inference.

      Confuses a parametric model with a lookup table. Hidden layers hold learned weights, not stored training examples, and nothing is replayed at inference time.

   2. B. They are optional decorative layers with no effect on the network's predictions.

      Dismisses the layers that give a deep network its power. Removing them reduces the model to a shallow mapping that cannot learn hierarchical features.

   3. C. They transform the incoming signal step by step, building progressively more abstract representations of the data.Correct answer

      Correct. Each hidden layer applies weights and an activation function to the previous layer's output, so early layers capture simple patterns and deeper layers combine them into higher-level features.

   4. D. They format the network's final prediction into the class labels the user requested.

      Assigns the output layer's job to the hidden layers. Producing the final prediction in the required shape is the output layer's responsibility.

   Explanation

   Data enters at the input layer, flows through the hidden layers where each one applies learned weights and an activation function to the previous layer's output, and emerges as a prediction at the output layer; that chain is how the network builds progressively more abstract representations, with early layers capturing simple patterns and deeper layers combining them. Hidden layers do not store training data, they are not optional decoration since they are the source of a deep model's representational power, and shaping the final prediction is the output layer's job.