The five questions
1. Defects per batch for eight consecutive batches read: 2 / 4 / 4 / 4 / 5 / 5 / 7 / 9 . The team reports spread as a sample standard deviation. What is that value, to two decimal places?
- A. 2.14
- B. 2.00
- C. 4.57
- D. 4.00
Correct answer: A.
The mean is 40 / 8 = 5. The squared deviations are 9, 1, 1, 1, 0, 0, 4, 16, which sum to 32. A sample standard deviation divides by n - 1, so the variance is 32 / 7 = 4.571 and the standard deviation is the square root, 2.14.
Why the others are wrong. B: That is the population standard deviation, sqrt(32 / 8) = 2.00; it divides the sum of squares by n rather than n - 1, which understates spread when the eight batches are a sample. C: That is the sample variance, 32 / 7; the question asks for the standard deviation, which is its square root. D: That is the population variance, 32 / 8; it is neither on the requested scale nor computed with the sample divisor.
2. A pipeline runs in this order: load 90 days of events | add each customer's mean spend over the full 90 days as a feature | fit a standard scaler on all rows | random 80/20 split | train . Which two changes are required to make the validation estimate honest? (Choose two.)
- A. Compute the spend feature only from events strictly before each row's timestamp
- B. Raise the validation share from 20% to 40% so the estimate becomes more stable
- C. Fit the scaler on the training rows alone and apply the stored statistics to validation
- D. Swap the standard scaler for a min-max scaler, which is less sensitive to the split
- E. Drop the spend feature altogether, since any customer-level aggregate leaks
Correct answer: A.
The mean over the full 90 days lets every row see spend that occurred after it, and the scaler statistics are computed from rows that are supposed to be unseen. Restricting the aggregate to the window strictly before each row's timestamp removes the future leak, and fitting the scaler within the training fold removes the second.
Why the others are wrong. B: A larger held-out sample sharpens a contaminated estimate rather than decontaminating it. D: Both scalers derive their parameters from data, so a min-max scaler fitted on all rows leaks in exactly the same way. E: Aggregates are perfectly legitimate when their window closes before the row being predicted; discarding a strong feature is an avoidable cost.
3. A condition is present in 0.5% of the screened population. The assay has 99% sensitivity and 95% specificity. A person drawn at random from that population tests positive. Approximately what is the probability that they have the condition?
- A. About 99%, which is what the assay's accuracy on a positive case reports
- B. About 95%, since the assay is correct 95% of the time on people without the condition
- C. About 9%
- D. About 5%, which is the proportion of healthy people the assay wrongly flags
Correct answer: C.
Per 100,000 screened: 500 have the condition and 495 of them test positive; 99,500 do not and 5% of them, 4,975, test positive anyway. The positive predictive value is 495 / (495 + 4,975) = 495 / 5,470 = 9.0%. The base rate dominates because the healthy group is 199 times larger.
Why the others are wrong. A: That is the sensitivity, P(positive | condition). Reading it as P(condition | positive) is the inverse-probability error that ignores the base rate entirely. B: That is the specificity, and it also answers a conditional question in the wrong direction; it says nothing about how many of the positives are true. D: That is the false-positive rate, an input to the calculation rather than its result; the posterior also depends on the 0.5% prevalence and on the 99% sensitivity.
4. A churn model trained on a year-end snapshot scores: train AUC 0.997 | 5-fold CV AUC 0.994 | replay against the following month's live data AUC 0.58 . Features include tenure_months, plan_tier, support_tickets_30d and cancellation_reason_code. What is the MOST likely explanation?
- A. cancellation_reason_code is populated only for churners, so it stands in for the label
- B. The model has overfitted the training rows, which is why the live score diverges
- C. Concept drift between the snapshot and the following month has changed the churn relationship
- D. Five folds are too few to estimate generalisation reliably on a sample of this size
Correct answer: A.
Cross-validation folds are drawn from the same snapshot as training, so a field that is populated only once the outcome is known scores as well out-of-fold as in-sample. It collapses on live replay because at scoring time the field is still empty. A fall from 0.994 to 0.58 in one month is the signature of a post-outcome field, not of drift.
Why the others are wrong. B: Overfitting inflates the training score relative to cross-validation, and here cross-validation sits at 0.994 against a training figure of 0.997, so almost no gap exists. C: Drift of that magnitude inside one month is implausible, and genuine drift would also erode the later cross-validation folds rather than leaving them at 0.994. D: More folds change the variance of the estimate slightly; no number of folds rescues an estimate computed on rows that already contain the answer.
5. A team holds 40 million unlabelled product reviews and 3,000 reviews labelled for sentiment. They first train a model to predict words masked out of the unlabelled reviews, then fine-tune the resulting representation on the 3,000 labels. Which paradigm describes the FIRST stage?
- A. Self-supervised learning, because the targets are derived from the input text itself
- B. Semi-supervised learning, because both labelled and unlabelled reviews are used
- C. Unsupervised learning, because the 40 million reviews carry no sentiment label
- D. Reinforcement learning, because the model is rewarded for predicting words correctly
Correct answer: A.
The masking stage invents its own target from the input: the label for each position is the word that was hidden there. That is self-supervised learning, and it needs no human annotation at all. The second stage, on the 3,000 human labels, is ordinary supervised fine-tuning.
Why the others are wrong. B: Semi-supervised methods propagate information from a labelled set into an unlabelled one; the masking stage uses none of the 3,000 labels and defines its own target. C: Unsupervised learning has no prediction target at all; masked-word prediction has an explicit target at every position, simply one generated automatically. D: There is no environment, no sequence of actions and no delayed reward here, only a fixed loss evaluated on a fixed corpus.
6. A registered model carries: code commit 7f2a91 | hyperparameters captured in full | validation AUC 0.884 | environment image digest sha256:41bc9e | training data 'customer_features table, extracted March' . Six months later the March result cannot be reproduced, and the table has been updated in place throughout. Which single missing element explains the failure?
- A. An immutable identifier for the exact training data state used
- B. A record of the random seed the training run was given
- C. A stored copy of the validation set alongside the recorded AUC
- D. A signed approval record from the reviewer who released the model
Correct answer: A.
Three of the four legs of reproducibility are pinned here: code by the commit, environment by the image digest, and configuration by the hyperparameters. The data reference names a table and a month rather than a state, and the table has been mutated since, so identical code reads different rows. The data needs an identifier as precise as the commit hash.
Why the others are wrong. B: A seed makes a run repeatable only when its inputs are identical; the input rows themselves have changed, so a seed reproduces nothing here. C: Keeping the validation set would let the stored model be re-scored, but the failure is in rebuilding the model from its inputs. D: Approval is a governance artefact that no rebuild step consumes; it identifies a decision, not an input.
7. An analyst reports cov(height, weight) = 0.085 with height in metres, and cov(height, weight) = 8.5 after re-expressing height in centimetres. Pearson correlation is 0.62 in both runs. What does this contrast demonstrate?
- A. The centimetre run captures a stronger association, so the unit change improved the model
- B. Covariance is scale-dependent while correlation is standardised and unit-free
- C. One of the two covariance figures must be a computational error, since covariance is bounded
- D. Correlation should also have scaled by 100, so the reported 0.62 is suspect
Correct answer: B.
Covariance carries the product of both variables' units, so multiplying height by 100 multiplies the covariance by 100. Correlation divides the covariance by both standard deviations, which cancels the units, so it is invariant to any linear rescaling.
Why the others are wrong. A: No association changed; only the unit did. The correlation staying at 0.62 is the direct evidence that the relationship is identical in both runs. C: Covariance is unbounded and both figures are correct for their units; only correlation is bounded to the range -1 to 1. D: Correlation is deliberately constructed to be invariant under linear rescaling, so an unchanged value is the expected result rather than a fault.
8. An analyst standardises every numeric predictor using means and variances computed over all 60,000 rows, then splits 80/20 and fits an L2-regularised logistic regression. A reviewer objects to the ordering. What is the MOST accurate description of the harm?
- A. The score is invalid to the same degree as fitting the model itself on the held-out rows
- B. No harm arises, because standardisation is a deterministic transformation of each column
- C. Held-out rows contributed their means and variances to the transform, so the score is mildly optimistic
- D. The harm is that the regularisation strength is no longer comparable between the two splits
Correct answer: C.
Fitting the scaler on all rows lets the held-out rows influence the centring and scaling applied during training. The leak is real but modest in a large sample, because a mean and a variance are coarse aggregates. The remedy is to fit the scaler inside the training fold and apply the stored statistics to validation.
Why the others are wrong. A: Two summary statistics carry far less information than a fitted model, so equating the two badly overstates the size of the optimism here. B: It is deterministic given the statistics, but those statistics were themselves estimated from data that was meant to be unseen. D: One penalty is applied to one fitted model, so comparability across splits is not at issue; the defect is where the scaling statistics came from.
