You are iterating on a binary classifier for credit-card fraud detection. The finance department states that a false negative (missed fraud) is ten times more costly than a false positive (legitimate transaction flagged). On a 10,000-transaction evaluation set you record:
Model R Actual\Predicted Legit Fraud Legit (negative) 8700 1200 Fraud (positive) 20 80
Model S Actual\Predicted Legit Fraud Legit (negative) 9800 100 Fraud (positive) 60 40
Which model-selection rule most appropriately reflects the stated business requirement when deciding which model to deploy?
Compute the expected misclassification cost with the provided cost matrix and choose the model that minimizes that cost.
Select the model with the highest overall accuracy on the evaluation set.
Prioritize recall (sensitivity) and deploy the model with the larger recall value, regardless of false-positive volume.
Rank the models by their area under the ROC curve and deploy the one with the larger AUC-ROC.
Because the business explicitly quantifies the relative impact of each error type, the selection criterion must incorporate those costs directly rather than rely on generic accuracy or ranking metrics. Multiplying the counts by the cost weights gives Model R a total cost of 1,200 x 1 + 20 x 10 = 1,400, while Model S incurs 100 x 1 + 60 x 10 = 700. Minimizing this cost leads to the choice of Model S. Metrics like accuracy (Model S at 98.4% vs. Model R at 87.8%), recall (Model R at 80% vs. Model S at 40%), or AUC may point to a different or ambiguous decision because they do not embed the asymmetric business loss. Therefore, evaluating expected misclassification cost with the provided cost matrix is the only rule that directly aligns the model choice with organizational objectives.
Ask Bash
Bash is our AI bot, trained to help you pass your exam. AI Generated Content may display inaccurate information, always double-check anything important.
What is a cost matrix in machine learning?
Open an interactive chat with Bash
Why is accuracy alone insufficient for model selection in this case?
Open an interactive chat with Bash
What is recall, and why can't it determine the best model here?