Case Study · 2026
Software Defect Prediction Dashboard
My final year project: a proactive software defect prediction system using the TabNet deep learning architecture on the NASA CM1 benchmark dataset — with a live dashboard that translates the model's attention weights into actionable Z-score deviations.
The Problem
Testing everything is impossible — so which modules get tested first?
Software teams can't deep-test every module. Defect-prone modules hide among hundreds of clean ones: in NASA CM1, 562 clean modules vs just 43 defective ones. Post-hoc explanation tools (SHAP, LIME) approximate a model's reasoning — for a research contribution, I wanted the model to be natively interpretable.
- ✗Severe class imbalance: 43 defective vs 562 clean modules
- ✗Limited testing budget — triage must target the riskiest modules first
- ✗Post-hoc explainers approximate; native attention is exact
- ✗Research question: can TabNet's sequential attention select features without wrapper tools?
Why this solution
TabNet: feature selection built into the architecture
TabNet applies sequential attention at every decision step — it chooses which features to look at natively. That means interpretability isn't bolted on afterwards; the top predictive features (LOC_CODE, LOC_BLANK, PERCENT-based metrics) fall straight out of the model.
- ✓SMOTE rebalanced training: synthetic minority oversampling of defective modules
- ✓Native feature selection — no post-hoc approximation tools
- ✓Interpretability is the deliverable, not a bonus
- ✓A live dashboard turns attention weights into Z-score deviations engineers can act on
How it works
01
Dataset
NASA CM1 benchmark — software metrics per module, defect labels
02
Rebalancing
SMOTE oversamples the 43 defective modules against 562 clean ones
03
Training
TabNet learns with sequential attention over features
04
Feature insight
Attention reveals LOC_CODE, LOC_BLANK, PERCENT-based metrics as top predictors
05
Deployment
Dashboard renders attention weights as Z-score deviations per module
Challenges I faced
⚠️ 43 positives among 605 modules
→ fixed: SMOTE on the training partition only — never the test set
⚠️ Accuracy lies under imbalance
→ fixed: Optimised and reported recall + F1, verified with a confusion matrix (19 TP, 1 FN)
⚠️ Making attention legible
→ fixed: Dashboard translates raw attention into Z-score deviations — a form engineers already read
⚠️ Reproducibility
→ fixed: Fixed splits, documented pipeline, versioned artifacts
Results
95%
recall on defective modules
0.6032
F1-score
19/1
true positives / false negatives
3
top features identified natively
🧰 Built with
💡 What it taught me
With imbalanced data, recall is the honest metric — 95% accuracy means nothing if you miss defective modules. And attention weights, done right, need no SHAP to be understood.
read another one →