Heart Failure Mortality Review

A care team wants a consistent way to decide which heart-failure patients need closer follow-up after discharge. Clinicians already have judgement, but it varies with workload and experience. A risk score sounds like the obvious answer. With a few hundred patients, the harder question is whether the data can support one at all.

The decision

The score would be used to prioritise follow-up calls and medication reviews — not to make a diagnosis and not to replace clinical judgement. That distinction set three requirements: the output had to be available at discharge, it had to be calibrated enough to discuss with a patient, and it had to be honest about uncertainty. A model that ranks patients but cannot say how uncertain each rank is would not survive clinical review.

What we worked with

A public clinical dataset of 299 heart-failure admissions with 13 fields:

• Patient context: age, sex, smoking status, diabetes, anaemia and high blood pressure.
• Clinical measurements: ejection fraction, serum creatinine, serum sodium, platelets and creatinine phosphokinase.
• Follow-up: the number of days each patient was observed and whether death occurred during follow-up.

The cohort is small: 299 records, no missing values, and death in roughly one third of cases during follow-up.

The traps we designed around

• Follow-up time is not a predictor. The observation period is part of how the outcome was measured. Feeding it into a model leaks information about the outcome and makes the score unusable at the point of discharge.
• Binary classification versus time-to-event. Treating "death during follow-up" as a yes/no label ignores when events happened and how long each patient was observed. A survival framing is often more appropriate, but it needs enough events to estimate a baseline hazard. We compared both and documented which question each one answers.
• Accuracy is the wrong headline. With roughly two thirds of patients surviving, a model that predicts "survives" for everyone looks accurate and is clinically useless. Discrimination and calibration matter more than accuracy.
• Small subgroups. Splitting 299 patients by age band, sex and comorbidity leaves groups too small to support stable conclusions. We pre-specified which subgroup checks were feasible and reported the rest as descriptive only.
• Skewed clinical measurements. Creatinine and creatinine phosphokinase are heavily right-skewed. Transformations and clinically meaningful cut-points have to be chosen deliberately, not inherited from a default pipeline.

How the score was built

1. Fix the estimand with the clinical owner. We chose the outcome horizon and the prediction moment first: death within a defined follow-up window, using only information available at discharge.
2. Audit before modelling. We checked ranges, duplicates and clinically implausible values, and confirmed that no field encoded information recorded after discharge.
3. Start with a small, interpretable baseline. A logistic model using clinically established variables gave a reference point. We avoided stepwise selection that would fit noise in a dataset this size.
4. Test a challenger, not a leaderboard. A tree-based model was compared on discrimination and calibration. It had to improve both to be worth the loss of transparency.
5. Report uncertainty. Bootstrap confidence intervals accompanied every performance estimate, because a single split at this sample size is not a reliable guide.
6. Translate scores into actions. Risk bands were tied to follow-up options, with an explicit "insufficient evidence" band where the model could not separate patients meaningfully.

What this changes

The value of the work was less a final model than a clear boundary around what the data can support:

• Risk is discussed in bands with uncertainty. A score of 0.18 is not treated as precise; the band and its confidence interval determine the action.
• Capacity drives the threshold. The cut-off comes from how many patients the team can contact, not from a default probability.
• Subgroup claims are limited by evidence. Where the sample is too small, the output says so instead of producing a spurious finding.
• External validation is a prerequisite, not a nice-to-have. The model is presented as a candidate for validation on a larger cohort, not as a deployable clinical tool.

What you receive

• A defined cohort and outcome, with the prediction moment written down.
• An interpretable baseline and a documented challenger comparison.
• Calibration plots, discrimination estimates and confidence intervals.
• Risk bands mapped to clinical actions, plus a limitations memo.
• A validation plan for a larger, independent cohort.

Where this applies

The method fits hospitals, health systems and insurers working with small clinical cohorts. The minimum inputs are a stable patient identifier, a defined outcome window, the clinical measurements available at the decision point and a clinical owner who can define the action. The first deliverable is usually a feasibility memo and a leakage review — not a model.

This is an illustrative case study using an anonymized public clinical dataset; it contains no patient-identifiable information and does not report results from a specific client engagement. It reflects our standard method for small-sample clinical risk modelling.