Transitional-care teams can follow up with only a fraction of discharged patients. Without a consistent way to prioritise, referrals depend on individual judgement, discharge volume and who happens to be available that day. The provider wanted a risk signal at discharge — but the harder question was not who is at risk. It was what the care manager should do differently for this patient, today.
The capacity decision
The programme had a fixed number of follow-up slots each week. The score had to be produced before the patient left, use only information available at that point, and produce an action rather than a label. It also had to remain decision support: a clinician could accept, override or dismiss a case, and that decision had to be visible. A model that quietly became a verdict would fail the clinical review before it ever failed statistically.
What the team could see at discharge
De-identified extracts from the electronic health record and related systems:
- Index admission: age band, admission type, length of stay and discharge disposition.
- Clinical history: diagnosis and procedure codes, prior admissions and emergency visits.
- Medication and labs: summaries available at or before discharge.
- Operational context: ward, discharge destination and whether a follow-up appointment was already booked.
Only information available at discharge entered the model. Post-discharge events predict readmission beautifully and cannot exist when the score is produced — the most common and most consequential leakage in healthcare models.
The traps we designed around
- Cohort ambiguity. Transfers, planned readmissions and short-stay observations are not the same event. Inclusion and exclusion rules were agreed with clinicians so that the outcome meant one thing to everyone.
- Post-discharge leakage. Follow-up appointments arranged after discharge, readmission-related calls and later diagnosis updates were excluded from the feature set.
- Score as verdict. A probability shown without context invites rationing decisions it cannot support. The interface was designed as a prompt for clinical review, with the drivers visible and the limitations stated.
- Subgroup performance. A model can rank the overall population well and still be poorly calibrated for age or sex groups. Subgroup checks were part of validation, not a later audit.
How the worklist was built
- Fix the cohort first. Index admissions, transfers and planned readmissions were defined explicitly, along with the 30-day window. The cohort rules lived in documentation, not in an undocumented SQL filter.
- Set a transparent baseline. A logistic model made the direction and size of each risk factor easy for clinicians to review.
- Test a stronger challenger. Gradient boosting captured interactions such as prior utilisation combined with medication burden and discharge setting. Discrimination, calibration and subgroup performance were compared against the baseline.
- Review with clinicians. A clinical panel examined the top risk factors, challenged implausible signals and helped define how the score should be worded on screen.
- Design the workflow. The score was delivered as a discharge worklist with contributing factors and suggested follow-up actions. Accept, override and dismiss actions were logged.
- Plan the monitoring. Calibration checks, subgroup reviews, drift alerts and a retraining trigger tied to changes in care pathways were agreed before launch.
What this changes
- Follow-up allocation became consistent. Every discharge received the same first pass, and scarce slots went to patients with a visible reason for review.
- Clinicians could challenge the signal. Overrides were recorded, which turned disagreements into feedback rather than silent workarounds.
- The programme became reviewable. Leadership could see whether follow-up reached the patients it was designed to help, rather than only how many calls were made.
- Privacy was part of the design. Minimum-necessary data, role-based access and audit logging were built into the workflow from the start.
What we delivered
- A discharge-time readmission risk score with a care-manager worklist.
- Reason codes and plain-language explanations for each patient flagged.
- A validation report covering discrimination, calibration and subgroup performance.
- Privacy safeguards: minimum-necessary data, role-based access and audit logging.
- A monitoring and governance plan for clinical and privacy review.
Where this applies
This fits hospitals, health systems and care-management programmes that can act before discharge. The minimum inputs are a stable patient identifier, admission and discharge dates, prior utilisation, diagnosis and medication fields, and a readmission definition agreed with the clinical owner. The first deliverable is usually the cohort definition plus the leakage and subgroup review — before the model, because those three decisions determine whether the result is usable in a clinical workflow.
This is an illustrative case study using anonymized data with no patient-level information; it does not report results from a specific client engagement. Any healthcare analytics programme must be validated locally and reviewed against applicable privacy law, including PHIPA in Ontario and PIPEDA at the federal level. This article is not legal advice.
