Exploring NHS Data:
why patients miss appointments

1. Data Preparation

Four datasets were analyzed sequentially to build a complete picture of NHS GP activity:

• actual_duration.csv: Appointment length data (7 months).
• national_categories.csv: National classification breakdowns (11 months).
• appointments_regional.csv: The core dataset — 742,529,759 appointment records over 30 months.
• tweets.csv: Public sentiment data scraped from Twitter for qualitative signal.

Dates were cast to datetime for temporal analysis. Over 21,600 duplicate rows were removed from the regional dataset alone. Rows flagged as Unknown / Data Quality or Unmapped were intentionally retained — their patterned occurrence was itself informative, and discarding them wholesale would have introduced bias.

2. Staffing & Capacity

The headline finding sounds reassuring: on average, the NHS stayed well below its estimated single-day threshold of 1,200,000 appointments. But averages hide the story.

When the top three busiest ICBs are isolated, NHS North West London alone approached the 1,200,000 threshold on its own — a figure the national average completely obscures. Conversely, the bottom three ICBs never exceeded 60,000 appointments in any single month, even during the peak COVID-19 period, dragging the mean downward.

3. The Booking Interval Effect

The single most actionable finding in the dataset: the further in advance an appointment is booked, the less likely a patient is to show up. Same-day bookings have a 95.8% attendance rate. This drops progressively, falling below 85% at 15–21 days out and reaching just 72.5% for appointments booked more than 28 days in advance.

4. Appointment Duration & Delivery

The duration data reveals a clear operational reality: GP appointments are overwhelmingly short.

• 52.1% of all documented appointments lasted 15 minutes or less.
• A further 24.0% had unknown or data-quality-flagged duration, making definitive conclusions difficult.
• Only 23.9% fell into the 16-minutes-to-1-hour bracket.

This distribution has a direct implication: the majority of GP consultations are short enough to be viably conducted over telephone or video. The NHS already demonstrated this during COVID-19 — the pandemic served as an unplanned proof-of-concept for remote delivery of short-format appointments.

5. Patterns & Recommendations

Reduce Long Booking Intervals

The further away an appointment is from the day its booked, the less likely a patient is to attend it. The data shows a consistent, steep decline in attendance for appointments booked more than two weeks in advance. Reducing advance booking windows or implementing automated reminders at the 14-day and 7-day marks would directly address the low attendance rate associated with long booking intervals.

Accelerate Remote Appointment Adoption

Given that over half of all appointments are 15 minutes or less, telephone and video consultations represent a strong option to lessen concerns about staffing and booking. This option would reduce friction for patients and has the potential to cut the costs of missed appointments.

Regional Concerns: Address Localized Strain, Not Just Averages

National-level averages mask the fact that some regions are more impacted than others. For example, the North West London ICB operates close to system capacity on its own. Meanwhile, resource allocation decisions made on the basis of national averages will systematically under-serve high-demand areas like North West London. Regional granularity is essential and requires proper acknowledgement for planning.

Twitter: A Cost-Free Patient Survey & Opportunity for Modern Communication

The Twitter dataset shows that patients often express their feelings about their healthcare experiences online. This kind of data can be immensely valuable to get a hold of patient sentiment about booking experiences, GP quality, and the accessibility of services. In the future, Twitter may also be used by the NHS to publicize incoming changes, such as, the availability of remote appointments.

Full Python Notebook available here.