Healthcare Analytics: Data-Driven Scheduling to Reduce No-Shows

Use Cases & Projects Romain Doutriaux
The healthcare industry suffers from a data problem. There is an abundance of raw data in healthcare, but no one really knows what to do with it. The good news is that all of this data can be used to solve a multitude of common, day-to-day problems using predictive analytics.


The effective use of data analytics can help healthcare institutions and professionals gain efficiency, improve patient care, and optimize profitability. In this blog post, we we will highlight how a health care-specific issue — no-show appointments — can be resolved with data. Hint: the solution is predictive analytics, which can be used to craft real-world solutions to a multi-billion-dollar problem.

The unfortunate reality is that no-shows have become extremely common — one study reported that the no-show rate in U.S. primary care practice can vary from as low as 5% to as high as 55%. Dealing effectively with patient no-shows has been a challenge in the healthcare industry especially now that reimbursement is more closely tied to performance measures surrounding physical appointments.

The long-term effect of this phenomenon is lowered reimbursement for providers and, more importantly, the health welfare impact on adherence, quality, and clinical outcome measures on patients. For patients, spotty appearances with healthcare providers results in less coordinated care, particularly in cases of chronic diseases and preventive encounters. Patients suffering from chronic conditions may require very regimented treatment plans — missing even one treatment may have debilitating consequences.

Annually lost due to no-show issues

Missing preventive care treatments leads to longer and more expensive care as potential issues become real health problems. No-shows also have a direct financial effect on healthcare providers as expected revenue targets fall short, labor hours are wasted, and inefficiencies are created. The challenge has always been, "What do we do with all this data? How do we add meaning to it?"

Dealing effectively with patient no-shows has been a challenge in the healthcare industry, especially now that reimbursement is more closely tied to performance measures surrounding physical appointments. Many providers are simply overwhelmed with the problem and resort to traditional stopgap policies, such as reminding patients the day before their appointments. The effect is marginal and, ultimately, is short-sighted because it does not directly address the problem itself.

The No-Show Problem

We will start by having a look at what is wrong with the current implementation of data analytics in the healthcare ecosystem and how it applies to the no-show issue. We will then offer an alternative approach to addressing no-show appointments that makes use of predictive analytics. Lastly, we will discuss how this method could be applied to the healthcare industry.

1. Data Fragmentation and Limited Skills Deteriorate the Data Analysis Process

The healthcare industry is no stranger to data — they have an abundance of it, from clinical measures and demographic information to lab results and staffing data . Figuring out what to do with all of the data is a challenge that lies at the heart of medicine in the 21st century. According to a recent survey released by the National Association of ACOs (NAACOS), 51% of Medicare Shared Savings Program (MSSP) ranked data-related issues, such as access, inconsistency, and deployment, as the biggest roadblocks in their accountable care journey 4. Core obstacles of data management & usage include:

  • Translating data into actionable information for providers.
  • Acquiring the required skill-sets needed to analyze the data.
  • Finding solutions that can report on the business aspect of clinical data. 

The “data issue” is exacerbated by the nature of the U.S. healthcare ecosystem: it is a highly fragmented industry across multiple sectors. Healthcare is rarely coordinated, incentives are misaligned, and variation is ubiquitous. Apart from the structural dysfunctions, healthcare suffers at the IT level: technologies are out-of-date compared to other high-tech sectors, institutions use proprietary platforms that are incompatible with other systems, and the IT skill-sets of employees are highly disparate.

Further, multiplicity of data sources makes collection and use difficult. As mentioned, there is an abundance of data in the healthcare industry and all of it flows from multiple sources:

  • Prescription, diagnostic (lab, vitals measurements), and demographic sources
  • Social media / web-based and machine-to-machine data (e.g., remote devices)
  • Transactional data (e.g., claims and billing activities)
  • Biometric data
  • Human-generated data (e.g., Electronic Health Records [EHR], physician notes)

According to the Institute for Health Technology Transformation, the amount of U.S. healthcare data reached 150 exabytes in 2012 and is estimated to double by 20226. One exabyte equals 1 billion gigabytes and, given that the human brain can only process around 7 variables at once, it’s obvious that the sheer size of the data involved poses a significant challenge.

A large percentage of human-generated and biometric data is transcribed into Electronic Health Record (EHR) systems; many of these platforms were invented in the 1960s and, frequently, little has changed in the basic approach used to categorize incoming data. These one-size-fits-all systems are not well-suited to individual workflows and they also lack the personalization needed to truly understand the needs of the patient.

Many organizations use data sources that are comfortable, familiar, and accessible. Over time the usage of these data sources become increasingly entrenched in healthcare environments, to the point where other sources of data are not even considered. The problem with this approach is that it only provides a partial picture and does not provide access to the value that big data analytics can offer.

Additionally, data diversity hinders data integration. In U.S. hospitals, the documentation of incoming data is mandated via the use of EHR solutions. Ideally, an organization would use a common data entry interface for all departments, from the emergency room to the finance division. Such a framework would enable an analytics solution to access multiple data point originations for comparison and analysis, effectively providing a holistic view at the operations & management levels. The reality, however, is much different: currently, 72% of healthcare organizations use more than 10 electronic interfaces to collect data.

This level of disparity between data sources is a product of environments that use individual silos of data: the accounting department collects data their way, patient biometrics are collected a different way, and so on. Consequently, there is no real standardization of data across an organization. In addition, single-function EHR systems do not have the capability to aggregate, transform, or create actionable analytics. In fact, intelligence is largely delegated to retrospective reporting which is insufficient for forward-looking healthcare data analytics initiatives.

There may be some healthcare organizations with advanced data collection capabilities, but there are few that possess advanced data integration at the intra an inter-organization level. Meaning, there are no mechanisms in place to support the sharing of data between healthcare institutions. There is an anecdotal story just across the street — data had to be printed and manually entered into the other hospital’s EHR. At the U.S. government level, there is much concern over a plan to share the EHRs of 10 million military service members from hundreds of hospitals and clinics across multiple public & private agencies — a monumental task estimated to cost at least $11 billion over a decade. 

Data integration issues are also present within healthcare institutions. For example, most internal HIT (Hospital Information Technology) systems do not offer real-time data APIs; typically, this data is processed overnight and available in the data warehouse on the following day.

The avoidance of offering real-time data analysis is indicative of the overall approach of the healthcare ecosystem to data. The reality is that organizations are rarely data-driven — there is little internal incentive to evangelize real-time data analysis. The reasons for this stem from the nature of healthcare: administration-level decisions must take into account a host of contractual, regulatory, and political decisions before being implemented. In addition, the focus of decision-makers—in terms of the use of data—has traditionally been applied to identifying volume & cost trends within fiscal reporting periods rather than the actual use of real- time data at the operations level.

2. Data Management Challenges Illustrated by the No-Show Issue

The ubiquity of no-shows has put a spotlight on a set of broader data management issues in the healthcare industry. The inability of healthcare organizations to deal with the no-show issue has had a profound effect on patient health, their experiences with healthcare providers, and on the financial bottom line. The problem is a difficult one to solve due largely to industry practices that are both archaic and ineffective. The financial and human cost behind no-shows is massive — upwards of $150 billion per year.

When a patient is unable to attend an appointment, there are multiple repercussions that affect much more than the healthcare provider’s bank account. The U.S. healthcare system loses more than $150 billion per year in no-shows alone; these costs stem largely from all of the associated issues that come into play when a patient is unable to attend an appointment. 

For example, missing an appointment means that the overhead related to that appointment is not reimbursed — items such as staffing costs, insurance, and utilities remain on the books. In addition, a significant number of appointments are made on a referral-basis — cancellations made at the primary care level means that those referrals are never made, while cancellations at the specialist level means that more revenue is lost and the patient’s health may suffer. Ultimately, no-shows have a significant impact on everyone, from physicians to patients, as physician costs increase in order to bridge the financial gap caused by missed appointments.

Beyond the Financial Cost: Deterioration of the Patient Experience

An unintended consequence of no-shows are negative patient experiences (e.g., long waits or abbreviated visits). This is due to attempts by healthcare organizations to solve the no-show problem using ill-conceived methods. For example, some health centers implement financial penalties for missed appointments. Another technique is to double-book patients; this results in short appointments (e.g., 15 minutes instead of 30 minutes) that do not give patients an opportunity to properly address their health concerns.

The people affected the most by no-shows are really the patients themselves. Preventive healthcare is responsible for discovering a wide array of potentially life-threatening diseases, but is frequently eschewed when patients either procrastinate or simply decide to cancel their appointments. For example, 18 diabetic patients with weakened immune systems can use urgent care to treat minor cuts... this does not cost much. If they decide to cancel their appointment, though, a small issue may turn into much larger (and more expensive) problem.

Guidelines to Solve the No-Show Issue

It’s been a typical, and frustrating, day. It’s 5pm and 12% of today’s 300 scheduled patients did not show up for their appointments. This means that 36 people did not appear and your staff worked to 88% of their capabilities. At this rate, you’ve been losing about $5,400 per day ($1.36 million annually) plus payroll & infrastructure costs. At the end of the day, you realize that you’ve been wasting money, frustrating your staff, and losing efficiency. Your healthcare payers are dissatisfied with your hospital’s efficiency and the patients they are in charge of are not well cared for.

A Solution

Now, imagine if we could somehow reduce the 12% no-show rate by scoring the patient likelihood of a no-show. For example, a scoring mechanism could isolate the 5% of your patients that represent 40% of those most likely to not appear for their appointment. Instead of using a reminder service to call all patients, which costs both time and money, why not allow your scheduling staff to contact specific patients, remind them of their appointment and, if needed, arrange more flexibility.

This would reduce your no-show rate down to 7% and would save your hospital $550k per year — happy patients and a less frustrated staff. So, let’s take this a step further. Instead of doing a one-time analysis of patient no-shows, imagine if you could use a predictive analytics methodology to determine no-shows in real-time.

Processing Your Data

The process may start with a computation of datasets to determine which times have the highest no-show rates. This analysis may provide some surprising insights into exactly when your patients are not appearing. Secondly, given the local time slot data combined with global dimensional data, determine the reasons why patients are not appearing for specific time slots — possible culprits could be the weather, the geography, the disease, transportation options, and/or the patient. Defining these items would enable you to create & assign time-based points to each of your patients, depending on their distinctive features.

Deploying Your Strategy

Armed with this knowledge, your schedulers could be advised, in real-time, on how likely your patients are to be a no-show based on time slot. The scheduling process could suggest 3 specific time slots when patients’ no-show scoring would be at its lowest. (i.e., when they are the most likely to appear). Combined with an overbooking strategy on specific time slots, this kind of proactive scheduling would enable your scheduling staff to suggest relevant time slots while also offering them flexibility. The end-result would be a no-show rate of only 4% and an annual savings of almost $1 million.

Looking Ahead

 

It’s clear that when a patient does not appear for an appointment, both time and money are lost. The issue has now reached a stage where the healthcare industry, as a whole, is losing billions of dollars each year. Attempts to fix the problem are really stopgap measures designed to address the symptoms.

In fact, the core issue is being ignored completely: healthcare providers are reacting to no-shows instead of proactively addressing the reasons why they are occurring. This knee-jerk reactionary approach has resulted in policies that not only do not stop the financial loss, but cause needless patient discomfort, increased waiting times, and negative doctor-patient experiences.

From charging fees for no-shows to cutting precious appointment times in half, these misguided remedies have inadvertently created contentious doctor-patient relations instead of fostering amicable & friendly relationships. Data analytics enables organizations to stop the guesswork and understand exactly when specific pa- tients are likely/unlikely to appear for any given time slot.

At its core, predictive analytics cuts through, clarifies, and conveys highly relevant information based on a wide array of diverse data. Local data (e.g., patient information and historical results of appointments) is combined with global dimensional data (e.g., transportation costs, traffic routes, weather, geographical distances, and patient diseases) to create a holistic view of the variables that are affecting no-show rates.

Models are created, tweaked, and refined until a clear picture emerges that ex- plains why patients are not appearing and, more importantly, what your clinic or hospital can do to directly address the core problem. Predictive analytics is rapidly changing the way business is done in the 21st century. No-shows are a critical issue that have a negative impact industry-wide but, at the end of the day, it is a singular problem in a vast sea of possibilities. The reality is that the healthcare industry faces a plethora of challenges whose solutions revolve around vast amounts of untapped data.

What if patient satisfaction data could be correlated with healthcare fees? What if Internet of Things (IoT) sensor data in hospitals could be used to predict medical appliance needs? What if EHR and global data could be used to predict patient non-compliance with medications?

The possibilities for predictive analytics are endless and are indicative of the world we live in: where vast quantities of raw data can be accessed, cleansed, collected, parsed, formatted, and elegantly visualized in a meaningful way. The future of predictive analytics in the healthcare industry is indeed bright and, whether the subject is no-show issues or a different challenge all together, we look forward to discussing the possibilities with you.

You May Also Like

Dataiku Solutions: How They Work and How to Use Them

Read More

5 New Dataiku Features to Streamline Your RAG Pipelines

Read More

Taming LLM Outputs: Your Guide to Structured Text Generation

Read More

From Vision to Value: Visual GenAI in Dataiku

Read More