There were many alternative health care proposals circulating before the passage of Obama’s Health Care Bill. The Republicans said none of their ideas were included and the Democrats said that some of the Republican ideas were included. If only there was an easy way to tell who’s right. Turns out there is. Many journalists are turning to Natural Language Processing (NLP) to aid in understanding political issues. Mostly these techniques are used to learn about individual bills, but another use could be used to compare documents to each other. And in particular to compare the final ObamaCare Bill to every other health care proposal that had been in discussion leading up to ObamaCare. Three real proposals come to mind. The Wyden-Bennett bill, the Massachusetts plan, and the short GOP plan. To round out our mock analysis I also included Obama’s stated goals from the outset and 2 fictitious bills, one from a fake Democratic Senator and one from a fake Republican one.

Each of these plans discuss a variety of proposals and the degree to which their take on these proposals matches what was included in the final version of ObamaCare is illustrated by that proposal’s tag cloud, the more prominent the tag the more it agrees with the same issue in ObamaCare. For example, ObamaCare and the Massachusetts health care plan share many common features, most prominently the Universal Coverage clause. Additionally since this was passed while that state had a Republican governor we’re counting it as a Republican plan.

Click for larger image

The easiest way to get datagraphics into the newsroom might be through NLP techniques and tag-cloud graphics like the proposed chart here. Word-oriented data graphics might be more approachable by traditional news media while also providing the deep content at-a-glance that all data graphics share. Only time will tell.

• Free tag-cloud generator. Just randomize a supplied set of words. http://www.tagcloudgenerator.com/
• Analysis of several alternate health care proposals. http://www.newsweek.com/id/234174/page/1
• Image for the Capitol building came from here: http://commons.wikimedia.org/wiki/File:Capitol_Building_Full_View.jpg

Consider, for example two categories of health care, cancer related spending and other non-cancer disease spending. At first blush since the cancer spending is so much higher than the non-cancer spending, it appears that should be the focus of our attention. If we could bring that cost down then we’d be saving a bunch of money. But suppose we also had access to similar data for England, Germany, Switzerland, or a European average (for example), and we then compared their spending on these areas with the US’s spending.

Now, suddenly we see a different story. Since the US is spending about the same amount per-capita on cancer treatments as the European average we should be skeptical that the US will be able to achieve significant cost reduction in cancer treatment. Furthermore while non-cancer disease treatment has a lower total expense, the US is spending more than the European average and realizing cost savings here should be easier.






This is why total spending is a red-herring. The items to focus on, the easiest savings to realize, are those where the US is spending more than the international average. Additionally, reader confidence in the numbers is enhanced if the charts cover the entire discrepancy. For example, as mentioned before the US spends $6096 per capita on health care while the European average is approximately $2723. If an analysis of European vs US spending were to sum up to something close to the total discrepancy ($3373) then there’d be increased confidence that the study was comprehensive and accurate.

However, the reality may not be so easily analyzed. Health care has many overlapping facets. One way to slice the data is like we’ve shown above, by the type of “ailment” the patient is having treated. But an equally valid slice is to follow the money and analyze where the patients dollars go. These items may include doctors’ fees, diagnostic procedures, drug costs, and etc. Some additional insights may be gleaned by looking at the health care data from that perspective. But telling the story from both perspectives would be tricky because of the overlapping nature of these expenses. For example, the full savings from a reduction in doctors’ fees and cancer treatments cannot be realized because some of those doctors fees are already counted toward treating cancer.

And this is why health care expense is one of the hardest infographic stories around. The data is vast, multi-faceted and interacts with itself in ways that are difficult to unravel. This isn’t to say that developing such a story is impossible. Indeed parts may have already been done by the Congressional Budget Office and by the staff of many members of the US House and Senate. But it’s also possible that in the rush to get that bill analyzed mistakes like the double counting example above may have crept into several analyses.

Unlike the other articles in this series I’m stopping short of suggesting a particular infographic that could have been used to tell the story. I do believe that an infographic can tell it, but without digging into the data the exact nature of that infographic is hard to predict. It could be similar to the 4th image above (Health Costs by Ailment) or it could be something more visually complex. But this story is different from the other stories in this series in another way, specifically, it hasn’t been told at all. What’s surprising is given the magnitude of this bill and the billions of dollars it will cost over the next several years and the dire need we are in of fixing our health care system that a comprehensive and simple-to-understand study has not been completed at all, either as an infographic or in text. It’s hard but possible, and more importantly necessary. Proceeding to overhaul health care without it is like planning an expedition without ever looking at a map.

This story is a natural for a data graphic. Indeed most folks probably drew a mental picture similar to the one below, clearly showing an increase in heart attacks the week after the switch to daylight savings time and clearly showing a decrease the week after the switch back. (Many might have thought the pop was bigger, but the image below is scaled to represent a 5% increase over the average trend.)

Any single year may not have been so clear cut. The story said that the change in heart attacks was about 5%, but did not mention what the normal variation from day to day was. The sample above reflects a 1% daily variation with a 5% variation on the daylight savings time change. The difference is easy to see.

It could be that the daily variation was also on the order of 5% making the annual DST change harder to see in any given year. See for example the image below.

However given a choice between informative but idealized graphic (the first chart above) or useless and irrelevant graphic (what ABC ran with), which would you use? Which would you rather see?

Finally even if the daily variation was the same as the annual DST variation, a simple and accurate graphic could have been made. If every year the switch to DST increases heart attacks by 5% then over a period of several years the trend should become clear. Imagine, for example, that every day you flip a coin, but that each March 31st that coin lands heads up. On any given year it is impossible to detect this effect, but after 10 years, you should notice that it lands heads up 5 times on March 30, 5 times on April 1, and all 10 times on March 31; a bizarre but detectable pattern. Similarly the simulation below shows this same averaging effect over a 10 year period. (Click on the image to be taken to the simulation.)

The top part of the simulation is a typical year’s data, the bottom part is the average of all the years so far. Let the simulation run for a few simulated years and the change in behavior across daylight saving times boundaries is easy to see.

Note that after a few simulated years of running the average graphic matches up well with the original “minds eye” version above. ABC news could have ran an actual 1-year graphic if that looked like chart #1 above or it could have run a several year average graphic if the 1-year looked like chart #2. Either way, they could have ran a graphic that conveyed information instead of eye-candy to accompany the spoken words.

This article is one in a series of articles that asks whether a data graphic would have altered the course of a recent news story. Actual data was not collected for this particular article (though a future article may show actual data) in part to allow us to focus on the issue of how the story would have changed with a good data graphic. Indeed without a data graphic many woman and insurance companies may have thought the recommendations meant that mammograms were 0% effective for women under 50 and 100% effective for women over 50. Subsequent interviews with the task force members made it clear that’s not what they meant, but words alone are weak tools at conveying the proper meaning here.

The graph above charts a hypothetical “effectiveness” factor for mammograms by age. Effectiveness is intentionally vague here, a real chart would undoubtedly blend many complicated factors together to get a single “effectiveness” number. Those factors might include the rate of false positives, risks and inconvenience from unnecessary biopsies, risks from unnecessary x-ray exposure and etc. But let’s assume the task force could have created a graphic like the one above as part of their recommendation. The task force might also have agreed to a “cut-off” value ahead of time, e.g. an effectiveness of greater than 50% would be recommended and one of less than 50% would not be recommended. This isn’t unusual, doctors have similar guidelines for other tests. Amniocentesis testing for example. The risk of miscarriage from an amniocentesis test is between 1 in 200 and 1 in 400, similarly for mothers between the ages of 34 and 36 the risk of having a downs syndrome child is between 1 in 200 and 1 in 400. Consequently the guidelines are for women to be tested when they’re 35, approximately where these 2 risk factors are equal. Similarly drawing a mammography effectiveness line at 50% is entirely plausible.

The hypothetical chart above shows that mammograms are clearly not the optimal screening method for women under 40. The ‘effectiveness’ ranking is below 50. Similarly it clearly shows that for women over 50 mammograms clearly are effective. But this middle ground between 40 and 50 is tricky to describe. Technically it too falls under the threshold, but only just. If you were on the task force and saw this graph in your mind’s eye and were forced to explain your recommendation in words alone, how would you do it?

Once you include a graph the question naturally turns to other graphs. The graph above is for the “average” woman. How would the graph look for women with a family history of breast cancer? How would it look for women with no risk factors? The graph below shows these possibilities. In this case women over 40 with a family history are shown above the 50% line and women over 40 with no risk factors are clearly below the 50% line.
If the Task Force’s report were issued with (actual versions of) these graphs (and if the news media carried them) the weeks long national firestorm that accompanied the announcement could have been minimized.

Now they are suggesting they might use a device which has never been used the for this kind of major systemic reform. We know it would be — the only thing bipartisan about it would be the opposition to it, because a number of Democrats have said, “Don’t do this. This is not the way to go.” — Senator McConnell on FoxNews

Recently the Sunlight Foundation ran an infographic examining the past 20 years of Senate Reconciliation bills. At a glance you can see which bills had bipartsian support and which didn’t, and the list is relatively small (13 bills) so deeper inspection can be had relatively easily. However, what’s not immediately obvious is an indication of what Senator McConnell alleges, namely the “magnitude” of these bills. Most bills seem on their face to be simple budgetary adjustments. The “Balanced Budget Act of 1997″ had wide bipartsian support and a relatively simple title. However the “Jobs and Growth Tax Relief Reconciliation Act of 2001″ had hardly any bipartsian support, perhaps more tax cuts were added in reconciliation than in the original bill? If the graph could be altered to show some sort of significance factor then that would be an infographic!

This significance factor or magnitude could be quantified by providing some sort of comparison of the changes against the original bill. By way of example, there are well established computerized ways to compare documents and a significance factor could be calculated by comparing the size of these changes. Bills with bigger “change files” could show up with fatter lines in the graphic above. Of course this technique would have some problems.  It could give false positives if there were a lot of words to describe a relatively minor change and it could give false negatives if a massive portion of the original bill was removed (describing a removal is a fairly easy task, essentially “delete lines 1-1000″). But even a flawed mechanism in the hands of knowledgable people can be a useful tool as knowledgable people can quickly weed out the false positives, reinstate the false negatives and focus the viewers’ attention on the issues that matter.

Such a tool could be useful here if we were looking at hundreds of bills, but we’re only looking at 13. A responsible journalist would have prepared for an interview with Senator McConnell by digging into these 13 other bills and asked McConnell to choose which of them would take 2nd place behind health care for “significant bills passed through reconciliation”. McConnell could then use that as a spring board to describe how much more of a change health care is from that previous “high water mark” or the question could reveal how hollow McConnell’s talking point was. Instead, we, as consumers of news, get neither.

Someone very close to me has been experiencing an incredible run of bad luck. Every day for the past few years my friend (let’s call him Paul) gets mugged. The robbers punch him then take his money but they leave his wallet. He’s not hurt enough from these muggings to go to the hospital, but he is sore for the rest of the day. He can never predict when these muggings will occur, sometimes it’s early in the morning, other times early evening or the afternoon, but strangely enough he’s never been mugged twice in the same day. He’s tried altering his route to work, taking a self-defense class, and in general doing everything he can to make himself a less appealing target, but nothing helps. He has found though, that if he has no money in his wallet he gets beaten up badly so he’s taken the tactic of keeping $20 in his wallet just to appease the robbers.

By and large Paul has gotten used to it. He keeps a supply of pain meds at home and his office and he goes out of his way each day to keep his wallet supplied with a $20 bill. Minor inconveniences to be sure, but he figures he’s losing about $6000 each year to these muggers and once a week heads to the police office to lodge a complaint.

The police tell him they’re doing all they can to catch the robbers, but they’ve had no success. Nevertheless they’ll keep trying. On his most recent trip to the police station, one of the inspectors, we’ll call him George, made an interesting proposition. It would cost the police $80,000 per year to have a policeman following him around all day long. And even that won’t guarantee success at stopping the mugging when the police weren’t around. So the investigators would rather focus their investigation on the larger community and not just on him. But to compensate for his monetary loss the police department would like to pay Paul $100/week while they pursue the true culprit. Sure the extra money won’t quite cover Paul’s losses (he’s frequently robbed on the weekends too) and it won’t compensate him at all for the pain and suffering he experiences by the light beating he endures during each mugging, but it’s better than nothing.

Now lets modify our basic story by a little.

One time during his weekly trip to the police station Paul hit on an idea. He’s going to suggest to the inspector that they compensate him for the money he’s losing. After all it is the responsibility of the police to keep him safe. Over the past 3 years he’s lost about $18000 and he sees no reason why he should bear this cost when there’s nothing he can do to avoid being mugged. He wants as normal a life as possible and losing $6000 of his discretionary cash each year just doesn’t seem right. When he proposes this to the inspector, George at first thinks it’s a good idea but needs to check something on his computer first. When George returns he has bad news for Paul. Turns out that the muggings started before he became a tax payer so he’s not eligible for victims compensation since it’s paid for by tax dollars.

While the above story is fictitious, I know someone who lives with a very similar run of bad luck. He has type-1 diabetes. Unlike type-2 diabetes which is related to factors people can control, namely diet and exercise, type-1 diabetes is an auto immune disease that strikes without warning regardless of your lifestyle or other controllable factors. He figures that he spends about $20/day in diabetic supplies, needles, testing strips, insulin. His insurance company covers some but not all of these expenses. And, of course, he tests himself (finger poke) 4-6 times a day and gives himself 3-5 injections per day. The analogies to poor Paul above should be clear. Fortunately my friend had insurance when he got diabetes, but in the US if he didn’t have insurance he’d personally have to pay for all his medical care and furthermore he’d find it difficult to find insurance afterward.

Poll: Does it make a difference?

I’m not going to run a poll on whether your opinion is changed based on the situation being crime or health care. (If you have an opinion, please leave a comment.) Instead I’ll share my opinion which is both blatant and nuanced.

Blatant:
Health insurance isn’t some form of a deferred savings plan designed to allow you to pay your own medical bills. Many people run their entire lives with low total health costs, others are afflicted with a chronic disease relatively early in life or for many other reasons have large medical bills. Health insurance is there for risk mitigation, everyone pays the average amount so that those with above average expenses can have those paid for by those with below average expenses. Everyone pays, everyone is eligible, no one can be denied coverage.

As a society, we have the ability and compassion to help those who are affected by a disaster to get back on their feet and resume as normal a life as possible. Folks from the US and around the world help those affected by flood, hurricane, tornado or earthquake rebuild their homes and resume their lives. Of the industrialized countries only America denies this sort of aid to those experiencing catastrophic disease or injury.

Nuanced:
Of course, if you make something free or low cost or fixed cost people’s behavior will change from what they would do if that same thing had a variable cost based on their actions. This axiom may even apply to health care. However there is a key difference between this and say “free housing”. Poor health is a cost in and of itself, while the dollars to treat diseases may be free, everyone would rather be disease free than to be disease ridden but have those diseases treated freely. Even with free health care there won’t be a sudden increase in people playing Russian Roulette.

But Russian Roulette is an extreme example. More realistically, some smokers may chose to continue to smoke with free health care while others (those who currently aren’t regularly seeing a doctor) may choose to quit. Some people may indeed choose to abuse the system. The question isn’t whether there will be some abuse, but will there be too much abuse. If even one person tried to get a free ride from free health care would that make you be against it?

Shatner - “Here’s my premise and you agree with it or not: If you have money, you are going to get health care. If you don’t have money, it’s more difficult.”
Limbaugh - “If you have money you’re gonna get a house on the beach. If you don’t have money you’re gonna live in a bungalow somewhere.”
Shatner - “Right, but we’re talking about health care.”
Limbaugh - “What’s the difference?”

Shatner, to his credit, wasn’t having any of this, immediately pulling the topic back to the health care issue and away from analogies. However Limbaugh’s response confirms my belief that the far right considers the free market the best solution for every problem.

I agree with Rush’s point of view for discretionary health expenses like cosmetic surgery (not to be confused with cosmetic surgery to correct deformities). If you got the bucks for a face lift or nose job great, else get a bungalow somewhere. I also might have agreed with him several hundred years ago when the GNP wasn’t to a point where public health care was economically feasible and when the US would have been the first country on the planet (I believe) to create a public health care system. However, where I disagree with him is every other aspect of health care.

After 9/11 the US government set up survivor funds to help mitigate the disaster that befell those families who lost loved ones and bread-winners in the attacks. Each year countries around the world offer aid to regions affected by earthquakes, hurricanes, tsunamis and other natural disasters. In a real sense this aid seeks to re-establish the status-quo for those affected. Is there any difference between these calamities and the onset of diabetes or leukemia? Yet even here the analogy breaks down but, I believe, in health care’s favor. Excepting for the 9/11 attacks regions around the world are known to be earthquake or hurricane prone. People do have a choice (albeit perhaps a very small choice) of where they live. People do not have a choice on their genes. Modifying their lifestyle won’t help avoid leukemia, MS or Parkenson’s disease.

In 2002 John Allen Muhammad and Lee Boyd Malvo went on a shooting spree in the greater  DC area. Randomly and without warning a bullet from Mr Muhammad’s sniper rifle could alter your life significantly or perhaps even kill you. We all felt outrage at the random injustice of someone being struck down by Mr Muhammad’s actions. Yet is there any difference in what Mr Muhammad did and what random chance does when we contract diseases that can strike us without warning and significantly alter our lives?

When I hear Rush Limbaugh comparing excellent health care to a beach front house that only the wealthy can afford it is not that big a leap for me to envision what he’d advocate to the residents of DC during Muhammand’s rampage. The wealthy could afford body guards and body armor. They could go about their business relatively risk free. In Rush’s world he’d be in favor of those who can afford it wearing body armor and those who can’t doing with out or perhaps even staying at home (if you can’t protect yourself then you shouldn’t be venturing outside). In this alternate world it is not that big a leap to envision how Rush would criticize these would-be victims for going out unprotected. “They knew there was a sniper out there. They chose to go out without body armor. That they got shot is their own fault.” The correct thing to do, of course, is what was done. Public dollars funded a police manhunt for the DC sniper. But in Rush’s world he’d rather spend his dollars to protect himself and no one else.

Financially, can the US afford to offer “free” health care to all its citizens? If we can’t afford it, then that’s that. Health care is a “nice to have” but I wouldn’t want to bankrupt the country to get it. Fortunately the answer here appears to be “yes” mainly because so many other industrialized countries are doing so now. Furthermore since we’re currently spending 2x to 3x more than the other countries in health care, there is a strong chance that, if done right, we’d reduce our health care expense by reforming the system.

Socially, will free health care demotivate people from having a strong work ethic? With free health care will we become a nation of free loaders? The answer may cut both ways. I wouldn’t be surprised to find that drug abuse and risky behavior (cigarette smoking) is more prevalent in the poor than in the rich. Some of the poor may engage in these behaviors because they’ve lost hope. Free health care may restore hope to some of them and return them to being productive members of society. But this is an easy number to get at. One could examine data from countries before and after the adoption of public health care or one could run a trial in this country.

Commercially the concern would be, would a public health plan demotivate the health care industry to be less innovative? Note, for clarity here, I’m referring to drug research, disease research and whatever motivates people to become doctors and nurses in the first place. (This excludes the health insurance industry which is in the risk mitigation business and not really in the health industry.) If the US had a strong public health system would there be fewer dollars or in general would the pace of innovation in the health industry slow down? I do not know, but this also can be tested. Are companies based in countries with a strong public health plan as innovative as US companies? Where are the new drugs coming from? Where are the new research papers coming from? For those countries that we have records from before they adopted a public health plan, can we detect a change in health productivity? Is there a doctor shortage in these countries?

There’s a final commercial aspect to this. How many would-be entrepreneurs in this country are stuck at their current jobs because they’re afraid to branch out on their own for fear of losing their employer based health insurance? On the one hand if someone believes that there are a large number of such would-be entrepreneurs then free market logic would indicate that some agency would step up to the plate and offer group based insurance to entrepreneurs. After all health insurance is largely available to any large body that can negotiate group rates.  But just because some agency hasn’t yet risen to meet this need doesn’t necessarily mean the need isn’t there. This one may be harder to quantify via comparisons to other countries because the US may already be above the curve in entrepreneurship. But it’s a factor that needs to be considered.

Shatner - “The sum total of what I want to ask you, politically, is HOW DO YOU KNOW?”
Limbaugh - “It’s my job. It’s my life. It’s my career. It’s my passion.”

Rush is claiming to be an expert on everything yet he’s offering no proof. He’s essentially saying “I’m an expert, trust me.” Experts who are truly worth their salt also know how to explain the issues to novices. Arguments that are fact-free are just shouting matches. Rush’s answer to Shatner’s question shows that he has no data. Asking us to trust him because he says he’s an expert is incongruous to the Right’s mantra to think for yourself and be self sufficient.

–

** By easy I mean in comparison to a $1 trillion health care package. It may still be a difficult number for NewsWithNumbers to get.

In earlier articles we discussed the effectiveness of the health care system, precisely to test the Right’s hypothesis. The measures for effectiveness we’ve used are: longevity, infant mortality, 5 year cancer survival rate and preventable death rate. In all but one of these the US has ranked last, and interestingly, on the other the US ranked first. This is a somewhat confusing result, so in this article we’ll combine all 4 of these measures into a single “health care effectiveness” metric and see how the US compares to other countries.

Before we begin it is important to note that we’ll give up some things to get a single measure of “health care effectiveness”. One thing that we’ll give up is meaning. The very nature of combining a cancer survival statistic with a longevity statistic means that the resulting number will be just a number. It will not have any simple connection to the real world. The number won’t mean percent of people living past a certain age, it won’t mean years of life after cancer, it won’t mean anything other than higher scores are better than lower scores. Again the point of doing this isn’t to get a number it’s to get a sense of how the US compares to other similar countries around the world. And with this in mind, an arbitrary or a meaningful number will do equally as well.

How to combine these 4 quantities? For any given country, we could just add them all together and divide by 4. But there are problems with this approach. Let’s take a broad look at all the numbers:

Instead of simply averaging, we’re going to first scale each individual quantity to make it fit from 50 to 100. The worst score in each category will be assigned a value of 50 and the best score will be assigned a value of 100 and the remaining values will be linearly scaled to fit in between. Note this also “flips” those scores where the best score is a lower one (like infant mortality). After we’ve done this then we’ll average the scores. Why a low of 50 and not 1 or zero? No real reason except to not lead the casual reader astray. Recall many of the earlier articles included the rankings of less developed countries. Even the lowest scoring developed country scored significantly better than the average developing country. So we’re going no lower than 50 here merely to remind the reader that other countries not on this list may well score lower than 50.

For those interested in the details, a few tables at the end of this article include the data used. Note here we are using the “adjusted infant mortality” number, where we ignore infant deaths occurring less than 24 hours after birth. This was discussed in an earlier article. Without further ado let’s get to the results.

The graph below shows the results for the 11 countries that were in all 4 studies. By this Combined Score Japan ranks first, with France and Sweden close together for second place. In last place is the United Kingdom with the US in the 2nd to last place. Some may have been expecting the US to score last, some may find a measure of vindication that the UK is in fact in last place. But is there anything to be learned from this ranking?

Let’s examine this by looking more closely at the components of the combined scores. The graph below is a little busy but it’s worth studying. It shows for each country the value of each component of its combined score. Japan has excellent marks in 3 of the 4 quantities and a moderate mark in cancer survival. Contrast this to France and Sweden. Each has an excellent mark in one area and moderate marks in 3 of the other areas. Japan really seems to be doing something right here and we’ll see just how well they stand out in a moment.

On the other end of the scale we have the US and the UK. The UK scores poorly (below 65) in all 4 areas, while the US scores dead last in 3 of the 4 areas (all 3 marks are in the same location so they’re impossible to distinguish) and first place in one of them. It is this first place showing that catapults the US above the UK. Indeed, a score of 83 on the scaled cancer survival rate would have left the US tied with the UK, so a score of 100 really pushes it ahead. But, for health care, which of those 2 countries would you prefer to live in, the US or the UK? It may depend on what stage of life you are in. Adults with grown children may prefer the superior cancer survival feature of the US and ignore the only slightly better longevity score the UK has. But by this measure my answer is neither.

And let’s drive this point home by concluding with a “dot plot” comparison of these results. Here the lead that Japan has over France and Sweden is very clear. Similarly the lead the US has over the UK is quite noticeable, but also noticeable is how far the US and UK are from the rest of the pack. To be sure, this is only 11 countries. Remember that only countries that showed up in all 4 measures are reported here. There are several countries that scored poorly on 3 of the 4 measures (Portugal and Denmark for example) which may fill in the gap between 2nd to last place US and 3rd to last place Netherlands. So don’t read too much into the size of the gap.

In terms of health care effectiveness, Japan, France and Sweden should be our role models. And anyone using the UK as a typical example of what “socialized medicine” will bring, is intentionally choosing the worst example.

Post Script

Just today the US released a new report that ranked the US infant mortality rate a bit worse than the data used above. This article points out that most of the poor showing of the US is due to premature births. Indeed we discussed this very fact several months ago because there was a conjecture that the US counts live births differently than other countries. The statistic we used above removed all infant mortality data for infant deaths occuring within the first 24 hours to offset this possibility. This updated study includes information on which countries count live births similar to the US. In short most of them do. (It also included infant mortality rates for premature births, data that wasn’t available to us earlier.) How does this affect our conclusions?

Recall that on two of the 4 statistics we used, we “leaned Right”. The cancer survival rate statistic may have a significant component due to the “lead time bias” where people aren’t really living longer, just having their cancer detected earlier. We acknowledged this possibility but since we had no way to quantify it we ran with the data as is. Also, as mentioned earlier, the infant mortality statistic was altered to adjust as best we could adjust, to varying reporting measures in different countries. It just turns out that this had a favorable result for the US. But it also may have masked some legitimate health concerns in the US.

Adjusting for these factors and adding in the new infant mortality statistics would undoubtedly lower the US ranking. But look at the charts above one more time. It’s unclear if the US would be reduced below the UK’s ranking, but that’s beside the point, isn’t it? If you’re on the Left, aren’t the reported statistics bad enough? If you’re on the Right, can you really come up with more reasons that will significantly improve the US’s ranking? For both, isn’t the US already sufficiently bad to be worthy of a change to its health care system vs the status quo?

Tables

Table 1 – Original Scores (sorted alphabetically by country)

Table 2 – Scaled Scores (sorted by rank of Combined Score)

In this article we’ll make the point even more succinct. For the purposes of comparing the effectiveness of a health care system, the cancer survival rate of a nation is ideally shown as a single number, not one number per cancer. Can we get there? The short answer is yes.

How would you blend the survival rate of breast cancer with the survival rate of colon cancer to get a single survival rate? One way would be to simply take the average the survival rates. This is the normal average, and it assumes that breast cancer and colon cancer should be valued equally. But valuing them equally may not be fair. Each year 200,000 women die from breast cancer in the industrialized world while 165,000 die from colo-rectal cancer¹. Also of the countries studied in the original report 145,000 women are diagnosed each year with breast cancer while, 62,000 are diagnosed with colo-rectal cancer². Clearly breast cancer should be weighted more heavily in a blended average, but which weighting should be used? This is essentially an arbitrary call. Since we’ve already used the mortality rate to establish these cancers as being significant, for the weighted average we will weigh based on the incidence rate.

One of the tenants of News With Numbers is that graphs should make complicated things clearer. In the case where an effect clearly stands out then graphs make it easy to see but in the case where the effect is more moderate then even a graph may require the eye of an expert to discern the difference. But, given the hype surrounding the health care debate, either is news. Either the US has a clear lead in some health care fields or it does not. On a cancer-by-cancer basis the earlier study showed the US in the top, but tied with several other countries for top honors. Not clearly in the top alone.

Below is a table that shows the weighting effect based on incidence rate. The original study provided statistics both on the survival rate as well as the incidence rate. Ideally we’d weigh based on the global incidence rate or at least the incidence rate in the industrialized countries, however, the US was by far the largest country in the study and using the incidence rate data from the study would effectively mean using only the incidence rate in the US. Thus for this comparison we’ve used the incidence statistics for every country except the US.

Below is a dot-plot of the above data. The US is significantly ahead of 2nd place Canada and 3rd place Australia. How did the US go from “tied for first place” to such a clear lead? The US didn’t have such a clear lead over the other countries when these data are examined on a per-cancer basis, but collectively the US pulls ahead because it was alone in having top marks across all the cancers studied. In a competition that’s scored by cumulative marks, it is possible that the overall first place winner could merely have placed 2nd in every event. As long as no one placed first in more than half the events, those who consistently place 2nd in all events can be the overall winner. That is essentially what’s going on here. The US was first in Breast cancer survival, with Canada #2. The US was first in Prostate cancer survival with Canada #3. The US was #2 in male ColoRectal cancer survival with Japan #1 and Canada #4. And finally the US was #2 in female ColoRectal cancer survival with France #1 and Canada #3. Any country with a higher survival rate than the US was ahead of the US only once.

Below is a chart that effectively proves this point. It shows the same information, but this time also weighted as in a normal average and by the mortality statistics. While US’s lead is larger with the chosen weighting, it is still quite apparent in these alternative weightings. (The US and Canada are ranked #1 and #2 in each graph.)

Conclusion

The previous article concluded saying while the US did have better cancer survival statistics than the other countries, that lead didn’t appear to be significant enough to make the case that it is a shining example of the US health care system. Here, though when combining all the scores, the US pulls even further ahead to the point where its survival rate is indeed noteworthy. Skeptics have to agree that the US scores unusually highly here, but subject to the same lead-time bias criticisms that the study’s original authors warned against.

However, of the 4 measures we’ve studied in earlier articles (Longevity, Infant Mortality, Preventable Deaths and Cancer Survival), this is the only one that shows the US in a favorable light. Next week we’ll combine all 4 of these measures into a single “health care effectiveness” number and see how the US fares there.

Notes

1. Mortality rates are on page 3 of this document

2. Incidence data is from Table 1 of the document available here. The total was computed by adding the incidence statistics for each country listed. We excluded Cuba for the same reason as the original article and Switzerland because it only had info on breast cancer. Also, some countries reported countrywide statistics and additionally city/region-wide registries. Where this occurred (notably England) we chose only the countrywide statistic. We did not adjust for the reporting interval differences in some countries as it will not affect the final result.

The standard way these sort of statistics are graphed is via bar charts. An advantage of bar-charts is it shows the total quantity. If you’re interested in the distance a quantity is from zero then highlighting the entire span of the data from zero is a useful thing. But if you’re interested in a comparison to others then the distance from zero is not so useful. The actual ranking of all 19 countries in the study is shown in the 2 graphs below. The graph on the left is the standard bar chart representation of the data. The graph on the right shows the same information but in a slightly different format. First it is zoomed in on just the range where the countries lie; it doesn’t go all the way back to zero. Next it just plots the position of each country. France, for example had a preventable death rate of 64.79 (slightly less than 65 people out of 100,000 died in 2002 due to causes that the authors called preventable). France’s ranking is shown as a red cross at position 68.79.

If we remove the country names the clustering becomes more clear. France is significantly ahead of 2nd place Japan. Then a big gap between Austria and Germany, next another big gap between New Zealand and Denmark and finally a big gap between Portugal and the US. School teachers who grade on a curve may be tempted to assign an A to the first group, B to the second and so on. The US would be in the last place “F” group. This is pretty dire for the US. Almost any rearrangement of the statistics would show the US in a more favorable light.

Keep in mind that one hypothesis in the health care debate is the US has the best health care system in the world. If that’s true then at worst we’d expect top marks in some areas and moderate marks in other areas. Moderate marks are not inconsistent with the best health care system in the world. The US is a diverse country and has a much more diverse population than many European countries. It could be the case that, where the US ranks only moderately, if you adjust for this diversity the US could again rank in the top. Unfortunately there have been no studies to this effect, but the effect may still exist.

However, we wouldn’t expect to see the US in last place on any of these measures. Even worse, we wouldn’t expect the US to score last by a significant margin. If the US scored closer to Portugal, then visually the US could be lumped in with the rest of the countries that scored a “D” and the case could be made that this is not a last place score but instead is a low but moderate score. However it is much harder to make that argument with the actual ranking of the US. So the US ranking is pretty dismal. But let’s examine the data a little further.

What’s a preventable death? It is fair to say this is a judgment call. But all of us can agree that there are some conditions that should not be fatal or fatal only rarely. People shouldn’t die from surgical errors and conditions like pneumonia and appendicitis, so most would agree these deaths are preventable. (Note we’re not talking about preventing the conditions themselves, but rather preventing death from these conditions.) The study’s authors used their judgment in a similar way to categorize many other conditions as preventable. For example, deaths from other conditions that are chronic but should allow for a long life like epilepsy and diabetes were also considered preventable. Less obvious are things like cancers and heart diseases and there can rightly be some disagreement on whether these should have been included however they were included for all countries so any error is thus mitigated. (Furthermore the authors of this study were following up on an earlier study and so listed the same conditions as preventable.)

The actual study did a deeper comparison between the US, UK and France, breaking down the data between various conditions and also comparing the first year of the study (1997/98). Looking at that data one would notice 23 points of the 45 point gap between the first place France and last place US comes from heart disease alone. One blogger thinks this fact should rule out this study as being flawed. However, if the US’s scores from the 1998 study improved at France’s rate of improvement, the US would have placed 15th, just ahead of Denmark (improve). And if the US had France’s overall scores on everything but heart diesase it would have placed 12th, just ahead of Germany (replace).

Certainly the difference in the population between the US and France could account for the US not getting first. But it’s harder to make the diversity argument when referring to the rate of improvement. The best health care system in the world should have a rate of improvement over a 5 year period at least tied with France.

Notes:

At the time I researched this article the original study was available for free on their website. Now the link I have is showing up as a for-pay link.  My apologies.