There is a core concept shaping the ‘market’ in health, the concept of an assay. Few doctors or patients understand it. This article explains what assays are, how they entered healthcare and the consequences of failing to grasp the role they play.
This post by Harriet Vogt and David Healy is an illustrated version of a peer-reviewed citable academic article Randomized Controlled Assays: A Category Error with Consequences that like the illusions below will look instantly obvious to some but impenetrable to others. Thanks to Bob Whitaker it has also appeared on Mad in America and the article ties closely into the post Has HealthCare Gone Mad.
Before Thalidomide
By 1950, we were starting to get the first new drugs that worked so well no Randomized Controlled Trials (RCTs) were needed to show they worked.
Who said no RCTs were needed? Tony Hill, who in 1947, created RCTs. For Hill conversations between doctors and patients were key. If we are taking a medicine, we depend on someone able to look closely at and listen intently to us. If, instead, we insist on randomization, placebos and double-blinding, Hill said, we’d have lost the plot. RCTs, besides, generate average effects that told him nothing about how to treat the person in front of him.
Louis Lasagna, a big Hill fan, was the leading American cheerleader for RCTs in the 1950s. He had an unfortunate idea. There were still some old drugs around that didn’t work and RCTs might help eliminate these.
This idea went nowhere, until a birth defect crisis triggered by thalidomide, a sleeping pill, struck. In 1962, needing to be seen to respond, politicians adopted Lasagna’s idea of getting industry to prove their drugs work using RCTs. They overlooked an RCT of thalidomide Lasagna had already done, which showed it worked and was free of side effects. Thalidomide had sailed over the hurdle being put in place to prevent a repeat.
After Thalidomide
For two decades after 1962, neither regulators, doctors nor industry understood RCTs. Doctors had been evaluating drugs successfully for a century without RCTs. They knew little about these new trials they were asked to run. FDA bureaucrats knew even less. Industry knew nothing but had an incentive to learn fast.
Doctors ran RCTs for industry, comparing new drugs to old assumed-to-work drugs. When both looked roughly the same, they said “this drug works”. Maybe it did but these RCTs didn’t prove that.
Some RCTs were well designed with important outcomes, such as how many people died on new and old drugs. Trials like this take nearly a decade to run, which is a non-starter for a company hoping to get a drug to market quickly. Worse again in some good RCTs potential industry blockbusters killed more people than the older drugs.
It was time for industry to intervene. They took the running of RCTs out of the hands of academic doctors and gave them to contract research organizations (CROs), whose mission was to get them done quickly. They gave the reporting of trials to medical writing companies whose mission was to get the product in print quickly.
But the most important breakthrough came from the FDA. Paul Leber, the man responsible for approving psychotropic drugs, said “wait a minute, comparing a new and old drug is nuts—neither might work. You have to use a placebo for us to know if your Assay is sensitive enough to pick up the drug’s effects”.
Suddenly, companies were no longer doing RCTs, no longer notionally expected to prove their drugs didn’t work. No longer in the business of demonstrating their drug saved lives or got people back to work. They might be randomizing, and controlling with a placebo, but they were doing Assays—Randomized Controlled Assays (RCAs).
What’s an Assay?
FDA approves Food and Drugs—on the basis of assays. The assay standard for butter is at least 80% butterfat. Chocolate must have over a quota of cocoa solids and under a quota of vegetable oils. When you put a dipstick in your urine to test for glucose, or protein against a standard the dipstick carries, you run an assay. If you want to call a product something, the US Institute for National Standards tells you what the accepted standard is. FDA assays possible butters, chocolates and drugs to see if they meet a standard.
Scientific experiments, like clinical trials, may have standards to avoid mistakes but they don’t operate to a standard any more than a clinical interview has. Standardizing clinical interviews risks disaster.
Science explores, bureaucracy tests. RCTs aim at an external validity that will inform clinical practice. RCAs aim at an internal validity that allows a regulator to tick a box and approve a label for a medicine.
Leber left ‘Randomization’ and ‘Controls’ in place in the new Assays. Seeing them doctors saw RCTs and later Evidence Based Medicine (EBM).
Companies saw meeting a standard agreed with regulators in an Assay. Show us an Assay with a 2 point on average improvement in Hamilton Depression Scale scores as its outcome and we in the FDA can tick the box that lets you call this an antidepressant.
Doctors had the illusion of seeing a Rabbit. Pharma and regulators saw a Duck, but also saw advantages in ensuring doctors do not spot the Ducks. And they don’t.
FDA and companies converged on replacing explorations of drug effects or clinical benefits with tests. Is there a change on standardized interviews (rating scales) or dipsticks? A minimal fall in an average rating scale or dipstick score lets FDA license a company to use words. They can stick words like antipsychotic or statin on drug labels and in adverts.
In 1962, figuring RCTs revealed the truth about a drug, academics had drafted regulations saying FDA could license a drug if there were two positive RCTs. Pretty certain a positive RCT showed a drug worked in the sense of produced a clinical benefit like saving lives, they were absolutely certain two did.
For 20 years after 1962, dealing in RCTs, FDA had preened itself as regulating by science, but it now transitioned back to being a bureaucracy. Looking at the 2 trial criterion, Leber said – “we can tick a box if a company has done 100 Assays and 2 are positive”.
To enable this to happen, a new group of drugs, the selective serotonin reuptake inhibitors (SSRIs) forced regulators into a creative maneuver. SSRIs were never meant to be antidepressants. They are essentially anxiolytics, but anxiolytics were getting a reputation for causing dependence. An antidepressant label offered better marketing opportunities. Convert cases of Valium into cases of Prozac? Of course, we can do that!
Running Assays in people diagnosed as depressed, however, was a problem. Unlike tricyclic antidepressants (TCAs), SSRIs don’t help severe depression (melancholia). SSRI Assays had to be run in mild depression, where it was difficult to beat placebo. Fortunately, however, if the SSRI didn’t beat placebo, the TCA likely didn’t either. FDA could then pronounce this as a failed exercise, where failure meant it lacked assay sensitivity.
There is no such thing as a failed RCT. Demonstrating a drug doesn’t work advances medicine. But assays can fail. If we know a drug works, or a patient has diabetes, but the assay system doesn’t pick it up, we need to change the assay parameters to pick up the known effect.
Dipsticks and rating scales made this a lot easier. It is difficult to say a treatment works if dead bodies are the outcome measure and there are more on treatment. But if the ‘it works’ assay standard is a dipstick or rating scale change, the ‘it works’ box can be ticked even if there are more dead bodies on the new drug. In the last 40 years, almost no set of assays that have led regulators to license a drug have shown lives saved or serious injuries avoided.
Between having to provide only 2 positive assays, being able to write off failed assays, and having a criterion for working that has limited contact with the real world, companies could now get weaker and weaker drugs licensed. A system supposed to eliminate Snake Oil, now encouraged the production of Snake Oils.
Looking at this New Deal, industry saw a beautiful siren. For those on the receiving end of debatably beneficial treatments with hidden hazards, this illusion might show a different face.
Tail Wags Dog
When politicians try to regulate drugs they claim to be regulating industry not medicine. In 1951, as the first really effective drugs came along, they were made prescription-only. Otherwise, companies claimed, they would have to put an entire medical course into a drug label. As a result, we could only get the new miracles from a doctor.
Prescription-only is a police function first introduced for heroin and cocaine in 1914. This is why consulting a doctor can feel like being nice to a policeman.
From industry’s perspective, 1951 also transformed doctors into the consumers of drugs. They consume by putting drugs in our mouth. Rather than having to market to 329 million consumers in America, pharma markets to less than a million. Companies spend vastly more on these medical consumers than any other industry spends on the 329 million, and figure few doctors have a thought in their head that is not put there by their company or their competitors..
There is roughly one cellphone per person in the US. On average every American has 22 prescriptions per year. If each prescription is once monthly and for 5 medicines, this is over 3000 pills per year, some of which cost more than the latest cellphone. Where do company marketing dollars go?
Industry claims that a drug costs several billion dollars to bring to market. This claim has led many to try to establish how much of this is marketing and how much research. Although good figures are hard to come by, the general conclusion is its mostly marketing (to doctors).
It may all be marketing. Big Pharma do little research. They buy small companies who discover new compounds. Their RCAs are commercial exercises, not science, and not research.
Glossy adverts in medical journals or sponsorship of free lunches look like marketing. Those who see healthcare going down the tubes get indignant about these adverts, saying all would be fine if there were no free lunches and doctors were guided by the evidence.
Doctors adamantly insist they are not influenced by adverts. They go by the evidence. Academics even boast that EBM shackles the pharmaceutical industry. Industry take them at their word. If there were no free lunches, glossy adverts or sponsored CME, industry would have to invent them. The indignation these sales (not marketing) techniques create distract from the marketing, the latest ‘apparent’ RCT that the adverts sit beside,
The published RCAs that influence doctors so much and look like RCTs are ghostwritten. Neither the ghostwriters, nor the notional authors, nor the regulators get to see the data from these commercial exercises. Information about what happened in these assays is protected by commercial confidentiality requirements rather than by clinical confidentiality.
When we read that companies will not show our data to anyone, we happily sign consent forms for these commercial exercises. In this way, we put everyone else who is injured by the drug in a state of legal jeopardy because companies can claim nothing like happened in our studies – it’s your anecdote against our evidence.
According to legal and clinical definitions of evidence, there is no evidence in company assays. Legal evidence means someone who can be examined and cross-examined to establish what happened. Did this man, coded as having burns, pour gasoline on himself and set fire to it in a suicide attempt? In a clinical study, people are the evidence.
Not all the people in commercial exercises even exist. Doctors have been jailed for entering non-existent patients into company assays. The people who do exist are increasingly likely to live in Africa or South America. We could manage the evidence issue if we had clinicians who had treated them and could testify. But the authors on the authorship line of any article reporting the results will be American, none of whom will have seen the effects of the company drug on either the far-flung, or even American, patients.
Fraud is a real possibility and it’s less hidden than people might think.
In 2001, a GSK paroxetine Assay in depressed children (Study 329), was published in the best journal in child psychopharmacology, trumpeting paroxetine’s benefits and safety. Doctors rushed to put children on Paxil.
In 2002, applying to the FDA to license Paxil as an antidepressant for children, GSK submitted 3 assays, one being Study 329, stating all 3 were negative. Agreeing the 3 Assays were negative, the FDA agreed to license Paxil for children and not to mention the negative assays in the Paxil label.
A few years later Erick Turner, ex-FDA, published a paper on antidepressant assays done in adults. Almost half of these were negative but like Study 329 many of these negative assays had been published as positive. If the FDA know a significant amount of the published medical literature is fraudulent, why don’t they say anything?
Simple. The FDA tick boxes that let companies make claims. They police advertisements claims but policing the medical literature is not their job.
Whose job is it?
Medical Journal editors see FDA approvals. While they know the articles are ghost-written and have seen companies charged with fraud, they do not view it as their job to police the medical literature either. In 2009, the New England Journal of Misinformation made clear that ensuring the integrity of published data is not their brief.
In 2022, one of us (DH) emailed 300 Professors and other Medical Grandees, who had reviewed articles for NEJM that year. The email drew their attention to the ghostwriting of the vaccine assays, clear evidence that vaccine hazards had disappeared from the published articles and NEJM’s public unwillingness to tackle this. There was no response. A Sodom and Gomorrah for our times?
The Cochrane Collaboration was established in 1992 to gather and sift the published ‘evidence’ so we really knew the truth about the drugs we used. They had an opportunity to say we will only count studies where we have access to the data, but they didn’t grab the opportunity.
In 2004, the evidence that pretty well the entire pediatric antidepressant literature was fake, with GSK sued for fraud, caused Cochrane problems. But Cochrane was wedded to the idea that RCTs produce the truth, whether done by industry or not, and, as by then company studies formed the backbone of EBM, Cochrane held its nerve and continued to collaborate with Industry.
In 2005, Iain Chalmers, the head of Cochrane, testified that ghostwriting is rare and only happens in peripheral journals who publish the dodgy stuff. Richard Horton, editor of the Lancet, said the same thing. The Lancet and NEJM these days likely take more ghostwritten commercial studies that the dodgy peripheral journals. What good would articles in dodgy journals be to industry?
Hypnotized
The upshot of this is doctors are hypnotized. They are now like a bunch of Xtians facing the Bible and the Sacraments and being asked to accept it’s all phony. They’d prefer to be eaten by lions. Company expenditures on hypnotizing these consumers to a greater extent than any other consumers has paid off.
The people we now face may be pleasant, but all too often are automatons. An automatism is another word for a hypnotic reflex.
As regards drugs, doctors have bought into a Magic Bullet ideology. A century ago, Paul Ehrlich, whose research helped create the modern pharmaceutical industry, saw the ideal medicine as a Magic Bullet, a drug that hits a target without causing collateral damage.
Industry now sells doctors on Magic Bullets. If there appears to be collateral damage, doctors increasingly think there was something else wrong with the patient rather than something wrong with the Magic.
Before 1980, the Magic used to lie in the doctor who, among other things s/he did, might give us a potentially risky chemical, a poison, out of which s/he would magically bring a benefit. Now the Magic lies in the pills and if things go wrong a bumbling prescriber is more likely to be blamed.
Now that Care has disappeared from healthcare, there are calls to get doctors and patients around the same table and admit the patient’s voice into the discussion. Like motherhood and apple pie, this is obviously a good thing. And some doctors will always agree to get involved in the illusion of communication.
Asked to sit around a table with our doctors, we miss that it’s their table. They will listen to our anecdotes but at the end of the day we all have to go by the evidence. Nothing else would be rational. Your anecdotes explain why you get upset but I can’t start treating you on that basis.
These tabletops are equal in size and shape—the legs give an illusion of difference.
Look more closely at these nice docs. You will see a glassy-eyedness. It’s the hypnosis. It’s not just the docs, non-medical prescribers are affected, and lots of us are also under the influence. It doesn’t pay in this situation to be an educated patient. As with the police, it’s better play dumb. For a conversation, in fact for science, to take place, the spell has to be broken.
Breaking the Hypnotic Spell
Hypnotic spells break if something happens that doesn’t fit the script. Drug-induced adverse events are key to spell-binding or spell-breaking in this case.
If you buy the idea your suicidality on an SSRI is evidence of a bipolar disorder, or your loss of libido, darling, is your depression, you and your doctor become more spell-bound. Standing your ground, though, breaks the spell you are under.
There is another way the spell might break more generally. If our meds work as well and are as safe as pharma claims, non-medical prescribers (NMPs) offer a cheaper delivery system. A switch from doctors to NMPs is happening at pace at the moment, which might wake doctors up and lead them to rediscover how dangerous medicines can be.
For many the natural answer to these problems is more regulation. But regulation has fed this hypnosis. We need to understand regulation’s risk of unintended consequences.
Regulators let companies use words like chocolate or laxative. There are 4 different laxative principles. They can introduce bulk, or fluid, into our bowel, soften our stool, or stimulate bowel contractions. Doctors used to work out which one we needed. The wrong one may put us on the road to treatment resistant constipation. Giving a company the rights to use the word laxative incentivizes their marketing department to ensure all people with constipation get their product. They don’t mind if you get other products also.
Antidepressants illustrate how badly wrong this can go. There are 4 antidepressant principles. Action on the serotonin system gives a serenic effect. Action on noradrenaline energizes. Drugs like mirtazapine improve appetite and sleep; this is a tonic effect. TCAs cure melancholia in a way other antidepressants don’t, perhaps because of anticholinergic euphoriant effect.
The average effect on a Depression Rating Scale that the FDA look at, when reviewing company Assays, obliterates these differences, making it easy to market SSRIs for all depressions.
The problem is that if an SSRI doesn’t help you, it will harm you if only by causing a dependence we don’t know how to treat. If the SSRI fails to work, your doctor likely won’t stop an antidepressant, which FDA says works. Instead, s/he will say “we thought you had a mild depression, but now we see it’s a more serious problem”. This sequence results in treatment resistant depression, a path increasingly likely to lead to medical assistance in dying (MAiD).
Regulation has facilitated a switch from “Therapeutic Principles” to “Magic Bullets”. This hands the narrative to industry, who claim to have made the Magic, which it is now your doctor’s job to prescribe and yours to take. There are no side effects to warn you about. If you get suicidal, your doctor will work out if you’re bipolar or borderline, or if your acne made you depressed and suicidal on Accutane.
Therapeutic Principles, in contrast, typically compensate for a problem rather than cure it and, in so doing, often harm. This sounds less like the Good News of Salvation, but it hands the narrative back to us. We need to think about what we do and monitor what we have done.
Regulation can work against us in other ways. To paraphrase Chuang Tzu (300 B.C), if you have a bunch of little fly-by-night drug companies, it makes sense to regulate and fine them if they cut corners. Rule books make Companies Big by forcing the little guys out of business. If a Giant Pharma then comes along, they will stuff you full of drugs with only one worry that the Red Tape Gag on your mouth isn’t secure enough to hold all of their drugs in.1
Despite all this, we turn now to regulators more than ever before. We get annoyed when they seem to have failed us. What’s going on?
In dangerous situations, we turn to a father figure. It used to be doctors. We saw regulators in a realistic light and did not think bureaucrats could keep us safe. But now that doctors are failing us, regulators have become father figures.
Pharma were quick to spot this. Since 2004, they have been..