MetaMed



I decided to ignore Michael after our first in-person conversation, where he said I shouldn't praise the Swiss healthcare system which I have lots of experience with, because MetaMed is the only working healthcare system in the world (and a roomful of rationalists nodded along to that, suggesting that I bet money against him or something). MetaMed was a medical consultancy started by Michael Vassar (who called himself Michael Arc for a while), a LessWronger and former President of the Singularity Institute, on the basis that superior rationality skills as taught by Eliezer Yudkowsky would make them obviously better at medical research than mere medical school and applicable scientific experience could. MetaMed was founded in 2012, started operations in 2013 and went broke by 2015, because it turned out rich hypochondriacs weren't dumb enough.

MetaMed was an instructive example on the value of experience and knowledge versus assuming one's own general brilliance and reasoning from first principles &mdash; and why people who think they're too smart to bother acquiring actual domain knowledge are potentially dangerous idiots. They seriously believed that LessWrong training was all you needed to out-doctor doctors reliably enough to base a business on it.

Medically speaking, MetaMed's rapid failure was probably the least worst outcome, and it didn't have time to rack up a body count.

The key thought everyone involved seemed to have was: "everything I don't know the tedious details of is simple because I am smart and radical in my thinking and know the One Weird Trick™!" Perhaps you are and do! But it's overwhelmingly likely you aren't and don't.

Origins
Vassar had been the President of Yudkowsky's Singularity Institute (now MIRI) and left to try his superior Yudkowskian rationality against the real world. He was very strong on the general applicability of skills gained from arguing on an amateur philosophy forum:

Yudkowsky helped market it to LessWrongers. He was a big fan, as they had helped him resolve his sleep disorder.

Vassar got together with Jaan Tallinn (a programmer, who had previously made it big with Skype and Kazaa, and started reading LessWrong in 2009 when sidelined by a lawsuit between Skype and eBay ) and Zvi Mowshowitz (previously a professional Magic: The Gathering player, who is also worried about Roko's basilisk ). They also scored $500,000 in venture funding from Peter Thiel.

Vassar explained to Harper's how MetaMed would leverage this to change the world, and serve as incontrovertible evidence of the power of LessWrong rationality:

The plan
Vassar explained the plan to Bloomberg:

The actual working model was: you paid Metamed $5000 (for the starting package), then someone who'd just heard of the disorders involved for the first time in their lives, but was experienced in arguing on LessWrong, would enter your symptoms into various scientific journal databases and give you their rational advice in return, e.g.:

Their doctors would review the reports first, at least.

MetaMed's target market was rich hypochondriacs — the "worried well", the perennial favourite targets of quacks and alternative medicine scammers — who they could convince of their ability to beat the system:

For Vassar, MetaMed was all about fighting the power:

They also aimed to aid doctors with advanced artificial intelligence and data from information experts. This, despite Vassar's complete lack of experience in any real-world AI development — only in talking about the risk of AI dooming humanity.

Scott Alexander worked for them early on (he's a doctor by day) and was very excited about the possibilities the MetaMod model offered.

At least one of the doctors involved, Hank Garstein, turned out to have been struck off for writing online pharmacy prescriptions. They did fire him quickly once others had pointed it out.

The hype
Despite touting their superior Bayesian statistics and research skills, their main promotional page "Vital Facts and Statistics", a collection of scary claims about the failings of the medical field in general, had a number of problems. One of them was a lack of hyperlinks, despite it being a web page and many of the cited references being online.

Unsurprisingly, it's full of vagueness, weasel wording and references that simply don't back the claims. Even if you thought MetaMed fundamentally had the right approach, the claims on this page showed worrying deficiencies in understanding basic statistical reasoning and researching and assessing scientific sources. Just a few examples:

CT scanning and radiation
The claim:

... but if you examine their own linked reference, what it actually says is:

That is, it is not a warning that a CT scan exposes you to atomic bomb radiation levels — but is a passing note that some atomic bomb survivors got doses as low as a CT scan (this is almost a certainty since radiation exposure decreases with approximately distance-squared from the source).

Giving someone excessive CT scans can in fact cause cancer, and this is a risk actual CT operators know all about and keep careful track of, where the risk from the X-rays is balanced against the risk of not having the test done; but the comparison to Hiroshima and Nagasaki is purely for radiophobic scaremongering.

One should note that lifespan studies — performed regularly since the 1950s onwards on the survivors of the aforementioned atomic bombings — actually fail to show an increase in cancer rates, even at radiation levels greatly surpassing that of the low-exposure survivors which MetaMed invoke for their flawed comparison to CT scans.

98,000 deaths per year from medical errors
The claim:

This is cited to the 2000 book To Err Is Human: Building a Safer Health System by Linda T. Kohn, Janet M. Corrigan, and Molla S. Donaldson, which actually says:

That is: from two single studies, we see 0.2% errors leading to deaths in Colorado and Utah, 0.8% in New York, which could have been halved with more attention. That's potentially important, but it's not clear how medical amateurs doing their own research could be applicable to this problem. Not to mention hyping the larger number.

It is also important to note two responses on this "44,000-98,000" number:


 * "How Many Deaths Are Due to Medical Error? Getting the Number Right" (2000) shows that the numbers are derived via a long chain of extrapolation, with very wide error bars that aren't noted:


 * The 2001 American College of Clinical Pharmacology response to the report, particularly the bogus nature of the 44,000-98,000 claim.

Surely MetaMed's research skills had found these as well, and surely people who literally bragged about their "statistical literacy" would have spotted these problems in the material they were citing.

"Deaths caused by medicine itself total 225,000 per year in the US"
The claim:

This claim is not substantiated by the references:
 * The first cite is not to a scientific finding, but to an article in a law journal, which is actually a lecture transcript, the text of which at no point backs a claim of "225,000."
 * The second cite is to a JAMA article on "Defensive medicine among high-risk specialist physicians in a volatile malpractice environment", but does not at any point contain the claimed numbers.

Memento mori
By about April 2015, MetaMed had gone to its richly deserved fate, because its original idea (LessWrong rationalists are automatically smarter than domain experts!) was stupid, and its pivoted idea (concierge medicine to the rich worried well!) didn't have much of a market.

Tallinn attributed MetaMed's failure to the difficulty of producing research results — they were actually surprised when medical research turned out to be harder than they'd thought — and the limited interest from consumers. Of course, he was still all about fixing medicine with One Weird Trick™ — the horrifyingly dysfunctional US health care system apparently being the only example he could think of in the world.

Zvi Mowshowitz blogged about why they went down. His take was that everyone else was too irrational and signaling-obsessed to understand that they ought to pay large sums of money for Metamed's advice:

This raises the question of how they understood marketing, where the job is literally to promote the sizzle in order to shift the sausages. MetaMed VP of research Sarah Constantin details that they were really quite bad at this, with a list of mistakes that are actually not unusual for failed startups; she considers they could have had a viable business had they made less retrospectively-obvious errors.

Of course, they spent early 2015 stiffing people.

After MetaMed
Michael Vassar started another company, BayesCraft. That's "Bayes" in the special sense of "talks transhumanist bafflegab but doesn't seem to understand actual statistics."

BayesCraft is:
 * 1) Another medical startup (though what it actually does, the internet does not reveal);
 * 2) that has "Bayes" in the name.

From this, we can presume that Vassar really does want to make a difference in medicine in particular, and considers his LessWrong-trained rationalist skills ideally suited to the task, without all that tedious mucking about with training as a doctor, scientist, or both.

It is quite possible he will attain fame for this &mdash; though the Bayesian likelihood from prior probabilities is that it will be the sort of fame that results in FDA warning letters, and writeups in Respectful Insolence, Science-Based Medicine, Edzard Ernst's blog, Quackwatch and RationalWiki.