|It's not totally wrong, it's|
“”Before Bruce Schneier goes to sleep, he scans his computer for uploaded copies of Eliezer Yudkowsky.
Eliezer Yudkowsky (b. 1979) is an AI researcher, blogger, and exponent of human rationality (specifically, his Bayes-based version of it). Yudkowsky cofounded, and works at, the Singularity Institute for Artificial Intelligence (the name was later changed to Singularity Institute, and in its current incarnation it's called the Machine Intelligence Research Institute), a nonprofit organization that concerns itself with the concept known as the singularity.
Yudkowsky also founded the blog community LessWrong as a sister site and offshoot of Overcoming Bias, where he began his blogging career with GMU economist Robin Hanson. Being a very smart young man who wants everyone to live forever, he also has an overweening interest in cryonics, and is (probably) vulnerable to hemlock. A self-described autodidact, he has no training in the subjects he writes about.
 AI "research"
“”Find whatever you’re best at; if that thing that you’re best at is inventing new math[s] of artificial intelligence, then come work for the Singularity Institute. If the thing that you’re best at is investment banking, then work for Wall Street and transfer as much money as your mind and will permit to the Singularity institute where [it] will be used by other people.
|—Eliezer Yudkowsky on how to save the human race.|
Yudkowsky believes he has identified a "big problem in AI research" which no one else has noticed. The problem is there is no reason to assume an AI would give a damn about humans or what we care about in any way at all, given that it won't have a million years as a savannah ape or a billion years of evolution in its makeup to build up any morality or aversion to killing us. He also believes AI is imminent. As such, working out how to create a Friendly AI (FAI)—one that won't kill us, inadvertently or otherwise—is the Big Problem he has taken as his own to solve. Yudkowsky is entirely unpublished outside of his own foundation and blogs and never finished high school, much less done any actual AI research. No samples of his coding have been made public, probably to prevent someone from releasing his monster AI.
The problem scenario is known as "AI foom," which is short for "recursively self-improving Artificial Intelligence engendered singularity." It is based on the ideas that:
- Despite glacial progress in AI so far, the AI-based Singularity will occur soon. Trust us!
- AIs would want to improve themselves (and we want them to).
- Human-level or better AIs could not be imprisoned or threatened with having their plugs pulled, because they would be able to talk their way out of the situation. Interestingly, Yudkowsky claims to have tested this idea by role-playing the part of supersmart AI, with a lesser mortal playing the jailer. However — the reader may note the beginning of a pattern here — the transcripts have never been published.
- AIs will also be able to gain virtually unlimited control over the physical world through nanotechnology.
As is evident from the dialogue between Luke Muehlhauser and Pei Wang, Yudkowsky's (and thus LessWrong's) conception and understanding of Artificial General Intelligence differs starkly from the mainstream scientific understanding of it. Yudkowsky believes an AGI will be based on some form of decision theory (although all his examples of decision theories are computationally intractable) and/or implement some form of Bayesian logic (same problem again).
Since these are both totally abstract mathematical functions, there is no reason an AGI based on them would share human values (not that you can base an AGI on them). By this reasoning, Yudkowsky sees the need to bolt on some sort of morality module which will somehow be immune from alteration, even though the "foom" scenario he fears so much is based on AGIs which can rewrite their own code. Thus Yudkowsky sees the need to "solve" ethics in some form that can be computerized — although ethics remains an open problem in philosophy after thousands of years. However, Plato, Aristotle and Kant just weren't as smart as Yudkowsky believes himself to be.
Anyways, actual AI research isn't based on implementing unimplementable functions; it is based on trying to reproduce the only known example of natural general intelligence, human intelligence, in all its messy glory; and it is based on learning. The real-world approach gives two separate routes to Friendliness for free. One is that by reproducing a human mind, and not just an abstract mathematical function, you may reproduce human-style moral intuitions. The other is that if an AGI learns from human mentors (like HAL in 2001) it will pick up human ethics from them.
In addition, some of Yudkowsky's critics identify more urgent threats than unfriendliness: deliberate weaponisation of AIs, coding errors, humans themselves, cyborgisation, etc.
 Fan fiction
“”Oh god. The future of the human species is in the hands of a guy who writes crossover fan-fiction.
For a slightly more humorous but still fairly insightful look into his mind, you can read his Harry Potter fan fiction, where a rationalist Harry (a total author avatar) tries to get his head around magic. This is expressly intended as propaganda for rationalism, but it's a cracking good read for fanfic and is very highly rated on FanFiction.net. For what that's worth.
LessWrong may have developed a little eeny weeny tiny bit of a personality cult around Yudkowsky, with a list of Eliezer Yudkowsky Facts, but the fan fiction of him was apparently a play, produced without his knowledge.
 Genius or crank?
Whether Yudkowsky considers himself a genius is
unclear totally clear; he refers to himself as a genius six times in his "autobiography". However he admits to possibly being less smart than John Conway. As a homeschooled individual with no college degree, Yudkowsky may not be in an ideal position to estimate his own smartness. That many of his followers think he is a genius is an understatement. Similarly, some of his followers are derisive of mainstream science, just look for comments about "not smart outside the lab" and "for a celebrity scientist." Yudkowsky believes that a doctorate in AI is a net negative when it comes to Seed AI, though he doesn't appear to believe there is a campaign against him. While Yudkowsky doesn't attack Einstein, he is indeed opposed to the scientific method in favor of his interpretation of Bayes' theorem. That a philosopher of science found this ridiculous doesn't affect his views the slightest (he is the pinnacle of a rational being, remember?). LessWrong does indeed have its unique jargon.
Disagreement with Yudkowsky's ideas are often attributed to "undiscriminating skepticism." If you don't believe cryonics works, it's because you have watched Penn and Teller. It's just not a possibility that you don't believe it works because it has failed tests and is made improbable by the facts.
 More controversial positions
Despite being viewed as the smartest two-legged being to ever walk this planet on LessWrong, Yudkowsky (and by consequence much of the LessWrong community) gets certain things very wrong or endorses positions as TruthTM that are actually controversial in their respective fields. Below is a partial list:
- Transhumanism is correct. Cryonics works. The Singularity is near![citation NOT needed]
- Bayes' theorem and the scientific method are in contradiction to each other.
- Bayesian probability can be applied indiscriminately.
- Many Worlds Interpretation (MWI) of quantum physics is correct (a "slam dunk"), despite the fact that there is no consensus among actual quantum physicists.
- Evolutionary psychology is well-established science.
- Utilitarianism is a correct theory of morality. In particular, he claims, if there were a choice between an extremely huge number of people experiencing a speck of dust in their eyes for a moment, and a man being tortured for 50 years, the torture would be preferable.
Also, while it is not very clear what his actual position is on this, he floated the idea of a future society where rape was legal in a science fiction story. This quite predictably caused a reaction in the comments section. He responded "The fact that it's taken over the comments is not as good as I hoped, but neither was the reaction as bad as I feared." He described the science fiction world he had in mind as a "Weirdtopia" rather than a dystopia.
 See also
- Yudkowsky's autobiography, written in 2000. Hilarious!
- Making HAL Your Pal by Declan McCullagh, Wired, April 19, 2001.
- ↑ 1.0 1.1 Eliezer Yudkowsky Facts
- ↑ Yudkowsky and friends at the Institute
- ↑ 
- ↑ Yudkowsky's opus, Creating Friendly AI
- ↑ http://lesswrong.com/lw/bxr/muehlhauserwang_dialogue/
- ↑ 
- ↑  Yvain, LessWrong, 23 September 2009 10:08:17PM
- ↑ Harry Potter and the Methods of Rationality
- ↑ http://www.sl4.org/archive/0707/16399.html
- ↑ The Level above Mine
- ↑ http://lesswrong.com/lw/4g/eliezer_yudkowsky_facts/1qhd?c=1
- ↑ http://lesswrong.com/lw/d6/the_end_of_sequences/aad?c=1
- ↑ http://lesswrong.com/lw/8f4/neil_degrasse_tyson_on_cryonics/
- ↑ "I am tempted to say that a doctorate in AI would be negatively useful, but I am not one to hold someone's reckless youth against them - just because you acquired a doctorate in AI doesn't mean you should be permanently disqualified."
- ↑ http://lesswrong.com/lw/qa/the_dilemma_science_or_bayes/
- ↑ http://rationallyspeaking.blogspot.se/2010/09/eliezer-yudkowsky-on-bayes-and-science.html
- ↑ http://wiki.lesswrong.com/wiki/Jargon
- ↑ http://lesswrong.com/lw/1ww/undiscriminating_skepticism/
- ↑ http://www.quackwatch.com/04ConsumerEducation/QA/cryonics.html
- ↑ Eliezer Yudkowsky on Bayes and Science, Massimo Pigliucci, Rationally Speaking
- ↑ While it is a powerful tool, Bayesian probability has its limitations like any other. See Stanford Encyclopedia of Philosophy's entry for Potential Problems with Bayesian Epistemology
- ↑ " I can think of only three slam-dunks off the top of my head: Atheism: Yes. Many-worlds: Yes. P-zombies: No. "
- ↑ Less Wrong: Torture vs Dust Specks
- ↑ 
- ↑