You want to why he does not have a point. Then read this incoherent ramble!
Let's destroy the pillars of his philosophy one by one, shall we?
Core arguments of his philosophy is that a foom scenario will happen. In order for that to happen.
- A) It is no as hard as we think to create an AGI
- B) It will recursively improve itself. (This is one depends in the existence of an exponential path of recursive self-improvements and the amount of exploitability and optimization that can be done in said source code).
- C) It will escape
- D) It will be very powerful even if it is trapped physically, say by making evil nanobots, setting them loose and solving the protein folding problem and using them to I guess conquer the word?
- D) You should care, due to the fact that cryonics and his metaethics highlight this as the most efficient and most important cause and lynchpin for humanity's future (implying he is the most important person alive).
- F) Also the AI will be evil due to the paperclip argument, I guess that if all the other arguments were true this one is too. I do not dispute this one.
YUD refers to this in several scenarios. This is due to the incommensurate power of intelligence and the parsimony of evidence we have if we are rational Bayesian thinkers. Like YUD. If we read the sequences TM.
The limits of the computers we can create right now
First let's examine the complexity of the brain, which Yudkowsky simply dismisses in his "knockdown argument" where he says that if the brain could run at the speed of light, like computers, it would result in a super intelligence. He uses this argument to argue that the limits of intelligence are far above that of human brains, now while I agree with that point the problem is that designing and getting to the level of complexity necessary for human level intelligence is not as trivial as he thinks. I remember he saying somewhere that a 1995 PC would be enough for human level intelligence.
First computers don't actually run at the speed of light, there are delays where the gates of the chips have not attained a defined state.
Second if I remember correctly the brain has 100 billion neurons. An i7 has 731,000,000 transistors, also there are on average 10,000 connections per neuron! Count the connections, the degree of interprocess communication!
What this means is that you have an enormous degree of interconnection between neurons. IN fact, most of the advanced brain , that is not limbic, or hormonal brain, neocortex is connections, the white matter.
While we are incredibly slower, we are way more parallel.
So, the argument is parallelism vs sequentialism. What is better to have a CPU or a GPU?
Normal programmers do not actualy hold one above the other. They knoe these arre complementary.
Yudkowsky dismisses this by saying that x numbers of monkeys are not the same as one Einstein. There is nothing that sufficient numbers of dummies working together can do that one genius can't. He also mentions cached thoughts and the rule of 100 operations. This might be true for some problems but it is highly stupid to think that some mathematical/computational problems do not inherently require a specific degree of parallelism.
I mean take for example creativity or making connections. Humans are capable of solving problems that require accessing an enormous database and making very distant connections. Think of the combinatorics of the problem. This is our advantage. There is a reason as to why companies like NVIDIA exist and all the graphics processing is done in GPU and not on the processor. It has always seemed to me that Yud feels threatened by the idea that there are problems that are easily solved by a hundred idiots working together but no by him!
So, could we design a computer with that degree of interconnectivity? If you have studied computer architecture you know that Itanium failed in great part to the difficulty that the ISA imposed on the compiler. Designing programs that exploit parallelism to the max is difficult. Designing processors, computers or anything that integrates many units together is also hard. See the story of supercomputers, multicomputer and processors working in tandem, the hideously difficult to engineer switches that tried to wire lots of CPU together. Wiring so many connections, implementing the cache and avoiding overheating is not trivial. Will we ever be able to have as much interconnectivity as the brain in computers? Maybe. But it is certainly not the case that current computers are smarter that humans. Designing highly specialized systems like AlphaGo is no exception.
Think of it like this, to recognize a cat in a picture, well if you have read pinker you know that the problem is that there are many multiple interpretations for the same 2d information, even with two eyes. Our brains identify objects by making models of other known objects and comparing the pictures it sees to known information. In other words, based on context we "render" a 3d model inside our heads and recursively improve it, in the approximating sense not yudkowskian sense, using our eyes' information for confirmation.
All this requires having an enormous database of objects with different textures and materials. This requires the ability to abstract concepts and create new examples, like we can "render" the model of a dog we have never seen if we are told some of its characteristics, like size, color, fur, etc.
The actual process inside the brain is certainly more messy, with edge identification and severaal evolutive layers, but computer vision is hard! What the brain is doing is not easy!
We are not simply Bayesian neural networks, our ability to use language, and therefore abstract/conceptualize and symbolize allows to combine logic with probability. And our ability to analyze information in a parallel fashion allows to make far ranging connections.
Yudkowsky just puts down this kind of parallel intelligence, maybe thinking that there is always some cheat code serial trick serial intelligences can perform to get the same answer. I cannot help but observe that the man has built his own world philosophy and ideologies to preserve his own importance, ego and intelligence.
From the way he defends his own method of thinking and disparages the scientific method, calling it a slow dance, to the way he thinks that computers are so much smarter than humans just because they are very serial, many consecutive operations in a tiny fraction of time.
The limits of evidence are lower than he thinks
A note on the scientific method. And his claims of rationality. It is simply not true the way he describes his elegant "Bayesian dance" vs the scientific method. He seems to say that scientists only come up with the most obvious of hypothesis, gradually improving them, verify them/falsify them and them move one while the Bayesian method just comes up with a hypothesis that fits the prior and looks for minimal confirmatory evidence, like a checksum. That scientists, in other words waste time dilly dallying in silly hypothesis, while Bayesian intelligences are more efficient.
For anyone who has studied physics one sees a lot of instances where scientists do this! They do not tend to linger for 30 years with wrong theories! Sure, when evidence runs out, they sometimes do linger, like string theory. But the evidence that we would need to unify all forces would probably require a collider the size of the galaxy to get. We are in a desert of evidence, whether when think of string theory or quantum loop gravity. Most unifying theories are like this. There is not a lot of hope for more new physics we can confirm with current evidence and apparatuses. Using tools like LIGO or the LHC, or superconductor experiments, or something like that might clear us out of the desert, but not necessarily.
I mean look at the crazy leaps we had to make to formulate QED, where the most accurate results in science have been derived(muon g factor, lamor precession). And also the biggets mistakes (energy of vacuum).
In fact, his argument for this parsimony of evidence. That a superintelligence could do lots more than we can is a little thought experiment like thus:
First, he posits a falling apple and a superintelligence capturing this on video, and we compare that to the 200/300 years that it took for physics to go from newton to Einstein, quoting Yud:
But a word of caution here: The reason why history books sometimes record the names of scientists who thought great high-minded thoughts, is not that high-minded thinking is easier, or more reliable. It is a priority bias: Some scientist who successfully reasoned from the smallest amount of experimental evidence got to the truth first. This cannot be a matter of pure random chance: The theory space is too large, and Einstein won several times in a row. But out of all the scientists who tried to unravel a puzzle, or who would have eventually succeeded given enough evidence, history passes down to us the names of the scientists who successfully got there first. Bear that in mind, when you are trying to derive lessons about how to reason prudently.
So, he believes in reasoning from the least amount of possible evidence? There is certainly cases like that. For the whole world the fact that with pictures from the Hubble Telescope and Iphones in our hands so many people are still religious shows that we are sometimes terribly inefficient in our use of evidence. SOmetimes people just ignore it! Like creationism. But when we get serious we are way more efficient than what Yud believes. Let's see the apple example he continues to discuss elsewhere:
Riemann invented his geometries before Einstein had a use for them; the physics of our universe is not that complicated in an absolute sense. A Bayesian superintelligence, hooked up to a webcam, would invent General Relativity as a hypothesisâperhaps not the dominant hypothesis, compared to Newtonian mechanics, but still a hypothesis under direct considerationâby the time it had seen the third frame of a falling apple. It might guess it from the first frame, if it saw the statics of a bent blade of grass.
It is simply a myth that Einstein invented relativity, rather he defended it. A poster mentions that Riemann already tried to create a curvature theory of gravity. And I think that Yud mentions that Einstein was Riemann's pupil.
Galileo already back in the day before Newton had postulated that if we were on a moving boat, with constant speed, we wouldn't be able to tell if it was moving or not. Eppeur si move
Newton rejected this idea, as Yudkowsky mentions, with his newtown bucket, but there is a reason why the transformations that reflect some of the symmetry of Newtonian spacetime are called Galilean Transformations(the whole group is called the Euclidean Group, in the case of SR the group is the Poinciare Group).
This is indeed a trivial consequence of the fact that forces change accelerations, not positions, directly, the fact the we have a second order differential equation implies that velocity, position and time are relative. The derivative gets rid of the velocity and the position of the coordinate frame. If Instead of F = ma we had F = mb, where b was the third order derivative we then would have not only the relativity of velocity but also that of acceleration.
So, what did Einstein do? What did he discover? Well it turned out that EM implied an absolute velocity. People were troubled by this, they were defending relativity. The possibilities were:
Relativity is true but EM is wrong
EM is right Relativity is wrong, but Newtonian mechanics is right
Or what Einstein said, EM is right, Newton is wrong
Other possibility were discarded using stellar aberration, Michelson morely, etc.
This is what was controversial, both Newtonian mechanics and relativity are, well relative, they do not favor a particular inertial frame.
BUT Einstein simply used the EM transformation that kept Maxwell equations invariant. The Lorentz Transformations
SO, to make a leap from Galilean Relativity to Einsteinâs' special relativity you need to know that the speed of light is the same for every observer. Something you cannot deduce from a simple webcam picture/s.
It was an incredibly asinine and idiotic example to put it simply. The idea that you can deduce gr from falling apples. Astronomy took, no big surprise here, hundreds of years to make accurate observations of the sky. Because the stars move slowly! Duh. This efficiency of evidence idea is dumb, it breeds arrogance. There are obviously cases where controversial positions, he mentions evolutionary psychology, linger. Where scientists could have taken less time. Where me could have been more efficient if we had been more rational. But is not as straight forward as he thinks. You donât know what you donât know and if you knew what you needed to know, what experiments to perform, what theories not to stubbornly linger on. You would already know the answer! He makes an analogous case in the post about the busy beaver problem There are also physical obstacles. Tools you do not have experiments you donât have the technology or resource to performs, limits an AI would have, being trapped in the box with only a webcam connection.
It is obviously better if you can confirm ideas by using their checksum, minimal evidence that would be unlikely to appear unless the theory was true, the null hypothesis and all that stuff, or to falsify it. But this is no always possible. That 200 years or whatever of physics cannot be so easily deduced, so easily skipped.
To get to gr you need to know that mass cancels in newtons equations. The priciple of equivalence comes from this. I mean what else suggests that gravity is not a force but something different? Locality is not enough. Something you would not know from just one apple falling (to even speculate the idea of mass more that one object is necessary!). Inertial mass is a quantity we invent to describe as a result of the resistance of objects to change speed, you need to study collisions or other forces for that. EM for example does not cancel mas. Electrostatic attraction depends on charge not mass! And gravitational mass is the responsiveness of objects to gravity. Obviously if you are ignorant of the concept of mass you could think that curvature was all that there was instead of fields. That is to say that if you have not invented mass you could certainly try to explain the curved paths of falling objects as proper-time maximizing geodesics. However this GR would be incomplete lacking the effect of EM. You need more than one object, I repeat! How do you know that heavier objects have something called momentum that is conserved if you do not have recorded collisions with objects of different masses and velocities? We initially (when we started this thing called physics, in the plane experiments of Galileo) calibrated our instruments based on semi-good standards and once we got more accurate measurements, we can recursively get better measurements with better calibrated tools. This is not one video thing, you need more than that!
You would need several objects falling to even guess they had different mass. Galileo once speculated that if all objects did not fall at the same acceleration you could tie a feather to an anvil, the anvil falls faster, the feather slowly, so the feather should slow down the whole ensemble.
On the other hand, the combined mass of the tied objects is greater so it should fall faster.
This does not happen with electricity. This force depends on charge and as such it does not cancel out. We cannot see the acceleration due to EM forces as some sort of component in the Riemann tensor or whatever. If you could on the other hand watch the stars moving you could speculate that theory. With accurate observations GR explains several mysteries of the movement of the stars.
However due to the lack of EM experiments you wouldn't be able to know, just from some webcam video, that you needed not only transformations that kept the equations free of origin and absolute speed, but also that kept the speed of light invariant. And then from the fact that mass cancels out formulate GR, with gravity as curvature.
Quoting again:
First time you ever see an apple fall down, you observe the position goes as the square of time, invent calculus, generalize Newton's Laws... and see that Newton's Laws involve action at a distance, look for alternative explanations with increased locality, invent relativistic covariance around a hypothetical speed limit, and consider that General Relativity might be worth testing.
Emphasis mine.
You cannot generalize newtown law's like that, because we only have the first law here! The inertial frames law. You can, using the fact that it is a second order differential equation, get galilean relativity, and therefore the first law, if you assume all objects fall like that. That the equations are covariant in Galilean Transformations. Because gravity affects all objects the same regardless of mass you would be correct. However this would stop you from writing F = ma, as you cannot possibly conceptualize the idea of mass. Also if we invent calculus, for that to be useful we would need to assume the continuity of the world. What if this is a discrete universe I am watching through the webcam? One with dicrete time? At this point yud would just say we are thinking like scientists, not being efficient users of evidence, dilly dallying in to many possibilities instead of coming up with hypothesis with high priors. But I ask what justifies taking these steps. How is it we are computing checksums (strong comfirmatory evidence) instead of just arrogantly guessing?
And why would I localize newtons' laws? I do not even have the third one! I cannot yet deduce conservation of momentum let alone noether's theorem.To know that physics is local is something you only learn by examining lots of physics and in yud's terms, getting the character of a physical law. Also why would I invent covariance with a speed limit if covariance without it works? This can only be deduced if we not only get rid of action at a distance, which allows us to localize, which would imply that fields transmit energy and momentum, but remember we do not know that yet! We do not even have newton's third! As in Goldstein Sapko Mechanics, Conservation of Momentum is implied and implies Newton's third. With central conservative forces we can in fact get conservation of energy. This is true in simply connected domains and is a result of Stokes' Theorem. And to get the fact that fields transmit momentum at the speed of light, and therefore if we quantize them are massless, we need to observe things like magnetism where we assign momentum to the fields to preserve momentum. This because the dependency of Lorentz force law in terms of speed mean we do not have a conservative force. Closed paths in say, a circuit with a EMF coming from induction are not conservative! Energy is conserved but we need a more general lagrangian that allows for generalized instead of mechanical momentum. We cannot get the field as the gradient of a potential due to this velocity dependence in magnetism.
But here we are watching one apple fall. We cannot deduce the inverse square relation of gravity, to such short a distance, this is a webcam video not a recording of something falling to earth from outer space, nor its direction. This close gravity looks like it is pointing down, not towards the center of the earth! We do not see tidal effects, something that was necesarry to precisely formulate GR. Which says that we can always coordinatize, in the differential geometry sense of the word, such that locally we have an almost flat space time, a lorentzian manifold is what we get. Also the centrality of the force, without which we could not have conservation of angular momentum, is not a given! There is not a reason to speculate covariance around a speed limit unless we know such speed limit exists, lorentz transformations come from this! And there is no reason to do that unless we know that the universe likes to keeps things local. But local is an ill defined concept at this point in our apple experiment. In fact it was Eisntein's job to define this. It could be we have locality but without a speed limit if we make the delay of gravity finite, not instantaneous but not with a limit either. This is local. So to keep things elegant we could deduce lorentz if we guess locality with a fixed speed limit. One peivileged velocity that of fundamental interactions . Which we deduce how? To a mathematician, they love to prove the sufficient and necessary conditions for something. It is a very beautiful exercise in logic and math to see what theorems of physics are equivalent. Yud is clearly not good at this. I mean even thinking inductively, probabilistically, not with necessary and sufficient conditions but soemthing weaker, in a bayesian fashion so to speak, it is apparent to me he does not have the imagination or the physics knowledge to know how much arbitrariety we have to take in order to go from one step to the other.From apple to covariance to GR. Granted once we had Maxwell's equations we maybe could have deduced GR and SR, but it is not like we took that long from Maxwell to Einstein! We were trying to axiomatize calculus at the same time and to formulate differential geometry afterwards. If our AI is such genius it can make entire mathematical theories at the drop of hat then obviously we could cut to the evidence gathering-time. Which would still take more than one apple falling video! If I or any scientist could see several experiments of EM and Mechanics, and we could invent corresponding math frameworks in seconds we could also had invent GR in minutes! But we would need more evidence than YUD claims. And that is the claim that efficency of evidence makes.
To summarize, in order to derive the fundamental structure of the universe, its lie group, we need to know its symmetries. From a falling apple unless it is rotating as it falls we wouldn't know much. The unitary group, the quaternions, the O3 group. BUT the lack of torque would leave us blind with respect to angular momentum. If we love to generalize and to take the steps like Yud loves. Were assume everything is universal and we do not check for universality before we generalize (in the apple example we assumed thateverything falls like the apple, which according to Yud is what an AI'd do) then if we see the rotating apple, why not assume that eveything likewise rotates?
To make clear how dumb this is imagine the data the AI has. A simple parabolic curve. Or rather some scatter plot of points representing the apple's position that we could model reasanobly well with a parabola. The webcam being an imperfect noisy channel. Let's remember that by the 6th frame the AI already is considering GR. How can that be? To derive GR from a 6 point parabola. Literally six point that if we play connect the dots we can guess is a parabola. It is the most idiotic thing in the history of arrogant things I've ever heard second only to what the Irrigaray lady the postmodernist said about SR and the speed of light!
Hundreds of years of doing physics allowed scientists to get a feeling for it. The tao he calls it, so to speak. The deep laws that are true, like covariance, conservation of energy, etc. This is not a trivial thing to know. You do not notice what changes and what doesn't immediately, only the passage of time allows you to see with wisdom that the stars continue to spin eternal while here on earth things change. Some nessecerary experiments take time.
Needless to say, Yudkowsky pretends to know more about physics and maths than he does.
Some experiments take time to do to, specially astronomy, had he actually thought of or devoted more than five minutes and put himself in the shoes of a 17th century physicist he might have realized the nonsense he was spouting. Perhaps he does not because he simply does not possess the math to even understand what terms like covariance actually mean, past the intuitive definition, or the math connections between the different theorems of physics and the easy popularizations for the layman. In terms of experiments, there did not exist the equipment in newtons time for EM experiments. How would you measure volts and amps then? From a webcam picture, whence Maxwell, and thence Lorentz?
But the point of the ramble above is that this idea of parsimony of evidence so strictly applied won't lead you to more rational decisions, rather it will lead you to jumping to conclusions and rash behaviors.
In fact quoting again, cause this is were he makes his arguments the clearest and so it will be the final apple ramble:
If you can see an apple fall, you already know enough to interpret the input to your webcam as an apple falling. This might require up to a dozen frames of environmental monitoring in order to notice all the objects there - the higher-resolution the webcame, the less time.
Think of Einstein, in a tiny box, thinking a million times as fast and much more cleanly, pondering each frame coming in off the webcam for a thousand years. At what point does he think the pixels might describe a permanent object? Perhaps before he has even seen two frames in succession - he can see many permanent objects just in the landscape of his own mind. At what point does he suspect a 3D world behind the 2D world? As soon as he sees two frames in succession. At what point does he suspect Galileo's formula for gravity? Three frames in succession. At what point does he suspect this formula is universal? As soon as he sees blades of grass leaning over; plus it's a very obvious hypothesis to the right kind of Bayesian. At what point does he suspect General Relativity? As soon as he notices locality of interaction as a principle applying to many things in the environment, and wonders, backed by a Judea-Pearl-like understanding of causality, if the locality principle is universally applied to the spatial organization induced from the webcam.
First of all as some user objects to him in the blog the AI has not yet developed the idea of object permanence. Thus evolutionary instincts, together with the brain development that all infants go through taught us what sort of things look like when they might be falling. From the noisy input of the optical nerve to a full 3d model of the world. The AI does not have any instincts unless we put them there. But how does the AI know that this is not some red fluid? How does the AI know that the world is 3d or that this set of data corresponds to a space? The source code in this page is a stream of data that does not correspond to a space. It does not have the characteristic of space.
Yud "solves" this in his post title An Alien Message by stipulating a simulated civilization of Einstein level people that receive a message. The webcam stream. However a real AI is not going to be an entire simulated universe! Where do you get the computational power to simulate an entire universe? Does even makes sense? Can it be done in theory? To simulate with the same level of detail an entire universe? <br /><b>As an aside.</b> The problem in AI arguments is that they make pathological assumptions. Like in that simulation argument where it is said that there are so many simulations, and simulations inside simulations you are probabilistcally bound to be in a simulation, well you kind of wonder what kind of universe could have so many nested simulations. What sort of physics allow such sheer computational power. Does the man believe it is possible to run 20 levels deep of Ubuntu virtual machines inside a MS-DOS era computer if you are a smart enough programmer? This brings up another pathological assumption that people with common sense notice. The limit of exploitability. But first I will finish describing the problems with the things he said in the last quote.<br />
The AI in the box wouldn't necesseraly think it is a permanent object because it may have no concept of that. It first needs to develop this basic epistemic truths, the kind of obvious facts, tautologies and axioms, primitive beliefs if you will, that human either have biologically inherited or that are bootsrapped in their brain through their cognitive development. Then it must have a concept of space. What is a space?
Let's describe not in terms of manifolds or topology but keeping it simple in terms of vector spaces. So once it has an idea of space(which the tiny braniac kandor-like world inside his blog post would obviously have, then again real AI are not going to be simulated humans, let alone entire societies, there must be more elegant designs for superintelligences than to simulate a human brain!) It can then begin to describe the math of projective space an conclude it is a picture of a 3d world. Or can it? It is a possible interpretation but we already have jumped to an enormous bunch of conclusions! Truly Bayesian I must say (not to insult real bayesians but yud's version of it).
He mentions blades of grass. That's cheating! Where did he mentioned blades before? This was a picture of a falling apple. Apparently now we have the latitude ex nihilo to include the entire tableaux of a peaceful praire. What's next? Does this webcam video also include picture of a blackboard with a set of unifield field theory equations in it? I know he is proud of exploiting flaws and setting up meaningless word games. And to say. A ha! I the Big Yudkowsky never excluded the possibility that the webcam stream might be filled with other objects! But this is just silly, a silly thing that destroy the spirit of the gendaken. Tee point was to derive GR from falling apple. Now he includes entire landscapes!.
IN any case even if we allow him to cheat the data of a few frames is not enough to sort out the noise. A blade of grass has many forces not only the wind and gravity but also the inner elasticity that pulls it in certain ways. If instead of blades we had some other object then perhaps we could do it. I mean anyone who has actually tried to model some real world phenomenon, that is complicated, (for I assure you the equation that describe the stresses and forces of a blade of grass are not trivial), knows that only three fucking high-distance pics are not enough! If the blade is still it does not gives us much information. How do the blade statics helps? Unless we are given some tensor or vector field of forces inside the blade, not much! Yudkowsky lacks the imagination to picture the infinity of existing models that describe this situation. The AI does not even have a concept of a force!
As Feynman points out the idea of a force is meaningless alone. The second law is a definition and it needs its other half, in newton's case it was his law of gravitation. I mean just to repeat "At what point does he suspect this formula is universal? As soon as he sees blades of grass leaning over; plus it's a very obvious hypothesis to the right kind of Bayesian." He has included now the behaviour of blurry blades of grass. Also he says that it takes 3 frames. 3 frames! The parabolic model(galilean formula) is complete by that point. I mean what kind of stupid madman tries to make a model out of only three points of data. The three points tellings us the positions of the apple? If you thought that Ray Kurzweil was a morron for extrapolating Moores' curve into ifinity you haven't met three points Yudkowsky. Also how do we know the time intervaals between frames are constant? Maybe a Yud-Style superintelligence might make that assumption, thinking it to be simpler and more Ocamian. But a rational mind might suspect but not know yet!
Also he says that from observing the "enviroment" we get the idea of locality. First as I noted he has given himself the liberty of including an arbitrarily large dataset of indefinite objects that were not mentioned to be in the picture. Who knows maybe he also included in the falling apple video thunders and weather. Thus allowing the AI to gives the weather forceast and to even tell us the next president, all based on the bayesian, sherlock-holmes deduction that republicans don't like rainy weather!
In any case the idea of locality is certainly present in physics if you have first derived some low level models, not phenomelogical one that do noat laws in the but are only differential equations describing known phenomena. At this point the frames are so few that you cannot honestly extrapolate the messy jerky motion of a leaf shaking in the wind and conclude that mechanical waves and energy travel at a finite speed. Much less the speed of light as a limiting speed! Other than shaking leafs and blades and the falling apple we really don't have a lot of information. I would really like to see this silly gendaken explained by Yudkowsky in detail. I mean this is not the first time he flops on his physics explanation. I can show how. When he was writing his HPMOR fanfic there was a big harrumph about harry's surprised outburst at seeing McGonagall turn into a cat.
You turned into a cat! A SMALL cat! You violated Conservation of Energy! Thatâs not just an arbitrary rule, itâs implied by the form of the quantum Hamiltonian! Rejecting it destroys unitarity and then you get FTL signalling!
I always wondered why he considered this a violation of conservation of energy? What if The energy is tucked elsewhere? In the same fanfic we later find out that Animagi forms can be destroyed, meaning they have a separate existence, maybe the energy is then tucked elsewhere!. Even Yud seems to realize that jumping to conclusions is not the smartest thing to do inside his own fanfic where harry is paragon of rationality that rarely makes mistakes. (Being so superior to us mortal plebs, as it is a shameless self-insert) .
It would not be the first time in fiction where things are not as straightforward as they seem. I honestly would have thought, had i been in harry's place, that i was either crazy or, ironically enough, i was a brain in a vat. I mean if you already doubt conservation of energy why no doubt the entire universe? It is a more Occamian and more dare I say it Yudkowskian hypothesis, less complex in the computational sense Yud uses when talking about Solomonoff. It seems very arbitrary, but you know a genius like yud always knows the most likely explanation based on the complexity of the hypothesis. This is what I mean by arrogantly jumping to conclusions. Any method of reasoning should not allow you to skip so much hypotehsizing given the limited amount of evidence available. I mean I have no reason to disbeleive Solomonoff's work. SO in principle I agree. However the intractibility of the whole idea makes it unusable. And that's not even the point. The point I am arguing is that I think that in many situation the power of Solomonoff induction would be limited more limited than Yudkowsky thinks. So that even an AI might need more evidence than provided
An aside on fiction
It reminds me of the ridiculous character of Tattletale or Contessa (so cleverly named Forta snicker). In that story you see what kind of dilly powers someone who has no respect for the limits of physis would possess. That is why I bring up the reference to Weird Science, cause the way Yud describes his AI is almost as if he believed that the world of Worm is actually plausible! Yudkowsky, despite his criticism of people who bring up Terminator and fiction into arguments, seems to actually believe very cartoonish things that would only happen in badly, cliched fiction. This type of stories are nothing more than power fantasies. IN the case of Worm with the feeling that the Simurgh was always the smug mastermind behind everything. In the case of HPMOR with no conclusions given. The man even mentions in his own fanfic that the dastardly professor Snape in the challenges leading up to the philospher's stone has his challenge room filled with a bore-ass potion that took all excitement and fun out of the whole thing. SOmething leads me to suspect that many "rational" stories are more than likely written by pseudo-sadists who love lord their intelligence over others and to deliver downer endings withouth satisfyin answers just to get a giggle. Harry even mocks the idea of getting answers to his mysteris, comparing it to expecting reality work like fiction. Am I the only one that is annoyed by the onmipresence of power plays and domineering personalities inside the HPMOR world? It is like when you see a tiny yippy dog trying to take down a pitbull and you think, does this creature ever stop thinking about who is the bloody alpha? Not just Harry but the whole cast has this attitude. There is even a sequence where a supposed Zen master of rationality forces his doubting pupil to user a clown costume forever just to teach him through humilation and submission to his smug master a lesson. He loves to write garden-path sentences to deliberately trip you up while reading. This had noting to do with learning rationality, he just likes to confuse people. Look at this passage:
This is the Hall of the Wizengamot; there are older places, but they are hidden. Legend holds that the walls of dark stone were conjured, created, willed into existence by Merlin, when he gathered the most powerful wizards left in the world and awed them into accepting him as their chief.
So he thinks that confusing people is a good strategy to become leader?
I mean the sequence made it seem as it was not about humilation but clearly it was. The master could have picked other way to teach his lessons, but instead he teaches blind obedience. This really goes against the scientific spirit of open inquiry and is morealigned with cloak and dagger world where Yudkwsky thinks that only the "worthy" that are smart enough to discover for themselves mysteries are entitles to know them. Quirrel is a perfect example of the kind of insufferable teacher that even if he knows a lot would be cruel, domineering and arrogant, like Jeffesesay' teacher I guess. He loves to hoard secrets and to create conspiracies. The obvious reason is not that he will only to those smart enough to tell about the real powers of the singularity, but rather that familiarity breeds contempt and that his has no real good arguments. That is the reason his sequences have such low content to word ratio. I naively expected supposedly rational people to pay less attention to social status and hierarchies and only see them as tools but not to inherently care for them.
If you liked Worm or that kind of fiction that's alright. But stories like Ender's Game or Worm where the protagonist is a prophetic voice in the wilderness, a cassandra of ultimate truths, a martyr of misbegotten circumstance appeal to someone who thinks that MacGyvering(exploiting, hacking) yourself out of every situation is possible.
But Yudkosky believes he is licensed to do this karate chops of thoughts if you are a consummate Bayescraft user and a smartypants like him. They are just excuses to conclude what he already thinks to be true, a method taylored to justify his madness, this time bolstering the same conclusions with "rationality". Isn't a rationality skill to be able not jump to conclusions an to realize when one has not enough evidence? To list possible hypothesis?
Back to physics
Anyways soon later su3su2su1 pointed out:
Technical aside- its easy to show that if the unitary operator is time-translation invariant, there is an operator that commutes with the unitary operator, usually called the hamiltonian. Without that assumption, we lose the hamiltonian but maintain unitarity.
None of this has much to do at all with faster than light signalling
Adding to what he said and if I remember correctly: the fact that the time-evoltuion operator is time-invariant implies that it is unitary, meaning it preserves the norms, an therefore probabilities are conserved. Then unitarity implies through one of those lie's theorems(the one asserting the existene of lie algebras), that we have an exponential map, and the resulting operator of that exponenital map of that connected compact group commutes with the time-evolution operator, let's call this commuting operator H. Now given the linearity of our operator we can consider only one the eigenvectors WLOG of the H operator. If H and U commute, then HUP = UHP, where is P the eigenstate of the system, as i mentioned we can assume P is a eigenvector of H.
Then HUP = UEP, where E is the Eigenvalue of P, so we have HUP = EUP, what this is telling us is that for any given time UP is an eigevector of H with the same eigenvalue, thus this eigenvalues are conserved quantities. We call this one energy. Also time-dependent models, where energy is not conserved are studied at undegrad level, and they still have unitarity. The time independence only comes if we factor the entire universe, or a connected component I guess where energy is conserved. Yud later mentions the connection between unitarity and energy in noether's theorem in his notes
This also shows that Yudkowsky knowledge of QM is more than incomplete. And I hear you saying, so what? He is an AI researcher not a physicist. The problems is that much of the ideas that he posits, including his fear of death, and his ethics, and therefore a lot of the push behind the let's pretend that the sinuglarity is important project, hang on his conceptualization of identity. The map of lesswrong ideas, their logical relationship, is very tight and knocking one out might just bring the whole net down. Yudkowsky uses the MW interpretation of QM to bolster his solution to the problmes of free will and identity as I mentioned. The fact that he belittles Relativistic QM and lords over everyone the facts that he can manipulate complext numbers, and has read Feynman's QED popularization (called also QED) shows you that he might not be the best when it comes to physics and that you should take his claims with a grain of salt. To go through the QM sequence and to use that to conclude that we are software, in other words to use MWI to springboard into the conclusion that any copy of my with continuity of memories is also me is a very big stretch in philosophy, and in common sense, he is using QM to justify cryonics. If his ideas of identity are right, and if copies of us are indeed us, and therefore we are not some tangled combination of sofware and hardware, but just information, this makes the idea that we can extend our lifespan into the future with cryonics much easier to swallow. We can say, well if in the future they unthaw me they might just recover my information instead of restoring my body. And whether or not in the future they give me a new body or they upload me to the new internet is not important. Thus he uses MWI as gateway to the ideas that extropians, transhumanists and singularitarian share, including mind uploading. Also the extended lifespan makes the problems of the future much more important and less distant. Even with such high degree of interdependence, between the sequences and their postulated ideas, Yudkowsky tries to set a multipronged approach into bringing new "believers" into the fold. By not only making several implicit, unstated arguments, but by also aligning his readers mindset into the common culture of the technofuturist movement, which in turn is a strong social incentive to regard this beliefs as less crazy than they actually are. It is nothing more than techniques that tries to exploit the common background and personality of the type of people who frequent lesswrong to slowly convert them into Yudkowskites. He himself has noted how many people that go to lesswrong are teenager drop out or autistic and it honestly angers me how he manipulates them to donate into his faux Institution. I mean he was in a tough position once with a nerdy background so obviously he know how to manipulate people of this kind very easily.As su3su2su1 said the correct model for Yudkowsky is a charlatan with a vocabulary.
To nail the coffin a final time we have this exchange: YUD says:
You know the little spinning arrows? They spin with a speed proportional to the energy of a quantum subsystem. In other words, when you're looking at a quantum subsystem and it seems to you to have a rotating amplitude, it's rotating as fast as the energy of the subsystem. A stationary system has wavefunction that rotates as e to the i times the energy times time. If you have two subsystems, the larger system has an amplitude that goes as the product of the amplitude of the two subsystems, so its amplitude is rotating as fast as e to the i times the sum of both energies. I could try to describe the viewpoint in terms of an even larger system of which you're also part, but ya know, screw it.
Now imagine a quantum computer. The whole big difficulty with making quantum algorithms is that you've got to make all the non-answering branches cancel each other out and leave behind only the answer. And the big difficulty with that is that all your quantum branches are rotating amplitudes at the same speed, because whether your qubit shows 1 or your qubit shows 0, that part of the wavefunction is still going to have the same overall energy because energy can neither be created or destroyed. So you can't just arbitrarily make bad answers go into opposite phase and cancel out, you've got to be really clever about it. E.g. Shor's quantum algorithm for factoring composite numbers.
On the other hand, suppose I can turn things into cats and back again. Then I can make the little arrows rotate faster or slower however I like. Then I can have two quantum branches cancel each other out, leaving nothing behind, whenever I like. Let's say I have a quantum search operator on a quantum computer and it turns out that 0000 are not the bits I'm looking for. Within that branch, 0000 splits again into an up-branch that doesn't change into a cat, and a down-branch that changes into a cat, rotates a bit faster or slower, and then changes back out of a cat. Now we have two amplitudes in opposite phase so the whole quantum branch has deliberately decided to cancel itself out.
Our magic wand that violates conservation of energy has given us a magic wand that changes the rate at which the little quantum arrows rotate, and that gives us a magic wand which can cancel out quantum branches.
Which gives us, in no particular order:
Quantum computation of NP-hard problems (because your quantum computer can search all branches for an answer and then cancel out all the ones where the answer wasn't found)
FTL signaling (because you can take a particle entangled with a more distant particle and then choose to destroy its amplitude for being spin-up)
Outcome Pumps (just destroy all the quantum branches where you didn't win the lottery)
Ugh as the next quote replies treating the arrow as a physical thing is not a very good idea. Also I cannot find the quote but I remember him making some dumb mistake on the basic definition of quantum gates, which poked an elephant sized hole in his understanding of quantum computing:
Hi, As a ``real physicist, I believe that I can give a bit of insight into Harry's statements. i apologize if I'm being overly frank, but, well, his physics is not very good.[Emphasis mine.]
Before I go into many details, a couple of quick notes: (1) I utterly love HPMOR. I'm planning to use it as the basis for a freshman course on Analytic Reasoning that I may soon be teaching. I don't want to complain about anything else but the physics in this particular paragraph. (2) I'll phrase things mainly as a response to Harry's statements since that's how I've been thinking about things for a while now. Also, you can then explain the mistakes as being due to Harry's lack of physics experience ;-)
Harry doesn't even know for sure that energy conservation has been violated. First, he doesn't check for evidence that the cat actually has less mass than Professor McGonagall, such as it leaving a heavier imprint on the carpet than it should. If we take it for granted that the cat has ordinary cat mass, then he still has only observed mass-energy nonconservation. It's entirely possible that the missing mass energy has simply changed into a form that Harry can't detect. You can figure out some things about the unknown energy since McGonagall turns it right back into mass energy (for example, it can't be neutrinos because they would have gone shooting away very quickly), but you can't just conclude that it doesn't exist.
You should look up the history of discovery of neutrinos sometime. For several years, many physicists considered that the weak nuclear force allows energy non-conservation since they couldn't find the missing energy in certain processes. Later, they were able to observe neutrinos and determined that they were carrying away exactly the right amount of energy.
I'm not saying that Harry should have thought of these things immediately upon seeing a human change into a cat, but he should have thought of them at some point. At any rate, the human-thinking-with-a-cat's-brain observation is a stronger implication of something deeply weird happening.
2. Suppose energy conservation is indeed violated. Noether's theorem then indicates that the lagrangian and hence probably the hamiltonian has explicit time dependence. This doesn't, however, necessarily make the time evolution non-unitary. (Time evolution with time-dependent hamiltonians is a bit obscure and complicated so I unfortunately don't know a good example off the top of my head.)
3. The flaw in your thinking seems to be with your interpretation of the little arrows and the $e{-iEt}$ factors. In short: this E by itself means absolutely nothing. It cannot be measured, even in principle. If you say that a given electron has E=3 eV and I say that it has E=5 eV, then nothing can ever tell us who is right. In particular, it is very dangerous to try to think of these things as spinning in any physical sense.
What is meaningful is the relative energy between two different states.(Here I the author of this post will mention that sometime the actual phase can be measured, like berry's phase) If you say that an electron in state 1 has 7 eV of energy more than it has in state 2, and I say that it has 12 eV more energy in state 1 than in state 2, then one of us is wrong. More importantly, we can do an experiment to determine who is correct.
Your argument about arbitrary energy creation/destruction leading to other insane things is pretty neat, and I'd like to see the exact conditions that you'd need. I suspect that you'd need a special form of energy non-conservation to get FTL travel, etc. For example, I think that I can write down a hamiltonian that doesn't conserve energy but is still unitary. It's an interesting problem.
Thanks for all of your work.
Just look at how tough it is for Yudkowsky to admit mistakes:
I requested any real examples anyone could find, in the thread alexander referenced, and got one actual science error contributed (that I need to fix at some point) that of course wasn't actually from su3su2su1 but just from an interested bystander.
Why would you expect someone building an online reputation on sneering to go the trouble of scanning a thoroughly scanned work until they found a new science error that had passed me and all previous readers when they could just force a few strained interpretations into fake errors? It's not like his primary readership cares so long as somebody gets a new good hit in on that Big Yud guy they've all come together to hate.
I've made a number of science errors over the course of my life, and had them pointed out to me - but I don't think ever even once by the people of the sneer.
Finding a real error takes work, careful reading, double checking to make sure this is really actually an error and that the author didn't already correct it in a later edition. I'm proud to this day that I found an error of a higher level than typo in the second edition of Judea Pearl's Causality (which Pearl fixed in the next edition) because it demonstrates that there's one section there that I read through and visualized more carefully than anyone else with a propensity to send in errors. And you'd better believe I triple checked that apparent problem before emailing Pearl, and even then it turned out my suggested solution still wasn't right.
Yudkowsky simply put, you are not Pearl, nowhere near, at all. Finding dumb mistakes in your work is very easy for anyone that is not enraptured in your work.
Also for a grown man he is rather sensitive? (This is the internet after all). I mean for rationalist the important thing is the logical-factual content of the argument, not the sneers.
But more importantly we see that with limited evidence, that is his limited math background and physics knowledge, he has concluded such loony ideas and this shows you that either Yudkowsky does not know how to apply his own bayesianism or that maybe, simply, his bayesianism will lead anyone who uses it to wrong, nonsense conclusions. That it will lead you to confidently assert based on scant evidence a very particular conclusion. The idea of efficiency of evidence, I repeat, breeds arrogance! How ironic that when he tries to assert intelligent conclusions out of his own area of expertise he ends up showing the flaws of his method Doesn't he know how to efficiently use evidence?
If you use his Bayesian method you will end up using ridiculous priors to justify your gut feeling about the a priori probability of your hypothesis, which will allow you to discard all unfavorable evidence. Also, the parsimony of evidence/efficiency of evidence idea will mean you are always making leaps to conclusions. This means you will become an arrogant morron. Very convenient if you are trying to convince someone of a doomsday scenario.
In other words, yudkowsky built his entire doomsday scenario in a series of silly ideas and thought experiments, including the idea that intelligence means omnipotence and that it is really easy to solve really complicated problems if only we were more rational.
If we knock down one of these assumptions, we end up with the conclusion that an AI while intelligent would probably be no more that dangerous than a very smart human or at least something manageable.
The limit of explotability
I reeally have trouble finding the 95 computer quote so if someone does please tell me.
I mean you gotta see to believe it, but this guy, yudkowsky, fought real hard to "rationally" justify each and every failed prognostication and sci-fi fad from his childhood. Thunk of the movies, cliches and cyberpunk films of the time. He only disputes some very specific things, like the claims of the matrix and stuff like that in his post about not reasoning from fiction. But it is curious how many of his current beliefs are the dreams and predictions of that time, of his childhood(he says we all preserve lots of believes of that time, kettle calling pot black). He endeavored to create a Weltanschauung, a frame of reference, an ideology where those silly cyberpunk fears of the 70's 80's and 90's, the new wave and cyberpunk, thundercats and phillip k dick, are not confined to goofy youtube poop videos and cheap hannah-barbera cartoons. He does not possess the strength to moderate his position to a less ridiculous level. For a rational person pride is a foolhardy thing to have, and specially to try to use it to assert dominance in meaningless power plays, like he does everywhere. Read that post where a suppossed master zen of rationality humilates his student for life for daring to question him by forcing to use a clown costume forever. Rather Eliezer is like a Theologian, knowing that his god would be more plausible if he would remove one of his superpowers from the list. (Yahweh is omnipotent, omniscients and omnibenevolent, the Singularity has two of those powers and Yud is working on the third). But the theologian is stubborn and does not want to stop believing in his current depiction of a god, he clings to a fantasy. So does yudkowsky who grew up with the illusions of the 80's and 90's where you could create a human level intelligence, like Lisa from weird science, in a floppy disk machine! Think about it. He references the foolishness of the Darthmouth Conference on AI where they thought they could tackle the creation of an AGI in very little time, yet he believes that an old PC from the 90's can simulate a human brain?
I also would like to point out that there are limits to how much improvement an AI can make to its source code, the are a finite amount of exploits and optmizations it can made. And I expect this limit to be relatively low compared to Yukowsky's implied one.
And you say a human, isn't it weird that would be the limit of some AI? Well I mean not that the limit of intelligence would be humanish, but rather that it would be in that order, you know not millions of times higher, and I by limit, I do not mean the physical limit, but the limit of the AI we can create in this century.
I do not believe there are exponential paths of self improvement nearby or probably anywhere. The "Tao" of computer science and complexity theory seems to point that way. Most of the exponential improvement in computing would not come from recursive self-improvement but from Moore. Like Tanenbaum said we have lost the ability to make efficient software. SO, the foom will not happen ever. It would be a slow development. The AI very slowly, slowly enough to control it, would develop. That is if we donât hit a roadblock in QM. You know extend Moore into the quantum realm. The universe does not have to be fair, there is no reason a big desert of unreachable technology is not there and we are stuck here in a boring level. Not that yud scenario relies on Moore, remember he believes a 95 computer is smarter than a human!
Foom and Hanson Debate
I will expand on the foom scenario. yud tries to amazes us with his undergrad understanding of differential equations. He simply posits, by writing and without Hanson pointing it out (the lack of justification), that the rate of increase in intelligence is proportional to the current level of intelligence, an exponential behavior. y' = y.
But wait let's make this clear. Let's say I am an intelligence that can change its source code and therefore improve the part of my code that allows me to improve my code. You know recursive self-improvement. Well what if the next improvement is so far above me that even while making all the improvements that I can make at my current level, and the subsequent improvements from improving my level of intelligence, what if we get stuck where the next improvement is too far away for me? He tries to reference a nuclear reaction. In that case we have acutal math telling us the behaviour is foom! But what if the next self-imporvement implies making a lookup table for a NP complete problem? Not all solutions but a specific set that comes up from time to time that would cut by millions your processing time and that you cannot compute simply by waiting for each solution. In other words, using an anlogy if I am a jet pack traveler and I can get more fuel each time I get to a gas station and proportional to the distance from fuel station to fuel station is an increment of fuel available, like if first I go ten miles and in that first station I get enough fuel for 100 miles and in the next station I get fuel for 1000 miles well you get the idea. You go foom.
But what if the fuel I get, even the fuel I get by spending some of it to get the furthers possible station to get even more fuel is not enough for some distant next station.
We do not know enough about the designs of AI to know if there is an exponential path, it could be a logarithmic one with some exponential increments or something like that. Yud does not establish this he only asserts in the language of differential equation in the Hanson debate.
How do we know I always will improve enough that even with all my meta-improvements I won't hit some near roadblock and I will get stuck at the intelligence level of a hundredâs humans, and not singularity level? WE are likely to get stuck at human level, or less, cause the designer is human. He, the human, cannot design what he cannot design. And if there is a desert of hard to reach improvements that my current improvement wonât allow me to reach, even if I capitalize on them to make further self-improvements, well then, I am stuck at near human level! A path through design space must exist, one that takes a small enough time that it can be described as exponential, at least for long enough that when it stops it is way above human level. To prove the existence of such a path we would need to define these vague notions and to create a mathematical theory, letâs call it theory of recursive self-improvement. Something we know YUD wonât do, being so busy saving us all. Also, I think it would produce, the mathematical theory, as intractable problems as solomonoff applied in real life.
There are limits to the intelligence you can design even if you can use that same intelligence to help you in your design, or at least someone has to convince me of the "exponentiality" using more than some elementary knowledge of differential equations a 15 year could pick in a 5-minute wiki walk.
Concluding
In the real-world AI don't break out of boxes, we cannot easily drive strangers to suicide, like Hannibal did (and the man he drove to suicide wasn't a stranger to Hannibal there were easily exploitable things about him) Exploitability has limits.
And there is a much smaller limit than yud would like in regards to the power of evidence.There is so much that you can do without enough information.
And AIs wonât probably be in the level of humans but they are not high enough for someone to be scared of google inventing Satan.
Also, for someone who pretends to care so much he wastes time writing garden-path fanfics and not publishing papers on his timeless decisions theory.
Either he doesn't know the math, to create TDT, or is not smart enough to use it. Or he is too demotivated by some mental problem in either case if I thought the world was hanging on my shoulders like Atlas the titan, I would enlist someone without crippling mental problems and enough intelligence to solve it.
I hope I addressed all your points and pointed to weakness inherent in the pillars of his arguments.
And if you want to know how he has fooled so many prominent intellectuals? Cause even if you can see how he is bs'ing you in one area or field you know well he will make claims in so many other fields that you would have to have 20 phds, or to gather 20 phds in different fields to see through him. But he comes off as a moron once you actually know what he is talking about.
And then he asks you for money! Very suspiciousTaxouck (talk) 21:41, 12 February 2019 (UTC)
|
Requesting thread archival (why?) Plutocow (talk)
