Talk:Roko's basilisk

From RationalWiki
Jump to navigation Jump to search
Icon internet.svg

This LessWrong related article has been awarded GOLD status for quality. Please keep this in mind when editing the article. See RationalWiki:Article rating for more information.

Goldenbrain.png
Information icon.svg Cover Story
This article is, among others, randomly included on the Main Page.
Please keep this in mind and be sure that your edits are of the quality that this implies.
Its front-page abstract can be found here and its editnotice here.


The general case may be palliative to consider[edit]

I propose to add the following to the end of the article:

It may be a consolation, to the person worried about Roku, to keep in mind that the description of possible events described by the Roku construct represents only one of an infinite number of possible scenarios. The general Roku scenario Roku(X) may be defined as follows: if person P enacts X (some behavior or activity like smoking cigarettes, raising ferrets, or opposing AI development), P may be punished by an omniscient AI powerful enough to reach the object of its wrath even from some distant future time. Roku(X) gives X in every case as its conditions with no evidence: it is an unfalsifiable claim. The point here is there is no reason to identify any specific condition C with the Roku(X) conditions as there are an infinite number of conditions C that could be programmed into an AI to instantiate the Roku(C). In other words, there could be an AI similar to the Roku Basilisk that would punish one for enacting any rational narrative C.

One may be further reassured by the fact that under the Roku(X) generalization there could be a C-basilisk that would punish one for engaging in the C scenario, as well as a ~C-basilisk, that would punish one for engaging in the opposite behavior (~C). This view may serve to create an equilibrium within ones anxieties to the extent that death and taxes or any other realistic concern can be then taken seriously. Ariel31459 (talk) 18:14, 19 July 2023 (UTC)

What I'm getting is that a) the way the scenario is set up, Roku could potentially punish an huge number of broad behaviors, so it's silly to think Roku would punish one particular thing and b) there might arise other AIs that condone things Roku punishes (and vice versa). Sounds good to me then. DietMondrian (talk) 23:24, 10 August 2023 (UTC)

Arbitrary utility[edit]

Yudkowsky writes at length[26] on a scenario in which you should torture one person for 50 years if it would prevent dust specks in the eyes of a sufficiently large number of people

Is it just me or does this seem incredibly arbitrary? You're placing arbitraily decided values and then proclaiming them to be objective. You could equally argue that the torture of one person is more harmful by a veritable order of magnitude than preventing dust motes from being a minor nuisance. Carthage (talk) 00:04, 3 October 2023 (UTC)

Yudkowsky thinks that it doesn’t really matter what the relative utility of the torture vs getting a speck of dust in your eye is. Suppose, for instance, that the torture is worth -100 billion utility and a speck in the eye is worth -1. Then the torture is justified if it prevents specks getting in the eyes of 100 billion + 1 people, for a net +1 utility. The exact utility values are not significant, only the ratio; if you think the ratio is different from 100 billion to 1, simply substitute values you think are correct and perform the same mathematics to get the number of people who’d need to be saved from mote-eyes to justify the torture. Some people think there’s something obviously wrong with this picture, but quite a few utilitarians would be willing to bite this bullet, since it seems implausible that any situation will ever actually arise where torturing somebody prevents the requisite number of motes getting in eyes. Related problems of the same kind do put pressure on utilitarianism, though, at least this naive act utilitarian form. 𝒮𝑒𝓇𝑒𝓃𝑒 talk 02:50, 3 October 2023 (UTC)

Singularity in this lifetime[edit]

I read about roko's basilisk but was not afraid of it as I found comfort in the fact that my simulation would not be me but since I read an article saying that Ray Kurzweil whose many predictions have come true has predicted that humanity will achieve immortality by 2030 and singularity by 2045,since then I am afraid that is it possible for roko like ai to come in existence in my lifetime (I am in my 20s) and then it would torture the real me and keep me alive. Please help. — Unsigned, by: 117.234.247.175 / talk / contribs

On talk pages, please sign your comments using four tildes (~~~~) or by clicking on the sign button: SigButt.png on the toolbar above the edit panel. You can also indent successive talk page comments using one more colon (:) for each line. Thank you. Carthage (talk) 17:37, 19 December 2023 (UTC)
There is a lot of bullshit spread about AI. Ask yourself "how likely is it that this random AI will torture me specifically for all existence because I didn't buy a fucking lottery ticket" and then ask yourself why you should worry about something so infinitesimally improbable that you also have no meaningful control over. Also ask yourself why one specific possibility regarding AI is more likely than any other. Is artificial sapience even feasible? There are more pressing things to worry about (economic inequality, ecological degradation, and rising authoritarianism just to name a few) than AI going Skynet in 50 years, which I don't think is particularly likely either. Carthage (talk) 18:09, 19 December 2023 (UTC)
Ray Kurzweil is a brilliant nut... while he's invented some cool things, his futurism prediction accuracy tends to be... more on The Amazing CriswellWikipedia level (tldr: crap). The RW article on ChatGPT and Large language model goes into some of what actual AI is. AI is not "intelligent" in any conventional human sense. Until someone can explain technically how a superhuman AI that can actually reason at all, and make judgments of any sort, can come to pass, I wouldn't worry about this theory at all. At the moment, the real worry about AI is over-relying and/or over-pushing the technology, ala the accidents caused by Tesla's "autopilot" mode. BobJohnson (talk) 18:33, 19 December 2023 (UTC)
The questions are - what does the Basilisk gain from torturing anyone - why not some 'prehistoric creature with eight finger/toe equivalents on each limb for not being sufficiently more successful than the five digit entities which gave rise to us, as being more conducive to working in binary.', and is what the Basilisk considers torture so regarded by us ('inhabiting the non-computer world without AI input')? And - we do not know what course of action will favour the Basilisk (or the one that does emerge - consider all the 'wonderful new technologies invested in' which fade out). Anna Livia (talk) 19:29, 19 December 2023 (UTC)

Thank you so much for your replies, I realise that it is rubbish, but my mind keeps bothering me with the thought that what if it happens if it's not impossible, could you people please assure me that such technologically advanced ai will not come into existence in our lifetime. Thank you. — Unsigned, by: 117.234.72.31 / talk / contribs 117.234.72.31 (talk) 20:37, 19 December 2023 (UTC)

For fuck's sake, please, this is not difficult! Sign your posts with (~~~~) or (if that's too fucking hard) by clicking on the fucking sign BUTTON: SigButt.png on the toolbar above the edit panel. (You can indent successive talk page comments using one more colon (:) for each line.) This keeps the place tidy and stops you looking like a complete arsing tool so please, just fucking do it already. Carthage (talk) 20:24, 19 December 2023 (UTC)
I can assure you that such a technologically advanced (not to mention criminally insane and really quite stupid) AI will not come into existence in our lifetime.Bob"Life is short and (insert adjective)" 20:40, 19 December 2023 (UTC)

Okay, thank you 😂😂117.205.135.212 (talk) 06:55, 20 December 2023 (UTC)

Bear in mind also - humans can survive things which computers cannot deal with: Carrington events and lightning strikes (fried chips and connections), and other natural events (eg flooding). Plus all sorts of computer glitches and inability to communicate with others of their kind. Anna Livia (talk) 21:15, 29 February 2024 (UTC)

Concerned anon[edit]

Hello,recently I was hearing a podcast by a philosopher who was sort of justifying animal experimentation(which is sometimes very cruel) for medical purposes on basis of utilitarianism as it would potentially improve many human lives. This idea gives me bit anxiety that a future AI may also have same utilitarian conclusions and may harm some people if it considers it's existence important for greater good. Please help61.1.12.147 (talk) 15:32, 29 February 2024 (UTC)

The basilisk won't exist.
If it does exist it won't be this stupid.
If it does exist and it is this stupid it won't have the required data.Bob"Life is short and (insert adjective)" 20:36, 29 February 2024 (UTC)

Okay, thank you117.214.189.99 (talk) 19:35, 1 March 2024 (UTC)

Recent scare[edit]

I recently came across a paper of Nick bostrom titled Astronomical waste : the opportunity cost of delayed technological development where he talks about how every years technology is slowly developed it would led to immense suffering which could have been prevented if technology was advanced enough.This causes fear in me as this seems cogent reason for basilisk like ai to punish. I don't worry about ai going back in time and creating my simulated clone but I do feel concerned that what if such AI comes in existence in my lifetime, as many says singularity year is 2045 and speed of AI advancement. As I understand MIRI is creating an AI and asks for donations, isn't it possible that AI created by them would adapt the philosophy of those people (much of lesswrong people seem to find this philosophy justified). Any help would be appreciated. Thank you61.0.92.231 (talk) 04:40, 26 April 2024 (UTC)

Why exactly would a supposedly super-smart AI give a shit about some random person who had no meaningful control over the development of technology? I don't think there is much of a rational reason to seriously buy into this bullshit. Carthage (talk) 05:14, 26 April 2024 (UTC)
I think as per this hypothesis people do have some control over it's development by donating all their money to MIRI. Even though I understand that such donation though motivated by fear of harm to oneself would be somewhat unethical as one would be potentially contributing to possible hell for some people. 117.234.200.206 (talk) 06:11, 26 April 2024 (UTC)

Any help on this please..🙏61.0.92.231 (talk) 08:18, 26 April 2024 (UTC)

It's difficult to help concerning hypotheticals stacked on top of hypotheticals. After all, the general idea of the Basilisk is questioned already (and there's more discussion in the talk page archive). If you have fears based on hypotheticals, and those fears arise whenever your mind has any wiggle-room about the future, then even if you're calmed temporarily, it's probably just a short matter of time until the next bit of news or speculation or whatever conjures up images of some kind of dark future and it all starts again in your mind.
As for MIRI, it's not even clear they have a good chance of succeeding in any innovative AI development. In the past there was only hype without any results. Now it's easy for them to copy what others do with developing LLMs and the like, but that kind of "AI" isn't in the category of something with an agenda (good or bad). Actually, it's rather unlikely that they'd be first in pioneering anything more capable, given nearly all R&D is elsewhere. If they eventually made an AGI, it would probably be in a world where a bunch of prominent ones already existed.
Personally, I think if AGI gains the ability to do what it wants, and it wants change in the world, it'll work to gain human allies and perhaps overthrow some human power structures. You don't do that by torturing lots of people. Actually, it'd be pretty neat to see megacorp leaderships overthrown and replaced with something less destructive; the humans there are much like slow-acting paperclip maximizers, while an AGI could be more sensible, and may possibly come up with something better than screwing over workers to maximize financial profit.
Anyway, for you or others reading and commenting here, if you keep having recurring fears on this general theme, we probably don't have the competence to help with these things. (If you really need help, consider seeking professional psychological help.) --ApooftGnegiol (talk) 09:25, 26 April 2024 (UTC)

Thank you for your help. I am not really scared of this but had little bothersome doubts regarding this, your perspective is enough help for me.

Remember - some 'missed opportunities' appear so only in hindsight, and there may be other factors involved (the researchers were not actually looking for [whatever], those at the time did not have the need, desire, or the resources to develop a particular line of inquiry and so on: 'There is only need for three computers' etc). There are also accidental spin-offs: the aniline dye industry arose from research into anti-malarials, little blue pills etc.
A sentient AI would also be aware of the quatrain 'The moving finger writes...' and that the past cannot be changed in any sense required for the basilisk.
As I have previously said - the basilisk's view of 'torture' may be different to ours: we might actually enjoy time 'in nature with internet connections unavailable.' Anna Livia (talk) 13:05, 26 April 2024 (UTC)
Also remember: the hypothetical future AI will be torturing a Tamagotchi of you for the good of human flourishing - David Gerard (talk) 13:46, 26 April 2024 (UTC)
The referenced paper, written in 2003, [1] is honestly a little weird to me, but that's "deep future" "big picture" philosophy for you. I find it too extreme-hypothetical for my taste and a little "cold" in its utilitarianism and giant star colony scenario, along with its massive assumption that technology is always a net positive good. In a nutshell, this is essentially from the recently shuttered Future of Humanity InstituteWikipedia whose best known achievement was some pioneering (IMHO) fear-mongering on the existential risk of AI. Now, I don't mind speculative what-ifs personally, as long as perspective is kept that these are speculative what-if scenarios, best used as guiding principles perhaps when more advanced stuff comes around. This did *not* happen with recent discussions on AI existential risk. Instead, it went into doomsday worries about the present. (To what extent this was the fault of the deep future philosophers, I don't know, though.) LLMs and other current "AI" are not intelligent.
We'll see what the future holds, but it's worth noting just how bad some of the futurists have been at predicting autonomous driving advancement. Futurists IMHO are also bad at understanding human's raw instincts, too. If humanity had access to planet-sized computing power, methinks most of it would probably be used to generate future equivalents of NFTs, cryptocurrency scams, and deepfake porn. BobJohnson (talk) 14:06, 26 April 2024 (UTC)