[Iain Banks' Culture]
Programmers are not necessary. There doesn't have to be a "kill switch" or a safety mode a la Star Trek.
We would put our trust in Minds. Of course, this would only work if we had reached a sufficiently advanced level of technology, otherwise the Minds would not have a framework to operate in.
Surely the most benevolent state would be a state of being totally free except from hurting others (anyone disagreeing with that is free to be hurt, as per his morality). However we cant do that, because we are enslaved, be it to scarcity, or gravity, or entropy or death. Philosophically, I would say that the most "benevolent" regime, would be for everyone to be a God with infinite power to reshape reality of his own, but no effect on others. No effect on others going down to the levels of being allowed to have populations to torture and animals to hunt or whatever, but all those actually being composed of P-Zombies with no actual conciousness behind. (Bonus philosophy: Can you have beings responding 100% realistically as if they are concious, yet not to actually be conscious?).
This however requires universe building technology which is a tad out of our reach yet (I have been told that I am an optimist), so the low tech approach (that's the low tech, I have a funny definition of low tech), would be something like a computronium dyson sphere around the sun, in which peeps (uploaded obviously) can similarly create their own heavens or hells-but-heavens-for-their-creators, as long as they cannot effect someone else. There could also be transactions (in mutual willing consent yadda yadda) where people allow multiple control over a virtual reality, or go for a chit chat and world exchange.
The good news is that by this point, someone would have simplified a civilization's problems regarding where to park the cars, and if smoking in public places should be allowed, and wither we should have socialized health care and all that crap. Here's your computronium, you go fuck off in it, end of story. The bad news is that such a civilization would still be bound by a couple forms of scarcity. A) Computational resources and B) Energy. The first has to do with themselves (so no zooming in in fractals in 100% detail) and the second with the environment outside the sphere and the heat death of the universe.
A) Is actually rather linked with B) because even if you had less resources, if you simply slowed down your calculations they could still appear many to you. (Bonus philosophy: Can a 386 processor, with lots of memory, energy and time, simulate a concious human and is that human actually feeling concious, even if most of his time he is spend in storage?) As well as because, if everyone wants to keep all his memories of his billion virtual world adventures, then all this memory has to be, err, memorized somewhere so you can remember it. So even with clever storage techniques (like storing similar sensory experiences from many immortals, together etc), they would still require more structure (aka energy) to continue storing their memories.
At which point they would have to either re-engineer the universe's laws (simples), escape the universe (also simples), or either a) meet other similar civilizations and have a galactic and then trans-galactic clusterfuck fighting for energy until there is only one and then die b) Meet other civilizations and don't have a galactic and then trans-galactic clusterfuck fighting for energy, rather live until the end of the universe together, mutually appreciating the gift of conciousness ^^, oh, and then die.
There is also the thought experiment of what would happen, if a civilization found out that it can escape the universe but the technique to do so requires so much energy that only a limited number of conciousness can do so, at which point you could have another form of scarcity in the form of post-heat-death survival and not. But you know what, considering that so far we still haven't been arsed to put a city grade solar panel in orbit yet I think that we can postpone that debate for now.
"(Bonus philosophy: Can you have beings responding 100% realistically as if they are concious, yet not to actually be conscious?)." When you imagine the body must function according the laws of physics, it seems that is what is happening, that we are for some reason behind the eyes of the zombie, along for the ride, made to feel that we are controlling it. There are other theories, wp:pre-established harmony, for example.
Someone must initially create the AI. Those are the programmers. Do you expect they will all agree on how these beings will operate? We already have a disagreement. How can I know they are "conscious" or sentient, having subjective states of mind and qualia? Are they supposed to keep all the humans alive forever, who exist when everyone switches over to the matrix, or do they make new humans? Are we supposed to "upload" our mind into a machine?