Colluding AI

As more and more grunt work (like, so much that’s done in the intenisve people farms called ‘offices’) is replaced with AI, how soon will we find that some decision by a human, hardly in control anymore but totally reliant on the precooked algorithmic outcome provided by AI, will be contested in court – that will be presided over by a judge, hardly in control anymore but totally reliant on the precooked algorithmic outcome provided by AI, and the two colliding against humans’ interests…

Note that “of course”, there will be humans nominally handing out the final verdict(s), because so many (not yet) fought so hard (not enough yet) to keep a ‘human in the loop’. But having achieved not much more than the nominal thing, and there quickly being far too little humans with enough experience (how would they gain that, when they haven’t gone through the grunt work themselves, including being allowed to err sometimes or how would they otherwise have learned ..?) to be able to usefully overrule the AI. Usefully, in the sense that the AI will have all the better, rational even if outlandish arguments… No more gut feelings … That may be part of what makes us human; whaddabout Kahnemann’s 90% System 1 ..?

And then, still, what when AI finds it rational to re-introduce the death penalty ..? Swiftly executed, to preempt appeals?

Oh how bright is our future! Also:
[There was supposed to be a shut-down button somewhere in one AI/pillar at least… Now they switch each other On again …; Córdoba]