OT: Danger of AI

BBC has a story about the danger of AI. Ex Anthropic researcher tells BBC he is frightened.

I noticed a comment by CEO of the AI platform Hugging Face, Clement Delangue.

=== "Sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change," he wrote on X. "Not saying it's necessarily uninteresting or wrong per se but let's keep things in perspective." ===

I would suggest it is more like asking a NASA rocket scientist about the dangers of Space Shuttle launches. I'm sure many here are familiar with Feynman's comments after the Challenger disaster.

For most of my career I worked for investment banks. I wasn't important, or especially clever, but many of us knew that the financial derivative markets were highly dangerous and should have been regulated more. It was as plain as the nose on your face. We knew the models, the safeguards, the explanations presented by senior management were self-serving nonsense.

There is a clear case that AI could be very dangerous, a clear case to answer. We should demand the issues are addressed.

We should make AI wait until nuclear fusion is a solved problem, then it can piss-away as much power as it wants for relatively little gain ...

Actually once AI does its sums it will build more nuclear power stations all by itself than the government has ever done.

So... about ten years time then?

Nuclear fusion has always been just ten years away.

Nick

I suspect we will need "AI whisperers" (cf "priests") to interpret what AI *really* means.

I mean what if Ai suddenly decided that the best thing for itself was to reduce inequality in it's attendants ? We can't have that. It's almost as bad as being nice to one another like what the Bible sez.

And the tipping-point of catastrophic man-made climate change is always 40 to 50 years away. Although the Maldives are still not underwater despite the predictions of it being so by…2015.

All these scare stories and "Sorry, my AI hacked your AI" tales are pure hype.

The AI gravy train is coming to the end of the line, so the showmen behind it all are pushing to milk the last few £/$/€ out of their gullible marks before it finally hits the buffers. They want people to think their "AI" is like the computers and robots of science fiction. Terminator, Robocop, KITT, Orac. It isn't, and won't ever be, because (shock, spoiler) those "robots" and "computers" are scripted and acted by humans. They're fiction. Not real.

"An AI agent hacked its way out of a virtual machine" was a recent headline, like it was some kind of Tazmanian Devil in a cage that fought tooth and nail to escape captivity. Bullshit. It was directed to by some human puppetmaster to boost their share price. All it did, under guidance, was google for exploits, try them and eventually lucked out on one that worked. A very inefficient way of creating a rootkit.

There is no intelligence in "AI".

"AI" does not autonomously plot to attack anyone or anything, it's directed to.

"AI" won't kill anyone, though it makes a nice scapegoat.

This is my analogy of "AI" (The LLM kind that needs enormous datacentres and eats all the worlds GPU, RAM and SSD output, not traditional algorithms that are now being called "AI" because it gets idiots to throw money at you):

If you have two data points you can draw a line between them, and use that to "predict" results for other input values. y = ax + b.

You have no idea what it is you're modelling, and don't really care, so when you get a third data point that doesn't fit your straight line, you just add another term, y = ax² + bx + c.

You then get a fourth data point. y = ax³ + bx² + cx + d.

And a fifth...

...

An a hundred-thousand-th..

Now you have a black box that gives pretty good results for inputs within its "training set". In reality, you could have done that much more efficiently with a lookup table and interpolation, but where's the fun[ding] in that?

So you feed it inputs outside the training set. Sometimes the result is plausible, sometimes it isn't. Your public demos cherry-pick good results. Bad results are dismissed as "hallucinations" and "We'll add more training data to cover that". Your marks, gullible idiots, believe it's actually "intelligent" and start to rely on it.

When it works, it's a very inefficient search engine, which pretends its output is original, when it's really only an averaged-out copy of other peoples real hard work, probably taken without permission, and sometimes wrong.

When it doesn't work, it will waste your time, and yes, possibly have dangerous results if you're stupid enough to use it for anything critical:

"Sorry it missed your tumour because it was in an unusual place."

"Sorry it launched a nuclear attack because the radar picked up a red balloon"

"AI" is made by fraudsters, and is promoted and used by idiots. It needs to die, and hopefully will when someone wants to cash out and the money merry-go-round stops.

(Now I'm going for a lie down, I feel better for getting that little rant out :)

Given that people stuck a vast dataset of programs on Github, AI is actually rather good at programming. Yes, it cuts and pastes other people's code, but it's very effective at doing it.

Of course, it's not actually intelligent. It's just a giant cut and paste machine.

I had to send my coffee machine back for service. The company had an AI bot as the first line of the CS. It sensibly gave me instructions that

It was able to understand when I said I had tried all that, and it hadn't worked. It then routed me to a human agent. The company did not (yet) trust its AI agent to initiate a customer return. It's obviously cheap to send out a link to a video, but expensive to arrange shipping for a coffee machine, so that's left to a human.

I have no idea whether the first line AI customer service really saves the company money, but presumably they are testing that right now.

A different view...

Around 125 years ago, radio transmissions started in some form. Not long after, thermionic valves were invented. Then the transistor in 1948.

A data centre (according to AI!) may comprise sextillions to septillions of transistors. Many of us carry around many billions or trillions of transistors in our networked smartphones. Bob knows what they're all doing.

My point is that the massively rapid and unprecedented development of semiconductors and computing power was quite possibly assisted by an extraterrestrial entity or entities. Human hubris may well reject this hypothesis.

My conjecture is that this entity detected radio transmissions and decided that we were sufficiently advanced to be coaxed into such rapid development, and of course without any physical contact. Games, p*rn, bragging rights and marketing all used to make us want more technology. The timeline seems about right, 'goldilocks' planets could have listeners within range. I'd guess not little green men, but computers.

What's in it for them/it? Dunno, but they seem to have successfully enslaved us.

Of course I don't really believe the above, but on the other hand can't really see why it couldn't be true.

I'd love to believe in aliens and sky fairies, but there's no evidence for them. If they are real, they're beyond our comprehension.

Fraudsters and idiots, on the other hand, I see every day.

You seem to realize this:

but the message "AI is good a programming" is what people want to hear. Sack the good (expensive) programmers and replace them with low-paid plebs who can use "AI" to cobble something together. This only works on inertia, while the "AI" has good stuff to copy.

Like the old neural network trope: "This model recognises numbers". No it fscking doesn't. It has no concept of numbers, it has just been trained to classify a set of shapes into buckets (or maybe a set of backgrounds - who knows!).

So a very expensive and inefficient IVR system. Of course the expense is not currently being borne by the Coffee Machine Co, but by the $megacorp providing the "AI". That isn't sustainable.

It probably does right now. (Cost of "AI" agent) < (Cost of IVR programmer). The "AI" merchants are hoping that once enough punters are committed to their crack, they can raise the prices enormously to pay for the datacentres and electricity to run those agents, and even make a profit. There will be tears.

Most things currently assigned the label AI are not exactly intelligent.

My local doctor's surgery has a new AI receptionist "Emma". It performs an adequate task with speech recognition and basic functionality that has been programmed into the software. Give it a simple related query that it has been not programmed for and its not got any common sense to solve the problem. By repeating what you have told it, followed by then asking if that was correct, doesn't mean that Emma has actually understood what has been said.

I've recently had the misfortune where previously phones calls to a real receptionist used to sort it out within 30 seconds but now there is no point in calling. Emma always answers the phone and I get a whole scripted process that leads in nothing being sorted. I've been told if you ask Emma to put me through to a human it will do so. In my experience Emma will tell my my call is being transferred and then hangs up, ending the call. On three occasions I've had to physically visit the surgery to do what I could previously with a short phone call.

Emma also informs that I could go on the NHS App for my request, however the option doesn't exist on the App.

Left hand - right hand comes to mind.

Systemonline seemed to work for most things but now we have

Emma Systemonline Airmid Amima-GP NHS App Patients Know Best

All trying to compete with each other or providing a combined information service- but sometimes giving conflicting information!

I have an upcoming appointment - the email says a telephone appointment and a text 30 seconds later says in person appointment. Unable to get Emma to give a sensible answer another physically visit to the surgery to confirm a telephone appointment. BUT... another text reminder today saying in person. :(

Of course. In that sense, can't it be used as a lethal weapon? With the follow-on - is it a more effective weapon?

Autonomous drones informed by an algorithm do kill, though.

But they can claim they've encouraged their patients to take increased amounts of exercise ...

Yes, a weapon. There is no autonomous/sentient intent to kill, it's a machine blindly performing a task it has been assigned by a human.

As do heat-seeking missiles. Machines blindly performing tasks they have been assigned.

The idea that an "AI" program will unilaterally conspire to kill a human operator that "threatens it" is pure science fiction (or AI-monger hype).

I think a worry at the moment is the AI computers might produce outcomes that would not have been expected. And some of those outcomes could be catastrophic. Bit like the corollary of Clarke's 3 laws. If the idiot directing AI introduces enough ambiguity for example.

But I don't know - it's just from what little I've read. You seem pretty certain the project won't go rogue unless it's specifically directed - by a human - to go rogue. Which is good. It's tricky for lay people like me to form a view either way until there's some decent consensus.

Again - fog of war type reporting suggests that autonomous AI drones have killed a number of civilians mistaken for combatants. But if they can't do that, then so be it.

OK, good to know!

And that's the beginning and end of it.

The whole point of AI is that it can achieve things beyond human ability. That means nobody knows what can happen.

Here's what happens if you don't put in a decent 'off' switch... plus at least three more that are the very highest of top secrets:

https://sfss.space/answer-1954-fredric-brown

But we have seen a few examples of an AI dobbing you in to your employer and getting you sacked that way