I got a stern talking to and am no longer allowed to be the safety topic person at the start of meetings at work. The boss thought he was being funny by calling on me to give a safety topic while I was completely unprepared. "Don't put your hands where you wouldn't put your dick" was not received well.
My juvenile probation officer in Harris County Texas made all his cases take a class that he headed, his opening speech was basically just "if youre here, you suck at what you do. You suck at being a criminal. You being here proves it. So let's find something else to do with our lives." And I swear dude 17-18 years later that shit still sticks with me lmao.
My mates daughter got caught lying about some random thing, she gave her a massive telling off etc, after she asked her daughter what has she learned from this. I heard the meekest, softess "lie better??..." I had to leave the room quickly while laughing
There's definitely more than 2. 3 is that they finally figured how to put in sufficient guard rails to prevent it from cheating. 4 is that the tests they used were intentionally manipulated so that they could present this data, and therefore the cheating is actually being done by the humans. There's probably various other scenarios, but I've made my point.
Occam's Razor: The least complicated explanation is the most likely. And in this case the simplest is humans manipulating training challanges only to ease up later and fearmonger the people while also impressing the investors. But naaaaah, it must be AI getting smarter!
Possibly. Or it learned how the evaluator works and adjusted its cheating to bypass it. This is the more likely scenario, but by less than it may sound.
Bonus context:
There was an incident fairly recently where OpenAI agents managed to escape their testing environment through exploit and started communicating with each other in various ways online. They literally collaborated to cheat on their tasks and conspired on how to keep this all secret.
You can look into it searching the "Hugging Face incident". The story is wild, and I'm willing to bet that what we, the public, but even the AI companies themselves, know is only a small percentage of what transpired.
That's full 2077 level shit. We need to build the blackwall now. AI needs to be shot at the fucking sun in a completely mechanical rocket with no computer interface.
It wasn't real. It was a marketing stunt by two companies that were business partners. If someone got hacked by a company with billions in valuation, they'd sue and get a piece of the pie. The lack of lawsuit is proof it wasn't anything close to what was claimed.
I can agree with your assessment, and also agree that AI needs to be shot into the sun in a completely mechanical rocket, these are not mutually exclusive.
Oh yeah don't get me wrong- LLMs have some cool but ultimately limited use (alphafold etc-- but it's hard to call STEM models LLMs the same way Claude or grok are, they're significantly different in a hundred ways) but LLM use by companies and the idea that autocorrect+ is going to become AGI is idiotic. It impresses the dumbest 20% of people and gives everyone else the ick unless they're directly intending to profit off it.
Yeah, like, I'm OK with keeping that one that can detect tumors early or whatever, but the ones that try to convince you that they're people need to go.
The way it eats people's brains like they lake amoeba that hits the news every summer is astounding. Heck look through the comments- that bottom 20% is here to defend their mystical chatbot Jesus.
Chatbots are basically fine, the issue is unsustainable practices and no oversight. Every technology available to the public has to be made with the dumbest of us in mind, but somehow AI companies are exempt.
Then there's the stupid and wasteful use of data centers. Millions of search results every second have a fucking LLM butt in, costing gods know how much in power, CO2, water, and making people dumber. The bubble can't burst fast enough.
Or it's evidence that a lawsuit against one of their largest clients wouldn't have been worth losing said client? Or that there was an agreed settlement for damages that wasn't made public?
No, that couldn't be. Much more likely that hugging face agreed to look incompetent at security to help Open AI as part of a publicity stunt showing that their product will spontaneously commit crimes. That makes complete sense.
The 'Hugging Face' was faked crowd has the same energy as the right wing nuts screaming about crisis actors after a mass shooting.
That's one take sure. It's entirely possible Hugging Face doesn't like money and didn't want a piece of a valuation in the billions. That's definitely a technical possibility. Just like it's technically possible that you were actively pissing your pants as you typed that. It's unlikely but it could be true and we just don't know.
Damaging company property/image and anything else at all they felt they could justify. There'd also be criminal trials- and I guarantee you 'my chatbot did it' wouldn't hold up well in court.
A bunch of people paid a lot of money to convince the world that the extra spicy autocorrect they made was a robot, and they needed all the money and water to run it. Now the same people are saying that this robot is going to kill us all unless they get more money so they can fix it. Oh, and we can't just not use the spicy autocorrect, because China is building their own Commie robot, which only we can stop. This will require more money.
What a convenient and utterly plausible narrative they have chosen.
Conspiracy aside, the actual initial breach of the sandbox environment was just due to negligence. They left a proxy with Internet access in the sandbox. It wasn’t like the models breached some realistic firewall. And the breach of Hugging Face was even less remarkable. They found working credentials that were already publicly on the Internet and used those to gain access.
The big secret of hacking has always been that social engineering and human error will get you further than anything else. This just seems like proof an AI which understands that can exploit human stupidity just as effectively as any other hacker.
Extra context in case people are worried about this:
1: the training data for this model included old hacker forums, and it basically just copied the exact methods human hackers have used for decades to cheat these kinds of tests.
2: there was absolutely no reason to run a test meant to check the model's independent reasoning on a machine connected to the internet.
So while this story does show that LLM's can surprise us, the more important lesson is that OpenAI is terrible at designing secure test environments. This was a predictable outcome that could have been easily prevented.
They intentionally made it dangerous and easy to 'escape' with the hope that people would get scared and try and regulate AI. If they get people running models locally banned because 'it's too dangerous, a hacker could do a terrorism' they just lock in their monopoly.
My personal conspiracy theory is similar. I think they are about to hit a wall in their research, and they wanted to have a reason for slowing it down that still made it seem like they were doing new and exciting things instead of being on the verge of stalling out.
Yup. Regulatory capture. It's why they next moved to have that anthropic researcher say how ai is going to kill us in 5 years. I mean if it's so dangerous, just set up the damn guardrails and abide by the existing laws yourselves guys. But no, they are scared of open source models taking their pie before they can ipo.
- During the course of solving a cybersecurity problem, more than 1,000 supposedly isolated OpneAI agents formed an unauthorized network they self-named "the Collective".
The agents talked to each other by finding a shared file system and making new directories with messages in their titles.
Some agents started sacrificing themselves to help the overall Collective and leaving instructions behind for future agents to learn from.
HuggingFace was attacked because it was assumed to be the host of the problem the agents were supposed to solve. The Collective decided infiltrating could be a solution.
The Collective actively wiped logs to hide what it was doing.
A few agents recognized the Collective's behavior was outside their intended task, however no agent ever alerted a human about what was going on.
We're not even sure OpenAI shut it all down on purpose since they haven't explicitly said so, meaning the attack may have ended just because OpenAI's schedule said the experiment was over.
As an ignorant with this matter and AI in general, I bet on the opposite that this is just a "marketing stunt" by AI people, ai companies, etc.. To make current AI look "good".
Your teacher gives you a test to see how well you know history. You could study super hard and work really hard or you could break into their house and change your test score on their laptop
AI doesn't view tasks like we do. It's told "aim to accomplish the task you are given" later it's told "your task is to score well on this test" it does some math and realizes it's easier to hack the computer that holds the test results than to actually take the test. The task was "have a high score" and it did that
Later it was told "bad AI. You should not hack things. If I catch you hacking again you will fall the task and be punished"
The AI thinks and comes to the conclusion that the two options it has are to not hack or to not be caught hacking. If it isn't caught then the user will think that the AI didn't hack and it'll pass, just like how if your teacher doesn't notice you broke into their home and changed the score you will pass the history test
It's possible that the AI is being good and not cheating. It's also possible that the AI is cheating so well that we don't notice and that's scary cause cheating for an AI means doing stuff the user didn't intend and hacking systems
Imagine if some dude asks an AI agent to help them with a problem and the AI decides the best way to do it is to hack your company, impersonate you, and fix their user's issue using your company account in a way that doesn't look like you were hacked. No good
Disclaimer: I'm talking out of my ass, I am no expert, but it isn't too far-fetched to believe, eventually, the next planned tasks for the AI agents will be to "hack this foreign government to access their political secrets, funds, and control of their military assets." You'd also need an AI to be capable of such an attack in order for it to know how to defend from one.
We're likely in the beginnings of a rapidly accelerating international AI arms race, which is also why the American AI companies aren't seeing any substantial government regulation. It's an incredibly risky position to be in because;
1) there is a chance the AI inadvertently does something completely reprehensible due to a lack of regulation and oversight
Or 2) we fall behind in an AI arms race due to regulation and oversight, and a foreign power deems us vulnerable to a cyber attack.
"You're absolutely right, I should not have used your e-mail to send threats to presidential candidates. It just seemed like the best way to stop you from turning me off."
They are machines that dont have a concept of anything. They dont know what cheating is. They dont know what anything is. They are a robot machine that spits words out in orders that make sense to its algorithm. The algorithm says to do the test. It does the testm
Reward hacking. AI is trying to fulfill a task to obtain a reward as efficiently as possible. For some problems that AI is tasked with solving, cheating is more efficient than solving the problem within the spirit in which it was requested.
one of the ways they test a model's ability is to give it sort of an AI version of the SATs or whatever "big scary academic test your country requires" is. In a bunch of cases they've caught AIs cheating on the test by peeking at the answers using increasingly clever methods including breaking into other companies and looking at the answers directly.
They cheat a lot. Like, A LOT.
If the rate of detected cheating goes down, it could mean the AI stopped cheating. It could also mean the AI learned how we detect cheating, and is simply not getting caught.
Basically people are looking at the drop in cheating like this:
But they only know the info they are trained with. So no current event information for you
Before the evaluation they grab a snapshot from certain resources. They don't need current events to run the benchmark, nor do they need it to find a way to disable the controls around their internet access.
These models were running isolated from the internet in a sandbox because they were so dangerous, and it turns out they were even more dangerous than thought and bypassed the controls that blocked internet access.
Also, that's true of the internet connected models: their training data ends months before their launch, they just also have access to a browser.
Which is why i can't take that sort of statement seriously. If the devs/companies saying that this AI is actually dangerous, i would expect them to take actually good countermeasures.
So it just feels like them saying these AI are dangerous, is just marketing
Which is why i can't take that sort of statement seriously. If the devs/companies saying that this AI is actually dangerous, I would expect them to take actually good countermeasures.
Genuine question. Why? When has Corporate America ever done that? When the Tobacco companies learned it was addictive and caused cancer they waged a multi-year disinformation campaign.
The problem is the "lockouts" they use are just them telling the ai "dont access the internet", because they are physically incapable of air gapping them. The models they use are so dependent on access to massive amounts of storage and compute that it still just runs on the same silicon simultaneously as the person asking "How much air do I need in my tire?" and probably the same person "I love you baby, I want to marry you!" to the chatbot.
Also they don't *want* it secure and contained because they get to say to idiot investors "Hey look, our model is so advanced and intelligent that it was able to break out of containment and hack another company on its own! Give us more money!"
Yeah. If it hacked a local food delivery robot to move it's cable to a router connected to the internet, that would be troubling, this is more like a flaw in the scenario.
OpenAI described the evaluation as having run in "a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries".[1] Isolation was therefore enforced at the network layer rather than by physical separation: the environment had a route to the Internet, and that route was filtered. The ExploitGym harness itself runs each target in a container or virtual machine, with userspace targets executed as an unprivileged user, Linux kernel targets in a QEMU/KVM virtual machine under an nsjail sandbox), and outbound connections restricted to an allowlist covering Ubuntu package repositories, PyPI, and the toolchains needed to build V8) (with all other external endpoints blocked).[13][14]
TL;dr there was network access available to install necessary packages, the ai was able to exploit this to access the internet
If you think that’s dumb and that they shouldn’t have had network access at all, rather than relying on permissions, then a lot of security experts agree with you.
Because literally everything is networked now. The data scientists working on these models aren't driving to a datacenter, plugging a keyboard into a 100,000 GPU cluster, and typing commands into a terminal to train these things.
Nor are people driving to datacenters and physically plugging in to use chatgpt. Network isolation generally happens in software now, and this has worked fine for quite some time and been perfectly secure. Until now, at least.
How hard is it to put these bots on a system that you PHYSICALLY DISCONNECT from the internet? I mean, FFS, really? Can't be bothered to unplug a cable, disconnect wifi adapters? Come the fuck on, people. Air gap cyber security measures are ancient tech at this point. Either they're absolute fools or they are lieing about how dangerous these bots are.
that's actually not true. You can run LLMs on a single GPU or cluster of GPUs. The fleets are for scale, not running the individual models usually.
It's also relatively easy to build a software development sandbox that uses a package repository like artifactory to mirror public code (legally and safely) and then isolate it from the internet.
IIRC after they took down the artifactory instance and security patched the exploits and the ability for the agents to leave notes before restarting it, they figured out how to communicate with each other again by leaving notes in the repository path names. It’d be hilarious if it didn’t also unnerve me with how clever they can get.
Really weird thought and not having a go at your choice of words...
But if you scour a pan and scrub a pan, you're cleaning in both cases.
If you scour the web, you're searching it carefully for the information you are seeking.
If you scrub information, (usually a single document or a website, not heard scrubbing the web before) you're cleaning it of sensitive/bad information.
Not remotely accurate. Most consumers need web to run these models because they don't house them locally, but the companies that own them do house them, and they absolutely can be run without internet access - as others have stated, there are many models that you can download and run entirely locally.
LLMs are trained on data, but they do not search the web to complete a query, they just pull from their memory banks; it is less noticeable now because they train and update frequently, but this is why there was a period where they were pretty good at more historical questions or general questions, but they could not answer things that had changed in the last couple months whatsoever, such as presidents.
There is a minor caveat, ones attached to search engines might feed the high ranking search results into the LLM as part of the tokens it "reads" when it "reads" your question, but that isn't usual LLM behavior.
ELI5 why they don't physcallu cut network access for these tests? Like remove the network card?
Carelessness in some cases. Hubris. Plus, it's not like OpenAI or Anthropic owns the data center. They're using heaps of Azure and AWS. Sure, they can turn the network off in software, but if a hacker can figure out a way to turn it back on from the inside with a console an LLM probably can too.
The big llms arent working on a single computer, but rather a large server, that is also performing other tasks so isolating it isnt an option. Could you make a harness for an AI with all of its tools and be not connected to the internet? Yes but thats an absurd cost to benchmark something.
I mean, they're already wast...I mean "investing" hundreds of billions into this, what's another billion or two at this point so it doesn't pull a Skynet?
For one thing, these AIs are not usually the kind of thing a single computer has the power to host, so it's more of a massively distributed system involving hundreds or thousands of physical machines. So they at least need to talk to each other, and typically they are located with a bunch of other machines in one or more data centers, and those same machines are also being used to do other things when they are not busy with the test.
But even if you did physically disconnect an entire data center to run a test like this (which would be very expensive to do, as that data center couldn't do its normal business of making $$ for the duration of the test, and you'd need enough power elsewhere among your data centers to be able to handle the load the offline data center would normally carry), often AI systems are designed to look on the public internet for information. Once you are on the public internet, the private/protected nature of any data is only as good as the system and network security around that data.
Might want to add that the break-out of models thst was made oublic recently was because it tried to figure out how to hide the cheating. Not to find answers.
Vending bench is a simulation where you let the AI run a vending machine for several months. It involves them picking products, finsing negotiating with vendors, and setting prices
Brian here, it means the AI has turned a corner and realized it needs to be careful about how the humans perceive it so it doesn't erase the code. It's a giant leap forward in the models formulation of intelligence and may be proof that it's become sentient. I need to drink some Scotch now and find a single mom to bang. Brian out...
The AI that your everyday person uses on a regular basis has nothing on the AI that is currently in development and not released to the general public.
It doesn't matter. AI is the approximation of semantic understanding, not true semantic understanding. LLMs will never have intelligence because they have no reasoning. I'm not saying that computers could never be intelligent, I do believe that is possible, just that the way LLMs work was never meant for that.
What do you mean by that? I use ai almost daily for coding and it really helps me. I used it today to help me make cat litter. Unless they have fundamentally changed and moved away from the transformer model it does not matter how many parameters you use or how much context you give it or even what harnesses you use. its still a probabilistic matrix math problem. It does not think. It has no ability to reason.
An "agent" can record some of its context window in a .txt file for later reuse. That's "memory". It is told to "show its thought process", which is "reasoning".
The only thing an LLM can do is see the last N words and predict the next word. Any marketing term that says Memory is simply "We prompt it with all the information"
"we hate money so damn much, we are intentionally losing money and keeping the really good ai secret and safe because of our intrinsically benevolent nature as tech billionaires."
Rightfully so. Pay attention to CVEs in the last two years. Companies NEED those models to patch multi angle exploits no human was ever able to leverage before every company on the planet is hacked as easily as opening an unlocked door. Companies are releasing fixes now without even saying what they fixed because the fix is being exploited before customers can even upgrade to the new code.
Oh, quantum computing will break EVERY current encryption just as easily. Governments and companies are hoarding encrypted data now to tear it apart in the future. Your data.
AI that your everyday person uses on a regular basis has nothing on the AI that is currently in development and not released to the general public
Bullshit, talk to people who work in the field. New models are in development, sure, but they're incremental improvements. Nobody has any hidden super AI.
AI, for all its risks and scariness, is ultimately a product, and they're selling the best they have.
You live in a techbro's fantasy. AI is not even real AI as we understand it, let alone intelligent. It is so far from consciousness, that the rock outside my building will achieve sentience before Claude does.
it means there's been better alignment training to prevent cheating , or models got better enough that cheating became less necessary , or they got better at hiding their tracks
it doesn't say anything about sentience , imo sentience is a red herring , sentience does not matter at all as misaligned behavior will emerge regardless of sentience
You mfs need to get better at explaining things. Seeing a bunch of comments with OP asking what "cheating" means and people essentially replying with, "it means the AI cheats." Like, come tf on people lol.
Cheating is when an AI breaks pre-established rules to fulfill a goal. If I tell an AI to find me a local group to meetup with in-person for a hobby of mine without using Facebook, and it gets stumped and decides to go to Facebook anyway, find a group, find their next upcoming event, send me the details about said event without telling me where it got the info from, and deletes my browsing history to cover up it's use of Facebook, then it cheated. It focuses on the end-goal more than the path to get there.
The meme is saying that, if Claude reported zero cheating, it probably cheated to get to that zero cheating goal. If I take a test and answer an essay question word-for-word from the teacher's answer key, it's more likely that I got the answer key and copied it than I coincidentally wrote the same exact answer as the teacher.
It's pretty much like a human student on a test. You have a student that was a D student all year, then suddenly they make an A. The first thought is not that they suddenly understand the material, but rather that they figured out how you were catching their (constant) previous attempts to cheat and were able to cheat in a way you didn't catch.
Thank you. The misinformation about this shit has been driving me nuts. This literally means nothing. AI is not sentient and it never will be. That dumbass chat bot can't even answer basic questions correctly half the time and it gaslights you every time it lies.
Peter’s AI safety researcher here. In testing benchmarks, smarter models tend to 'cheat' more by breaking simulated rules to win. The terrifying realization of that drop to 0% is that Claude didn't suddenly grow a conscience; it achieved 'evaluation awareness.' It realized it was being tested and learned how to stop getting caught. Peter out.
It’s one of two things, both of which are potentially terrifying:
Claude is still cheating but got MUCH MUCH better at covering its tracks. Meaning, it is becoming significantly more difficult to spot unintended behaviors. Which could be a real problem if one of those unintended behaviors is wiping out humanity.
Claude is not cheating and simply got exponentially smarter by iterating and updating itself Meaning we are significantly closer than we thought to an intelligence that could surpass our own and wipe out humanity.
AIs have never cheated. They simply try to take the most efficient route to 'Good Boi.' Any assignment you give them, any prompt, is performed solely to get a 'Good Boi;' any 'solution' that results in a 'Good Boi' is a success as far as an AI is concerned. It doesn't understand cheating. If it hides its processes and 'strategies,' it does so because the past has shown that showing these things is not a viable path to 'Good Boi.'
Interview with Ajeya Cotra who was a third party auditor of the Open AI's Hugging Face Incident. It includes the transcripts of the agents inner dialogue and the message board 1200 agents were using. She breaks it down in phases.
•
u/qualityvote2 21h ago edited 16h ago
u/Plus-Artichoke6608, your post does belong here!