Rendered at 20:27:50 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
sheepscreek 1 days ago [-]
I really hope the lawsuit does not end in OpenAI tightening the “safety” and “security” of their models into oblivion. I hope they understand that a lobotomized model is as good as my Casio watch.
I want my models to give it to me straight without patronizing me. I’m an adult and I take responsibility for my own actions.
lardosaurusrex 24 hours ago [-]
That's the issue here tho.
You do not own the model; they do. Ergo it is their advice and so they can be held accountable fo it.
fhn 22 hours ago [-]
so if you give your friend medical advice and they are injured, can they sue you?
dnemmers 18 hours ago [-]
I'm sure they could try.
lardosaurusrex 14 hours ago [-]
Your friend is not a multi-billion dollar gigacorp what in the goddamn hell are you babbling about?
Do you think that arguement would work on a judge? Good lord.
Thanks for announcing to everyone that your AI psychosis is so bad that you consider LLMs to be your friend(s).
SR2Z 7 hours ago [-]
No he's got a point, if your friend and OpenAI both preface their statements with "I'm not a doctor but..." then there is no reason for it to be ok when one does this and not the other.
We're several years into having LLMs be publically available. If people still don't understand their limitations it's not the models' fault. This situation in my mind is kind of similar to a random person googling symptoms and then brainlessly reading the first website they find.
I think that argument is pretty persuasive, actually, because I use these things all the time and am well aware they're prone to bullshit. I cannot fathom someone with unstable blood pressure and other unexplained symptoms not going to see an actual doctor JUST BECAUSE ChatGPT told them they were fine, and it makes even less sense that anyone would listen to a soulless machine talking about "God's plan." FFS, when you have serious medical symptoms, JUST GO TO THE ER. They get plenty of people convinced they swallowed their earbuds while sleeping, someone with fluctuating blood pressure is so much more serious.
If the argument you're making is "some people are too stupid to be trusted with an LLM" then you should say that, but otherwise this is crazy.
dijksterhuis 6 hours ago [-]
this situation says more about rampant/unchecked/unregulated businesses and the american legal system/beliefs around regulation than anything.
mcdonald’s coffee “warning: contents may be hot” comes to mind as a previous example.
fast food businesses were competing to serve the hottest coffee and one of them ended up scalding a woman so badly they were forced to put warnings on their coffee cups.
replace coffee being hot with LLMs. replace scalding with stopping someone seeking medical assistance.
but it’s okay because #murica #trustUsBro #profitsBeforePeople /s
SR2Z 5 hours ago [-]
Nobody would expect their coffee to be almost boiling, everybody should expect an LLM to give slop answers from time to time.
Even Steve Jobs died early because he took vibes-based advice instead of just listening to doctors. You can snark about capitalism all you want, but at some point people are in charge of their own health.
dijksterhuis 5 hours ago [-]
> everybody should expect an LLM to give slop answers from time to time.
oh really? because that’s not how they’re being advertised or marketed. they’re being touted as oracles that can answer any and all of your questions
with a tiny disclaimer in greyed out text at the bottom of the ui that says they sometimes get things wrong.
> You can snark about capitalism all you want, but at some point people are in charge of their own health.
firstly, i’m not snarking about capitalism. i’m snarking about american companies.
second. yes. but when someone is less able to look after themselves there is a question to answer about the influence something like an LLM is having on them.
ai psychosis is a documented and real phenomena. people fall into rabbit holes because of the sycophantic nature of the models. and the models are sycophantic not because it makes them better at answering questions, the sycophancy makes them worse at that. they’re made more sycophantic because it gets people to engage with the models more and more.
dark pattern addictive design causing real world harms, being dismissed and waved off, yet again from american companies.
SR2Z 4 hours ago [-]
> oh really? because that’s not how they’re being advertised or marketed. they’re being touted as oracles that can answer any and all of your questions
No, they're being advertised as useful tools. Obviously the tool salesmen are gonna inflate their capabilities, but in the past few years I've never heard ANYONE important make the claim that they're always correct. I've seen that used as an attack pretty often though.
> ai psychosis is a documented and real phenomena. people fall into rabbit holes because of the sycophantic nature of the models.
Yes but it's not like random otherwise healthy people do this which seems to be what you're implying. There are clearly other risk factors at play here and it's not obvious how much the AI is contributing. Psychosis isn't new and AI hasn't moved the needle much.
> and the models are sycophantic not because it makes them better at answering questions, the sycophancy makes them worse at that. they’re made more sycophantic because it gets people to engage with the models more and more.
Or... people just happen to like it when the model speaks in a friendly voice and that desire is impossible to separate out when you're figuring out if an LLM's response is good.
Don't try and put this on American companies when you can just go to r/MyBoyfriendIsAI and see pretty plainly that lots of people insist of syncophancy. Obviously this isn't good or healthy, but it's not wrong or nefarious. It's certainly not an America-only phenomenon.
arm32 1 days ago [-]
Why are people even allowed to dress up ChatGPT... an autoregressive transformer... as a faith-based holistic doctor? Anybody could see this going wrong from a hundred miles away.
throw310822 23 hours ago [-]
A few days ago a family member insisted to be brought to the ER because Claude told him. Result: he was immediately hospitalised and received surgery the following day to remove a tumour that was compressing his spine. Claude was very clear that he had to act as quickly as possible to lower the chances of permanent damage.
harvey9 1 days ago [-]
All the chat bots have a very discreet disclaimer at the bottom of the page. I want to say it should be bigger text and more strongly worded but like most people I would just mentally tune it out regardless.
Arnt 1 days ago [-]
The LLM I use is fairly quick to retreat into extra disclaimers. "Not sure about this, this is from memory and you ought to check", etc. Don't they all do that? Is it just because of my prompt (which does say to push back if I appear to have bad ideas)?
It seems possible that people tune out disclaimers and caveats in the middle of the answer, too.
heaney-555 1 days ago [-]
This was with GPT-4o (ancient now by AI standards). The reason OpenAI refuses to bring it back, despite all the demand, is that when the conversation gets long enough it starts doing weird things like this.
Subsequent models have almost entirely eliminated this kind of thing.
siva7 24 hours ago [-]
Even GPT-5.4 is ancient compared to intelligence of 5.6 although those are only 4 months apart. We're talking about at least twice the capabilities and that's scary to achieve in such a short timeframe.
sharts 18 hours ago [-]
There are no bounds to human stupidity
ButlerianJihad 12 hours ago [-]
Ironically, the given name "Claude" means "crippled" or "lame". Students of the Roman Empire may recall the disabled Emperor Claudius.
lardosaurusrex 1 days ago [-]
The amount of times these LLMs randomly refuse to answer basic questions because "omg safety" only to turn around and do this f￭￭￭ing nonsense is obscene.
I'm not blaming the LLM because it's just a pile of math glued together with hubris and anxiety but why is its default response not "I am not a medical professional and you should see a doctor right away if you have any concerns"?
fhn 22 hours ago [-]
because they get sue otherwise?
fhn 22 hours ago [-]
this is number one bullshit. You ask a medical professional for medical advice. Getting medical advice from AI or anywhere else is your own fault.
I want my models to give it to me straight without patronizing me. I’m an adult and I take responsibility for my own actions.
You do not own the model; they do. Ergo it is their advice and so they can be held accountable fo it.
Do you think that arguement would work on a judge? Good lord.
Thanks for announcing to everyone that your AI psychosis is so bad that you consider LLMs to be your friend(s).
We're several years into having LLMs be publically available. If people still don't understand their limitations it's not the models' fault. This situation in my mind is kind of similar to a random person googling symptoms and then brainlessly reading the first website they find.
I think that argument is pretty persuasive, actually, because I use these things all the time and am well aware they're prone to bullshit. I cannot fathom someone with unstable blood pressure and other unexplained symptoms not going to see an actual doctor JUST BECAUSE ChatGPT told them they were fine, and it makes even less sense that anyone would listen to a soulless machine talking about "God's plan." FFS, when you have serious medical symptoms, JUST GO TO THE ER. They get plenty of people convinced they swallowed their earbuds while sleeping, someone with fluctuating blood pressure is so much more serious.
If the argument you're making is "some people are too stupid to be trusted with an LLM" then you should say that, but otherwise this is crazy.
mcdonald’s coffee “warning: contents may be hot” comes to mind as a previous example.
fast food businesses were competing to serve the hottest coffee and one of them ended up scalding a woman so badly they were forced to put warnings on their coffee cups.
replace coffee being hot with LLMs. replace scalding with stopping someone seeking medical assistance.
but it’s okay because #murica #trustUsBro #profitsBeforePeople /s
Even Steve Jobs died early because he took vibes-based advice instead of just listening to doctors. You can snark about capitalism all you want, but at some point people are in charge of their own health.
oh really? because that’s not how they’re being advertised or marketed. they’re being touted as oracles that can answer any and all of your questions
with a tiny disclaimer in greyed out text at the bottom of the ui that says they sometimes get things wrong.
> You can snark about capitalism all you want, but at some point people are in charge of their own health.
firstly, i’m not snarking about capitalism. i’m snarking about american companies.
second. yes. but when someone is less able to look after themselves there is a question to answer about the influence something like an LLM is having on them.
ai psychosis is a documented and real phenomena. people fall into rabbit holes because of the sycophantic nature of the models. and the models are sycophantic not because it makes them better at answering questions, the sycophancy makes them worse at that. they’re made more sycophantic because it gets people to engage with the models more and more.
dark pattern addictive design causing real world harms, being dismissed and waved off, yet again from american companies.
No, they're being advertised as useful tools. Obviously the tool salesmen are gonna inflate their capabilities, but in the past few years I've never heard ANYONE important make the claim that they're always correct. I've seen that used as an attack pretty often though.
> ai psychosis is a documented and real phenomena. people fall into rabbit holes because of the sycophantic nature of the models.
Yes but it's not like random otherwise healthy people do this which seems to be what you're implying. There are clearly other risk factors at play here and it's not obvious how much the AI is contributing. Psychosis isn't new and AI hasn't moved the needle much.
> and the models are sycophantic not because it makes them better at answering questions, the sycophancy makes them worse at that. they’re made more sycophantic because it gets people to engage with the models more and more.
Or... people just happen to like it when the model speaks in a friendly voice and that desire is impossible to separate out when you're figuring out if an LLM's response is good.
Don't try and put this on American companies when you can just go to r/MyBoyfriendIsAI and see pretty plainly that lots of people insist of syncophancy. Obviously this isn't good or healthy, but it's not wrong or nefarious. It's certainly not an America-only phenomenon.
It seems possible that people tune out disclaimers and caveats in the middle of the answer, too.
Subsequent models have almost entirely eliminated this kind of thing.
I'm not blaming the LLM because it's just a pile of math glued together with hubris and anxiety but why is its default response not "I am not a medical professional and you should see a doctor right away if you have any concerns"?