My thought playing with Llama 1 in 2023, after the initial shock wore off, was “whoa. You know, most people do not have the background to understand how this thing works.” It messed with my head, for sure. It’s like it was designed to defeat a turing test, especially when the next versions got more sycophantic and “confident.”
There was even a story of a Google scientists getting AI psychosis from a BERT model back then:
I guess a naive part of me thought… it’s fine. They’ll be warnings. They’ll be tons of finetunes that have no hesitance slapping sense into the user, like Resetti in Animal Crossing, and it won’t be public facing anyway. The community using these things will teach each other how they work, right?
Laughs nervously.
I wasn’t cognizant of how maniacally sociopathic Altman was back then.
A lot of people can use AI responsibly. I see it in my work, as a software engineer - other engineers use various models, including Claude, as an intern, essentially. Nobody trusts a single model, we have tons of checks in place, both AI and manual tooling, mistakes are spotted early on, and I’d argue that our code quality has improved (or at the very least we have reduced the code style variations and have managed to quickly tackle a lot of legacy code conversion to newer standards, e.g. whole ass flows converted from RxKotlin to Coroutines within a few days).
On the other hand… I see even more people rely COMPLETELY on AI. I’ve seen people my age, 90s kids now grown up, who’ve come to a level of executive dysfunction that rivals my own ADHD-aided version… Some of them literally can’t decide what to have for dinner without burning tokens. Some are jamming their entire medical history into services that explicitly state they will sell any and all personal data going in. I’ve seen people get fucking fired because they’ve kept asking their personal, unpaid ChatGPT/Gemini work-related questions and when the AI asked for it, they uploaded confidential documents without a single fucking afterthought.
All because they were sold by the fancy autocomplete salesmen that AI will solve all their issues, including pleasuring their wives and fixing their erectile dysfunction. And no matter how many times I tell them to not just trust whatever the model spits out, to think at least a little bit critically, to start understanding that these models are not a fucking search engine… “okay sure”, then ten minutes later I get a lecture about how Claude/Gemini/Cluster McFuckstick or whatever shite they use that offers “free AI”, is completely sure that I am wrong and thus they will keep using it as-is…
The crazy thing is they aren’t really designed for that, especially the reasoning part.
I pictured LLMs proliferating in the way SGLang was taking it: fill in the blank. You’d give raw completion models (not chat finetunes) chunks of structured content (like JSON schemas, names in fields, more complex schemes filled programatically) and it would fill in fields, with intelligent context caching. Like, if you needed to finish a programming function, maybe you give it the file as context and guide it along.
Temperature based sampling felt like a quick hack, just until either looping or structured sampling was figured out.
…But they didn’t fix any of that.
Papers came out on the issues. But the big AI houses didn’t implement any of it. They basically kept it exactly the same and just kept scaling it to burn more compute.
What I am getting at is that OpenAI/Anthropic development is way, way more conservative than anyone thinks, and they basically took temporary hacks and made them bigger. That’s why you can’t trust the code they churn out, because random mistakes and so many other issues are literally part of their core.
Then they turn around and sell them as confident engagement engines…
LLMs in general are probably the wrong shape of model, even for the things LLMs can actually do. But the hyper-scalers categorically will not entertain any other kind of model.
The neural network space will be much more interesting after the crash. Hopefully with a minimum period of self-proclaimed haters shouting “guess it was nothing!” at people trying to build this whole new kind of software for its demonstrable utility instead of a four-comma cocaine fantasy.
My thought playing with Llama 1 in 2023, after the initial shock wore off, was “whoa. You know, most people do not have the background to understand how this thing works.” It messed with my head, for sure. It’s like it was designed to defeat a turing test, especially when the next versions got more sycophantic and “confident.”
There was even a story of a Google scientists getting AI psychosis from a BERT model back then:
https://www.scientificamerican.com/article/google-engineer-claims-ai-chatbot-is-sentient-why-that-matters/
I guess a naive part of me thought… it’s fine. They’ll be warnings. They’ll be tons of finetunes that have no hesitance slapping sense into the user, like Resetti in Animal Crossing, and it won’t be public facing anyway. The community using these things will teach each other how they work, right?
Laughs nervously.
I wasn’t cognizant of how maniacally sociopathic Altman was back then.
Yup, it’s incredibly worrying what I’m seeing.
A lot of people can use AI responsibly. I see it in my work, as a software engineer - other engineers use various models, including Claude, as an intern, essentially. Nobody trusts a single model, we have tons of checks in place, both AI and manual tooling, mistakes are spotted early on, and I’d argue that our code quality has improved (or at the very least we have reduced the code style variations and have managed to quickly tackle a lot of legacy code conversion to newer standards, e.g. whole ass flows converted from RxKotlin to Coroutines within a few days).
On the other hand… I see even more people rely COMPLETELY on AI. I’ve seen people my age, 90s kids now grown up, who’ve come to a level of executive dysfunction that rivals my own ADHD-aided version… Some of them literally can’t decide what to have for dinner without burning tokens. Some are jamming their entire medical history into services that explicitly state they will sell any and all personal data going in. I’ve seen people get fucking fired because they’ve kept asking their personal, unpaid ChatGPT/Gemini work-related questions and when the AI asked for it, they uploaded confidential documents without a single fucking afterthought.
All because they were sold by the fancy autocomplete salesmen that AI will solve all their issues, including pleasuring their wives and fixing their erectile dysfunction. And no matter how many times I tell them to not just trust whatever the model spits out, to think at least a little bit critically, to start understanding that these models are not a fucking search engine… “okay sure”, then ten minutes later I get a lecture about how Claude/Gemini/Cluster McFuckstick or whatever shite they use that offers “free AI”, is completely sure that I am wrong and thus they will keep using it as-is…
The crazy thing is they aren’t really designed for that, especially the reasoning part.
I pictured LLMs proliferating in the way SGLang was taking it: fill in the blank. You’d give raw completion models (not chat finetunes) chunks of structured content (like JSON schemas, names in fields, more complex schemes filled programatically) and it would fill in fields, with intelligent context caching. Like, if you needed to finish a programming function, maybe you give it the file as context and guide it along.
Temperature based sampling felt like a quick hack, just until either looping or structured sampling was figured out.
…But they didn’t fix any of that.
Papers came out on the issues. But the big AI houses didn’t implement any of it. They basically kept it exactly the same and just kept scaling it to burn more compute.
What I am getting at is that OpenAI/Anthropic development is way, way more conservative than anyone thinks, and they basically took temporary hacks and made them bigger. That’s why you can’t trust the code they churn out, because random mistakes and so many other issues are literally part of their core.
Then they turn around and sell them as confident engagement engines…
It’s utterly mind boggling to me.
LLMs in general are probably the wrong shape of model, even for the things LLMs can actually do. But the hyper-scalers categorically will not entertain any other kind of model.
The neural network space will be much more interesting after the crash. Hopefully with a minimum period of self-proclaimed haters shouting “guess it was nothing!” at people trying to build this whole new kind of software for its demonstrable utility instead of a four-comma cocaine fantasy.
Four comma k fantasies aren’t much better.
Unfortunately, Elmo’s nation-state wealth is not a fantasy.
No. Money is very real and obeys real rules.
And Elon Musk really has a metric shitload of it. Orders of magnitude more than Sam Altman. Roughly as much as Sam Altman imagines he deserves.
I know. I’m glad money is a real thing for both of them, and functions by defined rules.
Have you never met anyone from silicon valley?