Not to mention companies can also just self host open source models
Everybody has been freaking out about the Copilot price increases and even Uber at blowing 3B on Claude in a few months but I work at one of the frontier model companies and internally it’s been very chill. Just a 24 hour quota to prevent morons from token maxing thinking it’ll save them from layoffs
There’s also tons of optimization that nobody has really touched at all yet since everybody is token maxing. An easy one is a harness that uses the biggest model to research code and come up with an implementation plan and then switch to a small flash model to implement the plan
Lots of people actually already do that now manually just to increase velocity since the expectations are so high now with AI
Are the Chinese models that much cheaper? Enough to solve the problem? The issue appears to be how much power and hardware is required.
They are that much cheaper yea
Not to mention companies can also just self host open source models
Everybody has been freaking out about the Copilot price increases and even Uber at blowing 3B on Claude in a few months but I work at one of the frontier model companies and internally it’s been very chill. Just a 24 hour quota to prevent morons from token maxing thinking it’ll save them from layoffs
There’s also tons of optimization that nobody has really touched at all yet since everybody is token maxing. An easy one is a harness that uses the biggest model to research code and come up with an implementation plan and then switch to a small flash model to implement the plan
Lots of people actually already do that now manually just to increase velocity since the expectations are so high now with AI