It seems like the major providers are starting to cut back the usage included with their monthly subscriptions. How are people handling inference for side projects these days?
Are you still using Codex/Claude subscriptions, paying for hosted open weight models, running local models, etc
Claude subscription (the cheapest $20/month one) -- I very rarely hit the limits, particularly the weekly ones. But then again, the largest codebase I ever work on is 50k lines.
I'd prefer local models massively for both work and "pleasure" (I love LLMs as random cognitive sparring partners), but then again, my bank account massively disagrees :/
I've been on the Claude Max plan for quite a while now but I also run my own local models and every week there are new local models that are slowly but very surely closing the gap to frontier-level quality and so, pretty soon, I won't use APIs at all. Can't wait!
Are you still using Codex/Claude subscriptions, paying for hosted open weight models, running local models, etc
Claude subscription (the cheapest $20/month one) -- I very rarely hit the limits, particularly the weekly ones. But then again, the largest codebase I ever work on is 50k lines.
I'd prefer local models massively for both work and "pleasure" (I love LLMs as random cognitive sparring partners), but then again, my bank account massively disagrees :/
I've been on the Claude Max plan for quite a while now but I also run my own local models and every week there are new local models that are slowly but very surely closing the gap to frontier-level quality and so, pretty soon, I won't use APIs at all. Can't wait!
I have a Proxmox machine with dual P40 gpus that pass thru to a vm with ollama running for local inference. Been working great for over 2 years.