Math in particular is quite far behind. GPT 5.2 is recommended as the best for highest cost. Really?

Yeah, I mean, I think in theory I could push it to the next tier on Gemini, but at the same time I wouldn't mind trying something else, since maybe my workloads are not really that smart and I am wasting a lot of computational power on something a cheaper model with similar capabilities can do.

Ah OK, yeah, I was actually going to go for that earlier, I'll check it out once I get home, thanks!

1. In real life, most of us use token packages like OpenCode Go etc.

It would be handy to have a site like this one that takes into account the various deals and attempts to calculate the number of tokens per monthly fee for a chosen model. I realize this makes the task a lot more difficult.

2. It would be handy to have a chart like that for the AI hardware that people own. It helps you decide which model to run (resulting in different levels of intelligence and speed). Also difficult to please everyone (preprocessing vs token generation for example) and to keep updated!

I found https://llm-list.com/ yesterday and when I had a detailed look, I quickly found outdated entries, for example looking at GLM 5.3 flash it listed several providers as "free" that weren't free any longer.

If you maintain this over time, maybe include other sources than just AA and update the design to look less like zero-shot claude styling (I know that font! I know that color! Lol) it's genuinely useful :)

This definition of cost is not particularly useful. You want cost per task, not cost per 1m tokens. Artificial Analysis does a good job of this.

A vanishingly small number of people can or would want to go to the hardware expense of running something like GLM 5.3 Flash, much less something like K3.

It's probably worth letting the user specify their actual costs in such a tool. I run a Framework Desktop 128GB that I bought before memory prices got crazy; the current retail price is almost double what I actually paid a year ago.

Best comparison that has occurred to me is the cost a loaf of bread's ingredients might be slightly cheaper than a baked loaf, depending on how you source it. At home you get total control and know what's going in to it. Yet bake at home is still a niche, perhaps a hobby. So I say as someone who's spent hundreds of hours tinkering with local inference, go for it for anyone reading. But most people just want ... some slices of bread, you know?

Guessing that most people don't have machines powerful enough to run good-enough models locally.

For intelligence only as of 8:51 MT

It was expert timing on the commenter's and the developer's part. Opus 5.5 still doesn't show under coding though.

Opus 5.5 is indeed on the Intelligence tab, but I don't see it on the Coding or Math tabs.

As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.

For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.

No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.

Which is why I said "out of their own pocket"

I thought Air was JB's multi-model interface? What is Junie? (I see the buttons, but am very confused by JB's AI offerings in general)

Is it worth the ~10x extra cost over the subscriptions? (This is obviously a leading question). Also, I think you can use OpenAI's subcription login with Air, but not Claude's.

Why not use a codex or claude subscription? If you use the entire usage allotment on the $200 plan it's about $2,000 in equivalent API costs. Switching providers may be valuable but it's quite literally an order of magnitude cheaper.

I do. Sharing training data with OpenAI gives me a lot of complementary tokens. I go above that but it's still quite economical and I pick the right model for the task (Luna for most).

I do, 3-4$ a month of deep seek is enough for my usage

You can't use an API key on subscriptions, but I've gotten around it using the `codex exec` command to run requests outside the CLI or GUI if you're already authenticated on that machine. Won't work for all cases, but I've never ran into a limitation in my use case of not having an API key.

I use local LLMs on my Mac Mini.

which ones do you use?

I do, but via OpenRouter. Outside of work my use cases are small and cheaper models do great job at those. I noticed even if I "burn tokens like crazy" I still pay less than any subscription available (a few $ a month).

But I guess if I had an agent vibecoding on it's own, I'd go with subscription instantly.

You're better off going directly to artificial analysis, this is a feature-poor/misleading/outdated repackaging

Ex. this type of price estimation is quite naive - some models can require 2-3x the number of tokens to achieve the same level of intelligence. Artificial Analysis' own cost per task is a more fair estimation of cost.

On my 64gb m3 max qwen3.8 27b has been great for planning and then letting qwen3.6 35ba3b actually implement the planned changes.

Coding and math graphs are very interesting. Extremely cheap models make it into the upper echelon, delivering 90% of the performance for 1% of the price compared to the #1.

I find it interesting that with the given metric comparison, for coding at min 50 strength, every frontier model brand is from a distinct vendor: Ling, Qwen, Gemini, Muse, Grok, GPT, and Claude in increasing value.

Cost per token is an extremely naive way to index cost, renders this chart essentially meaningless.

The problem is that subscriptions with AI models get a bit 'vague' on what you get and how (or when) you might be rate-limited.

The page looks Claude generated and ranks Opus at the top.

Fun, but this seems to assume all LLM run on SAAS subscriptions. I would like to compare this to local LLM costs by converting my usage load x hardware costs into a token price. In addition if it games the LLM so often, these eventually are optimized and basically cheat on the score.

Astra worse than 5.6 Sol is crazy work

Luna at the bottom of intelligence is laughable...