Is anyone else in a panic over the Gemini 2.5 model generation (Pro, Flash) being sunset in October before there's even any Pro class model in general availability (with geo restrictions etc.)? Google wants everyone to migrate to 3.x Flash, which beats the older Pro models on the benchmarked tasks, but isn't the same thing as the Pro class on reasoning-heavy tasks like complex reasoning on very large documents (my big use case).
The Gemini family had a distinct niche in document comprehension, with thousand page input documents taking only 300k tokens. Nothing quite like that in OpenAI or Anthropic world, even at more than 10x the token adjusted price. Should we just give up on Google at this point and engineer around the competitors' limits and eat the costs? Totally unnecessary own goal by team Google.
Sounds like they did. IMO, building your business on anything but open-weight models is a bad idea. Unless you're on the s&p 500 you are an insect to google, anthropic, grok (ew), and openai - and they could crush you at any time without even noticing.
Might be missing something —- are there any issues with Gemini 3.1 Pro that aren’t there in 2.5?
I agree, though. 2.5 Pro is a great model. Very competent, knows a lot, and can process tons of text (and videos, and images, and audio too iirc?). Basically unlimited access to it too via AI Studio. I used it for processing and transforming bucketloads of data, ingesting masses of transcripts and converting them to flashcards, etc. I’ll be sad to see it go. None of the newer, cheaper, but obviously less intelligent benchmaxxed smaller models really seem to hold a candle to it for lots of things.
We sure did. It's a great writer, better in a harness, will process lots large context, but complex reasoning with convoluted rules and low hallucination tolerance? That's still larger model territory.
That's fine if you're doing interactive dev tasks, but we're in the large volume, cost effective, big inputs, business still with low error tolerance business, and tuned the heck out of what we can get with minimal fix cycles. Millions of cases at hundreds of thousands tokens each - after all the prefiltering by cheaper models - and the tasks still need them to do convoluted reasoning. So 'usually get it right the first time' is a big part of the cost equation.
Nope. The projects I'm on where we use it, we're carefully migrating to the newer models. Where we can we test with evals to try and get an understanding of how the models have changed.
It's not all roses -- I've seen some regressions -- but generally the 3.x Flash models are pretty great for our use cases.
The great thing about LLMs though is it's incredibly easy to diversify and have fallbacks. But of course that means additional costs, mostly centered around engineering efforts to test and integrate them.
As an alternative, it's not a bad idea to first convert each document to markdown via a thinking or agentic LLM. Embedded figures can even be embedded as readable tables or Latex or Mermaid. Do record the name and parameters of the model that performs the conversion. You can then query the markdown using any model with an input token cost that is exactly equal to the encoding of the markdown. For multiple queries you can also use input caching.
Yup, tried all variations of that. There's the advantage that you can use a lower hallucination OCR specific model for the pre-processing, at least for clean text. But for something hard like handwritten forms, applying VLM with context is less error prone than preprocessing to text.
Also - and this is bizarre - the token cost of doing that is higher, not lower, at least in Gemini world, and by a large margin. That's very counterintuitive, but a page encoded as image tokens can be smaller than same page as text, and is not meaningfully lossy on documents that are just typed text because the models are well trained on those.
And it's mostly from lacking vision and a clear direction, not from being behind on research or engineering. For example, Gemini 3.8 Flash is really impressive.
But their whole model product line is confusing, even the version numbering barely makes sense. They're barely selling agentic coding or the Gemini chat, the fumbled the coding harness race with the weird Gemini CLI / Antigravity rebrand - the whole thing is a mess.
The Gemini family had a distinct niche in document comprehension, with thousand page input documents taking only 300k tokens. Nothing quite like that in OpenAI or Anthropic world, even at more than 10x the token adjusted price. Should we just give up on Google at this point and engineer around the competitors' limits and eat the costs? Totally unnecessary own goal by team Google.
Sounds like they did. IMO, building your business on anything but open-weight models is a bad idea. Unless you're on the s&p 500 you are an insect to google, anthropic, grok (ew), and openai - and they could crush you at any time without even noticing.
Might be missing something —- are there any issues with Gemini 3.1 Pro that aren’t there in 2.5?
I agree, though. 2.5 Pro is a great model. Very competent, knows a lot, and can process tons of text (and videos, and images, and audio too iirc?). Basically unlimited access to it too via AI Studio. I used it for processing and transforming bucketloads of data, ingesting masses of transcripts and converting them to flashcards, etc. I’ll be sad to see it go. None of the newer, cheaper, but obviously less intelligent benchmaxxed smaller models really seem to hold a candle to it for lots of things.
It's still 'preview' and not generally available, so can't run it for US restricted workloads.
Check model garden on vertex ai for other models that you can access
Models you can download and use elsewhere if Google nixes access
Flash is the new Pro, try it first
We sure did. It's a great writer, better in a harness, will process lots large context, but complex reasoning with convoluted rules and low hallucination tolerance? That's still larger model territory.
There's still 3.1 Pro though, as ancient as it sounds now
Yup. The problem is that it's bizarrely still not in General Availability status.
sounds like you might need to beef up your harness first, and run multi-agent verification loops
That's fine if you're doing interactive dev tasks, but we're in the large volume, cost effective, big inputs, business still with low error tolerance business, and tuned the heck out of what we can get with minimal fix cycles. Millions of cases at hundreds of thousands tokens each - after all the prefiltering by cheaper models - and the tasks still need them to do convoluted reasoning. So 'usually get it right the first time' is a big part of the cost equation.
Sunsetting a model with a two-month notice is exactly why the open-weight argument keeps winning. The API is a dependency you don't control.
Nope. The projects I'm on where we use it, we're carefully migrating to the newer models. Where we can we test with evals to try and get an understanding of how the models have changed.
It's not all roses -- I've seen some regressions -- but generally the 3.x Flash models are pretty great for our use cases.
The great thing about LLMs though is it's incredibly easy to diversify and have fallbacks. But of course that means additional costs, mostly centered around engineering efforts to test and integrate them.
2.5 flash was also good to use with as the llm layer for voice products.
but google gonna google.
As an alternative, it's not a bad idea to first convert each document to markdown via a thinking or agentic LLM. Embedded figures can even be embedded as readable tables or Latex or Mermaid. Do record the name and parameters of the model that performs the conversion. You can then query the markdown using any model with an input token cost that is exactly equal to the encoding of the markdown. For multiple queries you can also use input caching.
Yup, tried all variations of that. There's the advantage that you can use a lower hallucination OCR specific model for the pre-processing, at least for clean text. But for something hard like handwritten forms, applying VLM with context is less error prone than preprocessing to text.
Also - and this is bizarre - the token cost of doing that is higher, not lower, at least in Gemini world, and by a large margin. That's very counterintuitive, but a page encoded as image tokens can be smaller than same page as text, and is not meaningfully lossy on documents that are just typed text because the models are well trained on those.
Google is falling behind in this competition.
And it's mostly from lacking vision and a clear direction, not from being behind on research or engineering. For example, Gemini 3.8 Flash is really impressive.
But their whole model product line is confusing, even the version numbering barely makes sense. They're barely selling agentic coding or the Gemini chat, the fumbled the coding harness race with the weird Gemini CLI / Antigravity rebrand - the whole thing is a mess.
3.8 Flash on high thinking is better than 3.1 pro.