Not a great impression to have your demo video demonstrate how one of your 'most advanced' AI models loses to the most common check-mate pattern in all of chess.

Gemini is underrated in that it produces the only prose that is somewhat bearable to read.

For heavyweight work I have been using Astra, but for rabbit holes and brain storming Gemini is far more enjoyable to interact with.

I'm worried in their push to catch up on the SOTA front, it's going to lose that natural sounding touch it currently has.

Mostly because it answers quickly and is more agreeable (too agreeable at times).

Meanwhile Claude and Astra like to couch all their agreements with caveats and provisos.

Meanwhile Claude and Astra like to couch all their agreements with caveats and provisos.

Sometimes that's what being smart sounds like.

This is commonly why, on Reddit in particular, you can get eaten alive.

Someone confident but incorrect, can often sound more convincing than someone with actual expertise. The expert must add caveats/hedge, because those are the facts on the ground, whereas the person reciting google can be entirely confident.

Of course the people judging aren't experts, so they side with confidence and simplicity. Heck, just writing shorter replies on Reddit is rewarded. Nobody reads the articles, let alone a paragraph-long reply.

That all being said though, there are limits. Sometimes LLMs on high-thinking go off on full tangents based on little, and don't have the self-awareness to bring it back.

In my experience actual experts don’t hedge because they have a perspective. They might say “I think”, but avoid weaseling.

I consider, on the contrary, caveating and hedging annoying 'typical redditor'/internet behaviors: they care more about being "technically correct" than conveying the message. On the internet, if you make even the tiniest mistake or simplification, someone will criticize you, so you're trained to always hedge. In normal discussions with friends you can just make general statements and people get what you mean.

“caveats and provisos” makes me think of Robin Williams’ genie imitating William F. Buckley Jr.

"couch all their agreements with caveats and provisos."

When you're a ChatGPT Projects or Claude Projects user, those caveats and provisos are your worst enemy because they'll change caveats into hard rules (either for the session or committed to memories) and you end up in absolute hell having to make it investigate to figure out why it can no longer produce anything but read-only pre-check code that never actually does anything but keeps performing stupid safety checks.

Agreed. My impression is that the more verbose output of sol, astra etc is that it helps it steer itself on long running tasks (but is worse for the human user to read)

Is Google still chasing frontier? Seems like they haven't had a "Pro" model in forever. I think a good niche for them would be right where they are now.

How do you know you're using Astra?

My ChatGPT env only says "low", "medium", "high".

Is this a "pro" thing? I have totally no idea what I'm talking to, so actually I'm thinking of stopping my plan. Gemini and Claude are much more clear about it.

Anyway, I like the speed at which Gemini responds so indeed for simple things it is preferable.

For me (Plus plan, iOS app), Astra only shows up under the Work tab.

I’ve been using Work for all my queries, since it seems to just be the same interface as Chat but with more features. I don’t understand why they’re two separate things.

Wondering if people have managed to have Gemini in-front of other models like claude/codex models and only interact with that. Having Gemini act as a pure human/llm translator.

Not in the principled sense you mean but I have in fact recently started having Gemini explain to me what Claude is talking to me about, lol.

BTW, as kind of a follow-up to this, I think the most important finding to report--for those who only use one model, as many people seem to--is just how much more often Claude seems to be extremely confidently (and even insufferably) wrong than any of the other three big models (all of which I use quite often... yet I only pull out Claude when I've given up hope in a problem and are looking for out-of-the-box brainstorming).

And like, it does this despite it speaking in extremely dense math, which both makes it sound correct and requires a lot more effort to prove when it is wrong... yet, it isn't actually correct more often, and so that time sink just isn't worth the benefit. I then think many people--including people who can speak math (as can I)--just stop bothering to check everything, as if you come across a human who speaks like this it probably does correlate with slow and careful thought that helps prevent errors.

Instead, Claude has the mistake rate of a somewhat accelerated beginner impossibly combined with the language of an expert professor; and we as humans just aren't good at that combination: it becomes very dangerous and makes it take longer to spot its egregious mistakes and trained-in biases. If you have to use Claude, I thereby claim you really need to have a team of not-Claudes to help insulate you from this, and Gemini (while being a bit senile) is a lot more collaborative and approaches problems in ways that makes it harder to get tricked.

(To translate this into more of an engineering analogy: Claude always feels to me like the engineer who put more effort into learning how to program in functional languages than into how to actually develop working code, and then confidently presents you answers in Haskell or Lisp that never quite work. To find their errors is then very costly. In contrast, Gemini feels more like a Java or Go developer who knows they are a cog... that's helpful! <- Which maybe just goes to show that AI has finally turned me into a manager, omg.)

It's also the only model that generates accurate translation and localization. No other frontier model comes close. Although Gemini's coding capabilities are subpar, its natural language processing is top-tier.

I’m curious how you guys keep track of each model’s coding capabilities. The landscape keeps changing. I don’t suppose you benchmark all frontier models every other month, right?

I use them. Daily. Gemini hasn’t been a contender by comparison for a long time.

I wonder if it's a harness thing or a model thing at this point. I feel all coding models are quite capable for most tasks I want them to do.

Most of the time I don't need what the bench tests and I'm not really giving them completely ambiguous tasks without any refinement.

I only find marginal differences between models at this point and it almost feels like personality quirks in each model than anything.

When comparing OpenAI and Claude thats pretty much true, but not Gemini... And have you tried Antigravity? Yikes

The CLI version of agy is great. Have you tried it?

Do you dangerously allow permissions? I absolutely cannot use it until they ship an auto approver. As it is now I have it write one bash/python script to do everything it wants to, then I review that. Otherwise it is COMPLETELY unusable and it shocks me when I hear people are using it.

I did a test involving implementing cobol control flow in Java for a source to source translation project. Gemini was the only model to get the edge cases. Cobol is very peculiar in this regard.

I found that it's shockingly good with R. (the only language I know and can correct for)

I doubt they even intended it to be, but it seems like I kept going from resorting to 3.5-3.8 (over time) to realizing that Claude and GPT, while great at Python, will make rudimentary mistakes with R; even when they compose giant complicated R code.

In my experience, with minimum prompting, deepseek also generates very decent text.

Try Gemini live in a multi lingual environment. It can pick out speakers and live translate to you. Truly underrated for its capabilities.

I actually did (was going to travel internationally), and it wasn't as useful as you'd think. I would be talking to someone, and in the background someone else would be talking, and it would translate both people.

Only worked in a 1:1 in a quiet place. Still, can't complain for free.

"Only worked in a 1:1 in a quiet place"

well then its not model problem

As sister comments have pointed out, I find that it's very much dependent on the phone you're using. If it can't even show me a waveform then I don't think it's going to be able to tease out any sort of speech

I was using Google's Translate app.

I was surprised when (finally) trying out Claude how much I preferred Gemini's way of communicating. I wont argue Claude is better at coding, but for knowledge work, I had to dig through Claude output to find what I actually wanted. At times, it even felt borderline incomprehensible.

Just today I had Sonnet 5 generate this (asking about always-on display in the iPhone e-versions):

This mirrors how Apple has always segmented Pro vs. non-Pro iPhones: base models got LTPS panels while Pro models got LTPO, and only with the mainline iPhone 17/17 Plus did that gap close the standard versions previously lacked the smoother 120Hz ProMotion technology and the always-on display feature, unlike the Pro models — the 17e is the one model line still using the older, cheaper panel.

(emphasis mine)

I mean, I can guess what it is trying to say, but who RL'd this nonsense?

Opus specifically talks as if having a stroke. 4.6 was the last version that was pleasant to work with

Chatgpt doesn’t seem so bad lately. At least as a Claude refugee.

I work for Google and we have the choice between Gemini models and Opus. Opus is slightly better than Gemini flash but I find the style unbearable.

It reminds me of a pedantic grad student.

Our company's Google Workspace Business only offers 3.6 flash & thinking in the Gemini App. Has anyone else seen 3.7 or 3.8 roll out?

3.6 Flash and 3.1 Pro are included in the basic Workspace subscription. The Workspace admin has to upgrade your seat for the access to newer models ($17/mo now, $24/mo starting Jan 2027).

Gemini's Live Mode is already much better than GPT Voice in my personal experience, even though it was much dumber. It really does feel like talking to a real person. ChatGPT keeps humming to whatever I say and has some weird voices.

Excited to try this out! Shame on Google for not releasing Gemini 3.8 for Google AI Plus users yet, though.

OpenAI just released the new full duplex mode to the API as gpt-live-1 or something like that. Very realistic.

Agreed. I've been using GPT-Live-1 this week, with Claude as the backend brain. It's amazing, feels like working with Jarvis. It certainly made me feel there's no point in human telephone support now - but I'm sure I'd find edge cases if that really was something I wanted to build out myself.

When will it be available on Vertex?

vertex is dead, for good reason too

I wonder when/if we’ll see Gemini beating Fable and Astra. Last year I would have confidently bet Google will overtake the others just because they have the data, the hardware (TPUs) and a fat advertising money pipe and yet they are still behind. Anyone anonymous at Google want to hint when Gemini 4 will be out?

If Google didnt have their ad buisness theiy'd be out by now. They're like BlackBerry and Nokia at this point almost.

Not sure what your comment mean in the context of parent's comment.

As opposed to what, them not having it and burning money that isn't their instead like openai and anthropic? At least Google is feeding itself instead of having to create a bubble to stay alive

Google spent most of their cash, they are now taking loans for data centers too. They recorded their first quarter of negative cash flows ever

Google has in excess of $121 billion of cash (& equivalents), net of total debt.

Big rich companies take on debt for reasons that are sometimes inscrutable from the outside. Recently, they have been borrowing for ~5%, about a half point above what the US government gets for 10-year Treasuries.

Apple has been financing operations with debt for a number of years as part of a complex optimization plan.

No, Google is not broke.

As an "everyday mans AI" I'd say 3.8 Flash definitely already has. Smart enough for the vast swath of people, and only slightly eeked out by Astra(Max) on vision capabilities, like the kind of "Point your camera at something and ask questions" that non-tech people like to do. It's crazy fast and very compute light, so not getting bogged down constantly.

I can't think of a better general purpose model than 3.8 flash right now. It also writes more naturally than the other big models too.

Yeah its good. Reasonably priced too (at current prices, if they do raise them in January I would stop recommending it). 3.7/3.8 were good releases.

To me it's a good replacement for search engines. I ask it things like 'If redshifting destroys energy ala Noether, than how can we say that time is reversible or that entropy will find an equilibrium?' and it will not only explain, but make nice interactive diagram/toys to help. A regular search engine would have taken me hours to find an answer.

However, if I want it to DO something then Gemini is in absolute last place. I don't trust it for anything more than renaming files that I don't care about very much or extracting data (though it's too expensive for data extraction at scale).

I was thinking about coding specifically. Also, see all these math and physics breakthroughs, it's usually not Gemini but Fable and Astra.

I think 3.7 Flash is very good at coding. I have access to both that and Opus 5. Opus is only slightly better IMO and it's much more frustrating to read.

For coding models? I don't think Google is motivated to fight in that market. There's no incentive for them.

Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models? They would just be competition.

Famously, Google just eventually discards almost all businesses that don't have the same fire hose of revenue that ads does. Selling coding plans isn't something they are going to want to do.

Google is clearly motivated to make better search and information finding tools and stuff that will ultimately drive users through their existing search/ads/youtube ecosystem. That's really why they're in Android, that's why they do Chrome. Everything else with them is a sideshow.

Google is also full of beancounters obsessed with data centre quota and resourcing. Even massively profitable ads projects have to justify and fight for it. (Source: used to work there).

I can't think of anything less resource & revenue sensible than providing outside parties access to your TPUs for the purpose of letting them write stuff which could just end up competing with you.

Yes, maybe as part of their cloud business, selling token access could be useful money. But I doubt they'd tune it for coding.

Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models? They would just be competition.

Bragging rights to say they have a SOTA model. I guess that was more like Google of 10 years ago with moonshot projects. Nowadays, yeah, perhaps if it's not helping sell ads, it doesn't make sense.

Ask yourself, how does Google -- a company that famously does everything -- benefit from SWEs outside of Google having access to powerful coding models?

Catch is, even Googlers internally do not have access to top-tier models. (or did not until recently, when apparently Claude was made accessible to the SWEs internally).

Googlers now have internal access to frontier models.

Funny enough, after trying it, I went back to G3.8

I’m wondering if they even see a coding agent as a valuable prize. It’s a competitive market in a race to the bottom economically, hard to establish consistent differentiation and virtually zero switching cost for customers.

I think they’ve made a shrewd move in focusing on search integration and everyday users (Gemini app) vs software power users. They have their corner and nobody is really competing with them, plus it feeds directly into their existing revenue stream.

OpenAI/Anthropic are being valued at 25/50% of Alphabet respectively in secondary markets.

Maybe everybody is wrong about how valuable these companies will be, but atm it looks very dumb to not be competitive in coding.

Anecdata, but I've been using gemini personally instead of what I used regular google searches for the better part. Even in car chat/search and some lighter and not so light research (if it's not heavy on technicals). It's good enough that I use it nonstop like that. I wouldn't trust it for coding at all - switching between fable and now astra.

Google in a sense won (me over) like that. I also expected them to brute force their way into everything and dominate. This is how it played out though. Image generation is great as well, but ChatGPT one is more lenient on copyright and nannying - for example when my kid asks me to "take a photo of him and Sonic". Gemini cops out either because of the kid or Sonic, disappointing us both, but ChatGPT can be.. persuaded.

Ensure transparency with SynthID watermarking

All audio generated by our AI products is watermarked with SynthID. This imperceptible watermark is woven directly into the audio output, ensuring AI-generated content remains detectable to help prevent misinformation. For details on our approach to safety and responsibility, review the model card.

Great tech but the voice is like nails on a chalkboard to me

They should just give up at this point, it's just embarrassing to watch.

As PrimeTime said; these are the guys that invented the 'T' in 'GPT', that deployed their first TPU in 2015, that is using billions on AI - and they are beaten by 300 people startup named Moonshot AI even. People are going to write books about this complete fumble.

And yet they might become one of the winners "in the end" because they have near infinite money and others have not. I will drink tea and watch the show.

None of the startups are profitable. What exactly are they getting beaten at?

My advice is to listen less to brainrot 'influencers' that optimise for engagement through sensationalism.

Intelligence.

They have "unlimited" resources and has researched AI since the very beginning - PageRank is a form of AI even. And still, Gemini is behind Claude, GPT, Grok, Muse, GLM, Kimi and is maybe on par with DeepSeek?

As I said, it is embarrassing.

If RSI is achievable, it will leapfrog everything produced so far and so it will make sense to focus on RSI instead of incremental improvements for your top model. Startups need investment and need to show progress. Google does not at the moment need to take lead in the current race.

Have you used Google search at all on the past few months? Every single search brings up a live chat prompt. They're serving fast AI to billions of users at huge scale everyday

And they're making money doing it.

Perhaps they don't have the best coding model right now (although 3.8 flash is arguably SOTA at some benchmarks), but is that the be-all and end-all of AI? Only coding matters?

It’s so annoying that everyone just points to the Artificial Analysis index (or even worse, Epoch AI, where part of the score is how good the AI is at chess) as a proxy for “how good” the model is.

Its 200-300 TPS. GPT is at 50-60. And its like 5% worse? Yeah that is a good tradeoff.

Most people talking about Gemini quality do so by experience.

sounds vibes and bias laden

brings up a live chat prompt. They're serving fast AI to billions of users

Just because it shows up does not mean it is being used. I only click the feedback button to tell them how much I dislike their Ai summaries. They have removed that feedback button this week. Can't take the heat I suppose.

I've also never know anyone who finds them trustworthy nor heard someone say anything besides how they also dislike them

Strongly disagree with that take. Kimi K3 is a distilled model. I'm not saying that as a moral judgement, or to disparage the team behind it, but distilling and building on that is significantly easier and cheaper than building from the ground up.

And Gemini is kinda good enough at everything. Never the top, but it is decent at every task, and it is much faster than Kimi K3 and significantly cheaper. Kimi is very focussed on coding, Gemini isn't.

More importantly, it natively understands text, audio and video. If/when we are able to make the jump to robotics, this becomes essential. As you say, Google has a lot of deep background and deep pockets, they are able to make more of a long play. No idea if it will pay off, but it is way to early in the game to count them out.

I'm disappointed with "Extended Thinking" for 3.8 Flash. On the plus side, it's a strong general-purpose model and the cost-benefit is still compelling.

However, the "Extended Thinking" should be renamed to "Slightly Extended Thinking". Considering that it's the maximum thinking option for Gemini Flash in the chat UI, it doesn't actually think a whole lot, leading to an uncomfortably high number of incorrect/poor replies.

My Gemini app is still stuck at 3.5 Flash-lite and 3.6 Flash so I truly don't understand how Google rolls this stuff out. I don't use Gemini for anything serious so I'm not going to use the API, but it's my go-to for just searching basic information (replacing google search) because it's so darn fast.

Yeah still on 3.6 here too, this is like the 4th or 5th model Google has announced since they last gave me access to the latest. And I pay for pro too!

I have been looking for a model that's good for GUI testing. Original computer use isn't right because it's a slow screenshot loop, which doesn't capture transition and animation. Docs says this one does up to 1 FPS. That might be fast enough. If not now, we must be within a few months of high enough sample rates to do it.

Did anybody watch the Primeagen's video on Google bag-fumbling? Interesting they released on the same day!

I love talking to chatgpt voice mode. Voice to voice AI is the only big leap that I see after the RL trained coding models.

My issue using the voice mode is the overly expressive mimicry of natural human intonation is distracting and starts to become extremely grating after a while.

There's one or two I find more understated but I would love a 2026 SOTA V2V model that speaks clearly but without the artificial personality layered on.

Human interaction/theory of mind relies so much on non-verbal clues for interpreting emotion/intent and so for me having those neurons firing constantly while talking to an LLM just for an emotional no-op is exhausting to put up with for more than a couple minutes.

There's one male voice that would make me assume someone was sarcastically mocking me if I was talking to an actual person because it's just so over the top.

Just gave it a try - very solid release.

Copes well with thick accent, voices are pleasant and latency seems low.

Oh and I can actually use it on a workspace account - which for most of the recent releases was an account stuck in limbo. Not personal enough for personal offering, not enterprise enough for enterprise.

Well done G - will definitely be using this

I'm wondering if Google intends to drop the next major version of Gemini Pro as a total bombshell drop to make Anthropic and OpenAI panic. They seem to be taking their sweet time on frontier model updates.

Google need to allow saving history and exclude it as training data. I will not use it seriously until this is resolved.

It is completely broken for me. After I ask a single question, it starts replying to itself in an infinite loop. It answers my question, then generates another reply to its own response, and keeps going. At some point, it even starts switching languages randomly.

My first language is Afrikaans, which is a somewhat niche language and hard to find teachers/conversation buddies outside South Africa. (I live in USA now)

I've been using Gemini to live chat in Afrikaans and do impromptu Afrikaans grammar lessons during my solo drives around town. It is phenomenal at speaking the language - like, it really shocks my family members when they hear it.

This is probably the most joy I get from any of my usages of LLMs/AIs. It's been really, really nice getting to speak my language regularly again. =)

So, I'm excited about this release and live chat getting better. I also hope the other frontier labs pick up niche languages like this as well so that I have more options.

Yeah Gemini has been consistently better than open ai's chat in icelandic, but I would still say it's far from passable as natural sounding. Lots of grammar errors and the pronunciation sounds like a non native speaker.

I have a similar experience using Gemini for quick Catalan translations for iOS apps given enough context.

I once asked it to summarize The Hobbit in Catalan to explain it to my daughter before sleep. I was expecting a lot of mistakes as I see regularly if I ask anything in my native language when using GPT or Claude, but it was surprisingly good. I was going just to kind of skim ahead and retell it my own way, but ended up almost saying it verbatim because it was good already.

She loves Zelda so I asked it to explain the story of Breath of The Wild keeping the original names, and to make it fun, etc.. I was surprised again. I did retell some bits in my own style and taste but it is very convincing.

I haven't tried Catalan on newer models like GTP-6 Astra or Fable tho. We have all these benchmarks based on software development, and AGI, etc.. but it would be cool to have some language benchmarks for different communities.

As I work in english and use them in english, I wonder if using LLMs in a different language to code renders a different result as well. Like, if some of these benchmarks were made in other languages, would the result be similar.

As I work in english and use them in english, I wonder if using LLMs in a different language to code renders a different result as well. Like, if some of these benchmarks were made in other languages, would the result be similar.

I am not a Chinese speaker but my understanding is that all of the models have substantially different behavior in Chinese, to the degree that it's kind of like a second model. Would be interested in hearing more if anyone has direct experience.

Fwiw, it's been very easy to nudge Qwen 27B/35B to think in Chinese. No need for <think> prefills like "思考：" ("Think:") or such.

Relatedly, when chatting with frontier models about science education content design, I've found it very helpful to mix in Chinese education terms. In English, for example, NGSS is such a massive attractor, discussing nearby topics often yields NGSS "slop". And "estimation" (educational) in the US means one (dysfunctional:) thing, which similarly distracts. Perhaps if AIs become increasingly multilingual, but remain weak at deep conceptual reasoning, it may be fruitful to have multilingual thesauruses, to use language-associated cultural conceptual differences as a way to convey conceptual nuances with which LLMs otherwise struggle?

I love Gemini in Google Maps for long drives. I start getting bored of music and podcasts and start grilling it with random questions I've always wondered about.

I do this too! Usually some idea in ML ...when I am driving I might suddenly remember what I was thinking about and then it is my personal podcast via Gemini-in-Maps. My only complaint is if you have follow up questions, you have to be quick, otherwise it cuts off the mic.

I feel like a 10 year old all over again with my frequency of questions.

How did the early Roman Empire interact with Greek city states?

How are LLMs planning on learning new information on the fly without new context or retraining runs?

Why did mom leave?

You know, standard stuff.

In Maps? Or is it Android Auto? I have Google built it. When I push the button on steering wheel I hardly know which layer I'm talking to.

Really? I am in Android Auto and I try to even just ask it to get me to "Costco Near Me". I know where it is. I wanna know if the sometimes there traffic jam that blocks the right lane is gonna expect me or not without being distracted zooming out from current location and then back in to the highway exit I know is prone to that at times. You know, I wanna be a good citizen and not be distracted by looking even a second at the screen I'm zooming around in.

It tells me there's a Costco 10km from me and I'm like, yeah let's go there, that's the one I meant! And then it sends me to one that's like 40 minutes from me all across town and I'm like WTF?!

Never mind asking Android Auto any question that's not driving related. It just doesn't understand me at all. Or even some driving related ones. "Find Alternative route". "Sorry, I can't help you with that". WTF?

I've been building a language learning app which uses AI voice chat to let people practice. Would that be something interesting for you to try out? It's all still rough, but we've been learning Japanese with it and its pretty cool.

I want to, I have like a 2000+ day streak on Duolingo and I know I can only gain fluency once I start speaking it more.

We (me and my teens) used Duolingo for a few years and.. didn't really get that far in actually being able to speak. It's kind of mind-blowing that they are the market leaders. In retrospect, I think of it more like an addictive language game and not really a good way to actually learn to speak.

If you're really interested, shoot me an email.

No more Gemma?

I don’t understand good experiences people are having with Gemini. It’s the only model that sometimes loses/forgets context in literally next message. Plus feeding unasked product links to responses.