Google has added a local “computer use” instrument straight into Gemini 3.5 Flash, permitting the mannequin to see and work together with graphical consumer interfaces throughout browsers, cellular apps, and desktop software program.
Previously, this functionality lived in a separate experimental system based mostly on the Gemini 2.5 Computer Use. Now it’s embedded straight into Flash, that means builders not want a separate mannequin to construct brokers that may click on, kind, scroll, and navigate apps.
According to Google, the mixing permits builders to “build custom agents that can see, reason and take action across browser, mobile and desktop environments,” bringing collectively visible understanding and motion in a single workflow.
From chat mannequin to lively agent
The improve strikes Gemini 3.5 Flash past text-based reasoning and conventional operate calling into what Google describes as full agentic laptop interplay. Developers can now construct methods that interpret screenshots, perceive consumer interfaces, and perform multi-step duties resembling filling kinds, operating workflows, or navigating enterprise dashboards.
Industry observers word that this successfully removes a long-standing barrier in AI automation: the necessity for customized APIs for each software. Instead, the mannequin can work together straight with software program the identical manner a human consumer would.
Google reported an OSWorld-Verified UI Control rating of 78.4% for the brand new integration. For comparability, the sooner standalone Gemini 2.5 mannequin scored roughly 70% on a separate benchmark referred to as Online-Mind2Web.
The security structure and enterprise tradeoffs
Giving an AI mannequin complete clearance to click on round a stay working system introduces large safety liabilities. If an autonomous agent wanders onto a malicious web site or opens an e-mail containing hidden directions, it might simply fall sufferer to an oblique immediate injection.
To counter this, Google is taking what it calls a “defense-in-depth” strategy. The firm used focused adversarial coaching to harden Gemini 3.5 Flash in opposition to these actual forms of visible exploits. Additionally, they’re providing two non-compulsory, opt-in enterprise safeguards:
- Explicit consumer affirmation: Requires a human to manually click on approval earlier than the agent can execute any high-risk or irreversible motion.
- Automatic task-stopping: Immediately freezes the agent’s workflow when an oblique prompt-injection assault is flagged.
The most important catch right here is that these options are completely opt-in and never enabled by default. Google acknowledges that no single safeguard is foolproof, signaling a remarkably candid company warning that the expertise continues to be too unpredictable to be left completely to its personal gadgets.
Analysis: Moving past the hype
While the technical benchmarks seem spectacular on paper, enterprise consumers have to look past advertising and marketing guarantees earlier than overhauling their workflows.
The determination to place this functionality into Gemini 3.5 Flash relatively than a heavier flagship mannequin is a deliberate financial play by Google. Flash operates on a pay-as-you-go pricing mannequin and is likely one of the most cost-effective choices in Google’s portfolio. This dramatically lowers the associated fee barrier for firms seeking to deploy large-scale automation.
However, the real-world utility of visible AI brokers stays bottlenecked by sensible limitations. AI fashions working by way of a screenshot-action loop are notoriously brittle. While they excel in extremely predictable situations resembling steady, repetitive software program testing or customary knowledge extraction from company dashboards, they regularly stumble when encountering sudden pop-up home windows, CAPTCHAs, or dynamic web site layouts they’ve by no means encountered earlier than.
Furthermore, the enterprise ecosystem is already intensely crowded. Anthropic’s Claude Computer Use pioneered this area…
