News

Google updates Gemini for Mac with computer use capabilities to perform agentic tasks

The era of the passive AI chatbot is ending, replaced by autonomous agents capable of manipulating a computer's operating system to perform actual work. Google has moved into this space by expanding Gemini for Mac to include "computer use" capabilities, allowing the AI to interact with local desktop applications and the web on a user's behalf.

This new functionality transforms Gemini from a conversational tool into an active agent that can click, type, scroll, and navigate menus and forms. Rather than simply providing information, the AI can now execute multi-step tasks across various programs. To maintain user productivity, Google has designed the system to operate within an isolated background window. This ensures that the user's main screen and cursor remain free, allowing them to continue working in the foreground while monitoring the AI's progress via a live preview in the Gemini chat interface.

The practical applications of this technology center on automating repetitive, tedious workflows that typically require jumping between disparate apps. For creative professionals, this could mean batch color-grading photos in design software or setting up complex 3D scenes in applications like Blender and Autodesk Fusion. For those managing e-commerce, Gemini can analyze clothing photos in a local folder, research market pricing, and stage listings on resale platforms. Other use cases include extracting data from local PDFs to fill out shipping quotes or pulling metrics and screenshots from live dashboards to build a presentation in Google Slides.

To trigger these actions, users can add @Computer use to their prompts. While the AI can handle a vast array of tasks, Google has implemented several safety layers to ensure the user remains in control. Users must explicitly choose which apps Gemini can access and can pause the process at any time. Furthermore, the assistant is designed to pause for logins and payments, requiring manual approval before taking any sensitive or "key" actions.

Access to these features is currently limited to users in the United States with a personal Google account and a subscription to the AI Plus, Pro, or Ultra plans, which start at $4.99 per month. On the hardware side, the software requires an Apple silicon Mac running macOS Sequoia 15 or later.

The Competitive Landscape

Google's move is a direct response to a growing trend among AI developers, as OpenAI's ChatGPT and Meta's Muse already offer similar computer-use features for Mac. This shift toward agentic AI places significant pressure on Apple to evolve Siri. While Siri can currently handle tasks within Apple's own ecosystem, its ability to interact with third-party software remains limited, though Apple has released frameworks to help developers bridge this gap.

The rise of these powerful AI agents has also prompted Apple to tighten security and transparency on the Mac. In the upcoming macOS 27.3 update, Apple will introduce new warnings regarding the risks of granting apps unrestricted access to data. This update will include a transparency indicator to signal when one app is controlling another, as well as more granular controls allowing users to limit access to specific folders or require permission each time an app requests access.

These developments suggest a looming conflict between the desire for seamless AI automation and the necessity of OS-level security. As Google and its competitors push for deeper integration into the desktop experience, the industry is moving toward a future where the primary interface for computing may no longer be the cursor, but the AI agent directing it.