Gemini on Mac rolls out computer use for desktop and web tasks
Google is rolling out computer use for Gemini on Mac to US English subscribers on Google AI Plus, Pro and Ultra, letting the app click and type across desktop apps and websites.
Wire notes
- Computer use is rolling out today in US English on Mac for Google AI Plus, Pro and Ultra subscribers.
- Users activate the feature by adding @Computer use to a prompt.
- Gemini can be stopped with one click and pauses for logins, payments and sensitive actions.
- Customize > Connectors adds Blender and Autodesk Fusion as “Desktop” apps alongside Git and Obsidian.
- Google lists color grading, shipping quotes, Google Slides presentations and resale listings as example workflows.
By Elena Vasquez
Google is rolling out computer use for the Gemini app on Mac to US English subscribers on Google AI Plus, Pro and Ultra starting today.
The new capability, computer use (the ability to click and type on a user’s behalf), works across local desktop applications and websites, Google said. It can execute multi-step tasks, including navigating menus and filling out forms. The app’s existing Task function, previously called Gemini Spark, could only access local files alongside Connected Apps (linked services) such as @Gmail and @Google Calendar.
How does Gemini control the screen?
Gemini works in an “isolated background window with a live preview right in your chat,” Google said. The main screen and cursor remain available to the user. One click stops Gemini, and users choose which applications it may use. The system also pauses for logins and payments, while sensitive actions require confirmation.
Google’s design separates visible supervision from execution. Users can watch the live preview, retain their cursor and interrupt an action, while Gemini pauses before sensitive steps. The preview keeps the sequence visible while the feature runs. Visibility gives users a check before any login, payment or sensitive action.
What changes from Task to computer use?
Google’s existing Task function provided a path from Gemini to a local file. Its new computer use mode adds direct interaction with menus, forms and controls inside desktop software and web pages. The distinction is operational, not cosmetic. File access gives Gemini information for a workflow. Clicking and typing lets it carry out later steps in that workflow.
Google said many everyday tasks span desktop apps and websites that do not connect to each other. Computer use addresses that gap by performing repetitive steps in the background while leaving the screen and cursor available. A shipping quote can combine values from a local document, entries in a web form and a completed response.
Which workflows can Gemini handle?
Google provided four example workflows:
- Batch color-grade photos in a preferred design application or set up complex three-dimensional (3D) scenes.
- Extract weights, dimensions and addresses from local Portable Document Format (PDF) files to fill out multi-box shipping quotes.
- Pull metrics and screenshots from live dashboards across multiple regions, then build a presentation in Google Slides.
- Analyze clothing photos from a local folder, research pricing, log item details and stage listings on resale platforms.
Together, the examples move beyond a single command. They combine local assets with web research or data and produce a quote, slide deck or listing. The mechanism handles repetitive transitions between applications rather than completing only one isolated action.
How do users turn it on?
Users add @Computer use to a prompt. In Customize > Connectors, new “Desktop” applications including Blender and Autodesk Fusion join the existing Git and Obsidian integrations, Google said. Google’s workflow keeps activation inside the prompt and application choice inside the connector menu.
Who gets access now?
Computer use begins rolling out today in US English for Google AI Plus, Pro and Ultra subscribers, Google said. The release centers on the Mac app and these named subscription tiers.
For those subscribers, the rollout will show whether a live preview, one-click interruption and required confirmations can support repeated cross-app work without slowing the user.
via 9to5google.com (Source)
More from Elena Vasquez
Show full bio
Senior reporter covering technology and product news in marketing, advertising, marketing technology, media, SEO and search at Marketing Herald.
372 articles
More on the wire
- Google Brings Gemini Live to CarPlay in Global Rollout
- Google opens Gemini Tasks to free users in six countries
- Google Now Lets You Text and Email Gemini Tasks, US First
- Google Brings Gemini to Apple CarPlay as Android Auto Cuts Off Older Phones
- Google Opens Public Preview of Connected Apps for AI Mode and Gemini