Legacy tools, API-less web apps, multi-step wizards — describe the goal in a sentence and the agent handles every click, keystroke, and page load. No connector, no script, no browser extension required.
Actionist runs a tight perceive-decide-act loop for every step. It captures the screen, sends it to a vision model, receives an action instruction, executes it, then loops back. Repeat until done.
A screenshot of the active display is captured and sent to a vision model. Every loop starts fresh — the agent sees the actual current state, not a cached assumption.
The vision model interprets the screenshot against your task goal and produces a precise action instruction: click a coordinate, type text, press a key, or call a wait.
Actionist executes the instruction through OS-level input — the same accessibility APIs a human uses. The screen changes, a new screenshot is captured, and the loop runs again.
Every team has tools that resist automation — a vendor portal, an internal dashboard, a multi-step wizard. Computer Use dissolves that wall. If it is visible on screen, it is automatable.
Actionist needs Screen Recording (to see your screen) and Accessibility (to control UI elements). Both must be granted before any automation runs.
Send any task in plain language. Actionist classifies the request, opens a Computer Use session, and starts working. The AI Process panel feeds every step label in real time — you never wonder what the agent is doing.
Actionist's agent is explicitly instructed never to proceed autonomously when it encounters a password field, payment form, or ambiguous choice. It stops and asks you instead. Delegation stays safe.
The AI Process feed is frozen. No further tool calls run until you reply. No credential is ever sent to the model — your reply goes directly to the session continuation.
The session resumes from exactly where it paused — no context lost. The agent uses your reply for the current field and continues the task.
While "Automation in progress..." is shown, the chat input area gives you a live stop control — and the agent itself will pause and ask whenever it needs you.
Every team has a handful of portals and tools that resist automation — no API, no integration, just a browser and a grind. Computer Use dissolves that wall in plain language.
The pattern works for any portal — here is what the same flow looks like over a phone-style channel.
{/* --- Phone vignette: Lena posts a job to a no-API board --- */}The Northwind Careers portal has no public API. The agent navigates it the same way Lena would — opening the browser, clicking through the form, and pasting the job description — in about 3 minutes instead of 15.
Computer Use interprets your intent from plain language. A few habits make the difference between a task that completes in one shot and one that needs redirecting.
Screen automation is a built-in agent tool. It is scoped per agent so you can restrict which of your agents can operate the screen.
The tool is named Screen Automation (Actionist) and described as "Click, type, navigate via CUA." Toggle it on the agent's Tools tab in the studio.
Grant the permissions, describe the goal, and watch Actionist handle every click. No script, no connector, no extension.
The agent stops and asks before it ever touches a password or payment field.
Built-in safety gates · Live step feed · You stay in control.