If you can click it, Actionist can automate it.
Legacy tools, API-less web apps, multi-step wizards, describe the goal in a sentence and the agent handles every click, keystroke, and page load. No connector, no script, no browser extension required.
The cycle that runs every action.
Actionist runs a tight perceive-decide-act loop for every step. It captures the screen, sends it to a vision model, receives an action instruction, executes it, then loops back. Repeat until done.
until done
A screenshot of the active display is captured and sent to a vision model. Every loop starts fresh: the agent sees the actual current state, not a cached assumption.
The vision model interprets the screenshot against your task goal and produces a precise action instruction: click a coordinate, type text, press a key, or call a wait.
Actionist executes the instruction through OS-level input. The same accessibility APIs a human uses. The screen changes, a new screenshot is captured, and the loop runs again.
The API wall is gone.
Every team has tools that resist automation: a vendor portal, an internal dashboard, a multi-step wizard. Computer Use dissolves that wall. If it is visible on screen, it is automatable.
Before Computer Use
With Computer Use
Grant the two required permissions.
Actionist needs Screen Recording (to see your screen) and Accessibility (to control UI elements). Both must be granted before any automation runs.
Look for the red banner
Open Permissions Required
Grant Screen Recording
Grant Accessibility
Confirm success
Watch a task unfold step by step.
Send any task in plain language. Actionist classifies the request, opens a Computer Use session, and starts working. The AI Process panel feeds every step label in real time. You never wonder what the agent is doing.
Open an agent's chat
Describe the task
Watch it work
Receive the result
Passwords and payment info.
Actionist’s agent is explicitly instructed never to proceed autonomously when it encounters a password field, payment form, or ambiguous choice. It stops and asks you instead. Delegation stays safe.
The AI Process feed is frozen. No further tool calls run until you reply. No credential is ever sent to the model. Your reply goes directly to the session continuation.
The session resumes from exactly where it paused. No context lost. The agent uses your reply for the current field and continues the task.
Agent reaches a credential or payment field
Input area changes to 'Reply to continue...'
Type your answer and press Enter
Agent proceeds
Control an automation in progress.
While “Automation in progress…” is shown, the chat input area gives you a live stop control, and the agent itself will pause and ask whenever it needs you.
- Stop
- Agent-initiated pauses
Where the API wall used to stop you.
Every team has a handful of portals and tools that resist automation: no API, no integration, just a browser and a grind. Computer Use dissolves that wall in plain language.
The pattern works for any portal. Here is what the same flow looks like over a phone-style channel.
The Northwind Careers portal has no public API. The agent navigates it the same way Lena would (opening the browser, clicking through the form, and pasting the job description) in about 3 minutes instead of 15.
Writing tasks that work.
Computer Use interprets your intent from plain language. A few habits make the difference between a task that completes in one shot and one that needs redirecting.
Name the target app explicitly
Name the target app explicitly
Describe the goal, not every click
Describe the goal, not every click
Use chat for one-offs; use workflows for recurring tasks
Use chat for one-offs; use workflows for recurring tasks
Minimize the Actionist window, do not close it
Minimize the Actionist window, do not close it
Expect short pauses after key actions — this is intentional
Expect short pauses after key actions — this is intentional
Check the AI Process panel if the agent appears stuck
Check the AI Process panel if the agent appears stuck
Enabling screen automation on an agent.
Screen automation is a built-in agent tool. It is scoped per agent so you can restrict which of your agents can operate the screen.
The tool
The tool is named Screen Automation (Actionist) and described as “Click, type, navigate via CUA.” Toggle it on the agent’s Tools tab in the studio.
Scoping
From task to result in four stops.
Your first screen-automation task is waiting.
Grant the permissions, describe the goal, and watch Actionist handle every click. No script, no connector, no extension.
The agent stops and asks before it ever touches a password or payment field.
Built-in safety gates · Live step feed · You stay in control.