Computer use
Let sessions and assistants see and control the desktop apps you allow, with a per-app approval, a Stop button and a hard blocklist.
- Topic
- Computer Use
- Available on
- Available onDesktop
- Reviewed
- October 12, 2026
On this page
Computer use lets a coding session or an assistant work with the native apps on its computer the way you do: read an app window, take a screenshot of it, and click, type, press keys, scroll and drag in it. Agents reach it through thirteen tools on Glueprint’s built-in MCP server, so it behaves the same in Claude Code, Codex and ACP agents such as Gemini.
It is off until you turn it on, it runs only in the Glueprint desktop app (the CLI daemon does not offer it), and it only ever acts on the desktop of the computer that runs the session. Reach for it when an app has no command line, file or API an agent could use instead: checking a desktop build, clicking through a native dialog, filling in a form in an app.
Approval is a guardrail, not a sandbox. The approval card, the blocklist and the Stop button give you consent, visibility and a way to halt an agent that follows the rules. They do not contain one that does not. Every agent Glueprint starts already runs as you, with the operating-system access Glueprint holds: on macOS, an agent inherits Glueprint’s Accessibility and Screen Recording access and could script other apps through a shell command without these tools. Turn computer use on only for agents you would trust at your keyboard.
Turn it on
Open Settings > Computer use and switch on Allow computer use. Sessions and assistant wakes that start after you turn it on get the computer-use tools. A session that was already running keeps the tool list it started with, so start a new session. Spec interviews never get these tools.
Turning the switch off ends all access at once. Sessions that still list the tools get a computer_use_disabled answer from then on.
The Readiness group in the same section shows what this computer can do and a Helper row for the background program that does the work. These settings belong to this computer: a session on another of your hosts follows that host’s settings and controls that host’s desktop.
macOS
- Glueprint needs Accessibility (to read app windows and click, type and press keys in them) and Screen Recording (to take screenshots of app windows). Readiness shows whether each is granted, with buttons to request it or open the matching pane of System Settings. The access belongs to Glueprint itself: the helper is Glueprint’s own program, so there is nothing else to approve.
- After you grant Screen Recording, quit and relaunch Glueprint.
- Without Screen Recording, agents still read and act through the accessibility tree; screenshots answer
permission_denied. - Window screenshots need macOS 14 or later. On an older macOS, agents work from the accessibility tree alone.
- Glueprint never shows a macOS permission prompt on an agent’s behalf. Prompts come only from the Readiness buttons.
Windows
- There is nothing to grant: Windows has no privacy permission for reading or capturing another app’s window, so Readiness shows Not needed.
- Glueprint refuses input and accessibility actions to an app with a known higher integrity level than its own. It returns
permission_deniedand sends nothing. For example, this covers an administrator app when Glueprint is not elevated. Apps at the same integrity level pass this check. - Start, Search, the taskbar, widgets and the other shell surfaces are never listed as apps.
Linux
- Desktop accessibility (AT-SPI) must be installed and turned on. On GNOME, run
gsettings set org.gnome.desktop.interface toolkit-accessibility true, then restart the apps you want agents to use. When it is off, reading or acting on an app answerspermission_deniedwith what to install or turn on. - An X11 session gets everything. A Wayland session gets the accessibility half only. Agents read windows, press buttons, run exposed menu actions, set values and insert text. Ctrl+A can select all through AT-SPI when the focused field supports text selection. Screenshots, synthetic keystrokes, scrolling, drags and clicks at a point are unavailable.
- Flatpak apps are not listed, because Glueprint cannot verify which app they are.
- Shells, panels, window managers, desktop portals, password prompts and polkit agents are never listed as apps.
How approval works
The first time a session uses an app, Glueprint shows an approval card on that session titled with the app, for example Computer use: Safari, and the desktop notification for it names the session and Computer use: Safari. The card names the session, the app and its identity (bundle id or executable), and what the agent could do: read the app’s windows, take screenshots of them, and click, type and press keys in them. Listing running apps and checking status never ask.
- Allow — this session may use this app until its access ends.
- Always Allow Safari — allow it now, and add the app to Always allowed apps so no session asks for it again.
- Deny — the agent is told you declined. For the next five minutes that session’s calls for the same app are refused at once, with no new card, and the agent is told not to ask again unless you ask it to.
Only you can answer a computer-use card: in the desktop app, the web portal or the mobile app, wherever your plan shows approvals. Another session cannot answer it: not a parent session that delegated the work, and not a session calling gp_approve_permission or gp_deny_permission. When a delegated child session needs access, its card comes straight to you instead of waiting on the parent.
Calls from the same session share an open card for the same app. If the agent cancels a call or closes its connection before you answer, that call never runs. The card is withdrawn only when the last waiting call is cancelled or disconnected. Other waiting calls keep it open and run if you approve. If an agent stops waiting without cancelling or closing its connection, the card stays open and approval runs the action.
Approval in Settings > Computer use has two choices:
- Ask once per app in each session (the default) — every session and assistant asks before it first uses each app, except apps on the Always allowed apps list.
- Don’t ask: allow all apps — every session and assistant can use any app that is not blocked, without asking. Glueprint asks you to confirm, because this includes assistants that a routine wakes while you are away.
Always allowed apps lists apps that never ask. Add app picks from the apps that are running (available while computer use is on), Remove takes one off, and Always Allow on a card adds one. Under Don’t ask the list is kept for when you switch back.
Some apps cannot be identified by a bundle id or executable. Those always ask, even under Don’t ask, and their access lasts for one session only; the card says so and offers no Always Allow on the desktop.
The tool-permission rules on an agent or assistant do not answer computer-use cards. Only the settings above skip them, and blocked apps are refused under every setting.
Stop, the status bar and assistants
While any session has access to an app or a call in progress, the status bar shows a Computer use chip with the session and the app (or the number of sessions). The tray icon also shows attention while a call is running. Click the chip for each session’s current call and its access, with Stop for one session, Stop all, and a link to the settings.
Stop ends that session’s access to every app and withdraws its open cards. It asks the helper to cancel any call in progress, release keys and restore the clipboard. Glueprint ends the helper only if it does not clean up in time. If ended, a replacement starts on the next call. It also stops the related sessions in the same delegation tree, and a notice says how many. A stopped session asks again before every app for the rest of its life, even under Don’t ask or for always allowed apps, and the agent is told you stopped it and not to retry on its own. Screenshots taken before the Stop can no longer be used for clicks.
Stop all does the same for every session. Unrelated sessions started afterwards begin fresh. A session delegated by a stopped session still asks before using an app. It is also in Settings > Computer use > Active access, which lists every session’s access with how it was granted (Approved, Don’t ask or Always allowed) and a Revoke button for each. Revoking one app does not stop the session: it asks again next time, or under Don’t ask or for an always allowed app it regains access on its next call.
Access also ends when:
- the session stops, completes, fails or is archived, or you interrupt its turn;
- you turn computer use off or reset Glueprint’s settings;
- you change a setting so it no longer covers the access (switching from Don’t ask to asking, or removing an app from Always allowed apps);
- Glueprint quits (access is never saved to disk);
- for an assistant, its wake ends. A regular session keeps its access between turns.
Computer use covers assistants too. The card for an assistant says the access lasts until that wake ends, so an approval in the morning never lets a routine wake drive your desktop overnight. Under Don’t ask, or for always allowed apps, an assistant can use apps without asking even while you are away.
Always blocked
No setting or card can open these apps to an agent. Settings > Computer use > Blocked apps lists the ones for your platform, and the running-apps list marks them as blocked.
- Glueprint itself, so an agent can never click Allow on its own card.
- Password managers: 1Password, Bitwarden, Dashlane, LastPass, NordPass, Proton Pass, the KeePass family, Enpass, RoboForm and Keeper.
- Credential stores: Keychain Access and Passwords on macOS, Passwords and Keys and KDE Wallet on Linux, and the Windows credential prompt.
- System settings and permission prompts: System Settings and the macOS security and authorization prompts; Windows Settings, Control Panel, administrator consent prompts and Windows Security; GNOME, KDE and Xfce settings and polkit authentication agents. An agent cannot grant itself operating-system permissions.
- iPhone Mirroring, which can show the Glueprint mobile app’s approval cards.
- Device simulators: the iOS Simulator and the Android Emulator. Agents drive those with the Mobile Emulator tools instead.
Matching goes by bundle id, executable and app name, and errs on the side of blocking. An app drawn inside another app is not recognized by the list, which is why the browser guard below exists.
Safety rails
- One window at a time. Screenshots capture one window of an app the session may use, never the whole screen.
- Accessibility first. Element clicks and values use the app’s accessibility actions when supported. Those actions need no keyboard focus and send no synthetic input. macOS and Linux can also insert text through accessibility. Windows typing uses synthetic keys, and Windows paste uses the clipboard.
- Focus fences. Synthetic input needs the target window focused (scrolling needs it under the pointer). Glueprint checks before each press and after each release and stops with
window_not_focusedthe moment the window loses focus, saying whether any input may have landed. Keys and buttons pressed are always released, including when a call fails or is stopped. - Your own keys stay yours. On Windows and Linux, synthetic input is refused while you hold a modifier key the action did not press, so your Ctrl or Alt never combines with an agent’s keystroke. On macOS the agent’s events carry their own modifier state, so a key you hold does not leak into them.
- No automatic focus theft. An agent never pulls an app in front of you on its own. When synthetic input finds the window in the background, nothing is sent unless the agent asked for
restore_window: true, which brings the app forward and takes keyboard focus from whatever you are doing; that shows in the transcript. When reading a window,restore_windowun-minimizes and raises it without taking focus on macOS and Windows; on Linux, un-minimizing activates the window. - Other Spaces stay out of reach (macOS). A window on another Space (another desktop, or the Space of a full-screen app such as Glueprint itself) cannot be read or acted on, and Glueprint never sends input to it.
gp_computer_list_windowslists it as on another Space, and reading or acting on it answerswindow_on_other_spacewith nothing sent. Onlygp_computer_activate_app, orrestore_window: trueon that window, brings the app forward the way a Dock click does, which switches your screen to its Space and takes keyboard focus; that shows in the transcript. - System-wide shortcuts are refused. Keys and clicks that act outside the app never go out. On macOS: the app switcher, Spotlight and input-source shortcuts, lock screen, Force Quit, screenshot shortcuts, Mission Control and Spaces, the Dock shortcut, log out, VoiceOver, Zoom and Invert Colors, the keyboard-navigation shortcuts that move focus to the menu bar, Dock, status menus or another app’s window (Control-F1 to F4, F7 and F8), and Command-Option clicks. On Windows: any Windows-key shortcut, Ctrl+Esc, Alt+Tab, Alt+Esc, Ctrl+Alt+Tab, Ctrl+Shift+Esc and Ctrl+Alt+Delete. On Linux: Super shortcuts, Ctrl+Alt shortcuts, Ctrl+Esc, and Alt with Tab, Esc, Space or a function key other than F4.
- The browser guard. Actions are refused in a window that shows the Glueprint web portal, where approval cards appear. Reading is still allowed. An incomplete check also refuses actions in a known browser, an Electron app on Glueprint’s allowlist, or any window with web content. On Linux, actions are also refused in a window showing content from an unrelated app.
- Clipboard. On macOS,
gp_computer_paste_textfirst tries accessibility insertion. If the field does not support it, paste uses the clipboard and needs the window focused. On Windows, paste always uses the clipboard and needs the window focused. Both clipboard routes save and restore your clipboard. A newer copy from you or another app is kept. On macOS the pasted text is marked so clipboard managers skip it, and the clipboard route is refused while the clipboard holds concealed content such as a copied password. On macOS 15.4 and later, Glueprint never triggers the clipboard privacy prompt: when macOS does not allow a silent read of a non-empty clipboard, the paste is refused and the agent types instead. On Windows, the pasted text stays out of clipboard history and the cloud clipboard. On Linux, paste uses AT-SPI EditableText insertion and never uses the clipboard. - Fresh state only. Element numbers are checked against the window before every action, coordinates are accepted only from a screenshot the agent actually received, and a stale reference is refused instead of guessed at.
- What is recorded. Glueprint’s local audit log records which session used which app and tool, with the outcome. It never records typed text, values, keys, coordinates or screenshots. Tool arguments still appear in the session transcript, like any tool call.
Guide for agents
Agents learn this from the tool descriptions. It is here so you know what to expect, and for agents that read the docs.
| Tool | What it does |
|---|---|
gp_computer_status | Whether computer use is on, the approval mode, the platform’s capabilities and permissions, and this session’s access. |
gp_computer_list_apps | Running apps with the value to pass as app (bundle id, executable or name) and the pid; blocked apps are marked. |
gp_computer_list_windows | An app’s windows with window_id, window_index, title, size and state, including windows on another Space. |
gp_computer_activate_app | Bring an app forward with one of its windows, switching to its Space, and launch it first if it is not running (macOS). |
gp_computer_get_app_state | One window’s accessibility tree of numbered elements and, by default, a screenshot. |
gp_computer_click | Click an element or a point; up to three clicks, any button, with modifiers. |
gp_computer_set_value | Set an editable element’s value directly. |
gp_computer_type_text | Type text into the focused element. |
gp_computer_paste_text | Insert text into the focused element, through accessibility where it is supported, else through the clipboard. |
gp_computer_press_key | Press one key or chord, such as Return or CmdOrCtrl+Shift+P. |
gp_computer_scroll | Scroll an element or a point by pages. |
gp_computer_drag | Drag from one element or point to another. |
gp_computer_perform_action | Run an accessibility action an element lists, such as “show menu”. |
- Prefer other routes. Use the shell, files and command-line tools whenever they can do the job. On macOS,
gp_computer_activate_applaunches an app that is not running (passlaunch: falseto only bring a running one forward); elsewhere, open apps with the shell first. Computer use does not open documents or windows. - Pick the target.
gp_computer_list_appsgives theappvalue;pid:Npicks one process when several match. Never try a blocked app.gp_computer_list_windowsgives the window; pass itswindow_id(orwindow_indexwhen it has no id) on every later call. - Loop: read, act once, read. Call
gp_computer_get_app_state, do one action, then read the fresh state the action returns before the next step. - Prefer element numbers and semantic tools. Use
gp_computer_set_value, element clicks andgp_computer_perform_actionbefore coordinates and raw typing. Element numbers come only from the latest state of that window and go stale after navigation, scrolling, focus changes or a delay. - Coordinates come from a delivered screenshot.
xandyare pixels of the latest screenshot of that window that the agent actually received, within the last two minutes. Passwindow_idorwindow_indexand thesnapshot_idprinted with that screenshot. Do not convert for display scaling; Glueprint maps the image’s pixels to the screen. Moving or retitling a window with a native id does not invalidate the capture. Resizing by more than 0.5 points (pixels on Windows) in either dimension does. Without a native id, the title as read by Glueprint must stay unchanged, and each origin coordinate must stay within 0.5 points (pixels on Windows) of its captured value. When a result says its screenshot was not delivered, read the state again before using coordinates. - Check the verification. Every action reports the route it took (accessibility, synthetic input or the clipboard) and a verification. Only verified means Glueprint read the effect back. An UNVERIFIED action may not have worked; confirm it in the returned state before reporting it done.
- State the expected result. Each action from
gp_computer_clicktogp_computer_perform_actioncan takeexpect: {value, element_index, match}. After acting, Glueprint reads that element (the focused one whenelement_indexis left out): its value (an empty value counts, so{"value": "", "match": "equals"}checks a cleared field), else its title, else its description. It re-reads for up to about a second while the app catches up. A password field, and anything inside one, is never read or compared and givesexpectation_unreadable. The verification then says verified, orexpectation_not_metwith the expected and the actual text, orexpectation_unreadable.matchiscontains(the default) orequals; line endings compare alike, and the comparison is otherwise exact. The element number is checked for staleness before anything is sent. This is macOS only for now; elsewhere an action withexpectis refused before it runs. - Line breaks and the caret. A value with line breaks shows in the state as a quoted literal, such as
"first line\nsecond line", keeping its line breaks and surrounding spaces; a text field or document that holds only blank lines shows them too. On macOS the focused text element also shows where typed text lands:Caret: offset N, line LorSelection: offset N, length M, with offsets in UTF-16 units of the value and lines counted from 1. - After a failure, look before retrying. An error can carry a Delivery line:
performedmeans the action ran,maybemeans it may have. Read the state before retrying, and never repeat a timed-out action blindly. Acancellederror means you stopped it (Stop, Revoke or an interrupt): the agent stops and tells you what it was doing. - Background windows. On
window_not_focused, retry once withrestore_window: true, which takes focus from you. If it still fails, ask the user to bring the app forward. - Windows on another Space (macOS). When the app’s window is on another Space (often because Glueprint is full screen), the agent gets
window_on_other_space. It callsgp_computer_activate_app(or retries withrestore_window: true) to switch to that Space, after telling you, since it takes your screen; it does not switch back and forth on every call. If activation does not bring the window forward, the agent getsactivation_failedand asks you to switch to the app. - Screenshots.
gp_computer_get_app_stateincludes an image by default and actions do not; passscreenshot: trueorfalseto change that. An agent that cannot view images can passformat: "path"to get a PNG file under.glueprint/computer-use/in the project. - Text and keys. Prefer
gp_computer_set_valuefor fields.gp_computer_type_texttypes into the focused element;gp_computer_paste_textsuits long text. Ingp_computer_press_key,CmdOrCtrlis Cmd on macOS and Ctrl elsewhere. - Ask before consequences. Never submit, send, buy, delete or change settings unless the user asked for exactly that.
- The first use of an app may wait for you. The call waits until you answer the card; there is no time limit on Glueprint’s side.
Limits and known gaps
- Password fields. A password field’s value, and the text of everything inside it, show as
[redacted]. The field’s own label, title and placeholder stay visible so an agent can find it; a secret typed into one of those would show. Redaction is a guardrail, not a guarantee. - Approval cards shown by other apps. Known portal hosts are refused regardless of the page title. A portal at an unrecognized address can escape the guard if no other evidence matches. Screen-sharing and remote-desktop apps can also show approval cards without being recognized. Do not give computer use to a browser signed in to the Glueprint portal, or to a screen-sharing or remote-desktop app that shows Glueprint.
- New sessions start fresh. Stop and Stop all cover existing sessions and their delegation trees, including children delegated later by a stopped session. A new unrelated session begins fresh, including one an agent starts outside its delegation tree. Under Don’t ask or for always allowed apps, that session gets access without a card. To shut everything off, turn computer use off.
- Clipboard restore timing. After a clipboard paste, Glueprint polls for evidence that the text landed, with a nominal budget of about 1.5 seconds. Slow app readbacks can extend the elapsed time. On Windows, the budget counts polling pauses, not readback time. Glueprint then restores the saved clipboard unless a newer copy replaced it. A slow app can read the restored content instead of the agent’s text. Avoid keeping sensitive content on the clipboard while an agent pastes.
- A helper that has to be ended. If the helper stops answering, Glueprint ends it and starts a fresh one on the next call. A key held at that moment can stay down until you press it, and a paste in progress can leave the agent’s text on your clipboard.
- Keyboard layouts on macOS.
gp_computer_press_keypresses keys by their position on a US keyboard, so on another layout a character shortcut can press a different character (on German QWERTZ, Z and Y swap). Typed text is not affected. Windows uses the target window’s layout. Linux uses the active layout, and text typed as keystrokes cannot include characters that need AltGr (text inserted through accessibility can). - Keystroke typing is capped. Text typed as keystrokes is limited to 8,000 characters per call (counted in UTF-16 units on macOS and Windows); split longer text or use
gp_computer_paste_text. Text inserted through accessibility has no such limit, and any text or value is limited to 1 MiB per call. - Saved screenshots pile up. Screenshots saved with
format: "path"stay under.glueprint/computer-use/in the project (or the assistant’s workspace). Git ignores them, and Glueprint never deletes them; remove them yourself. - Windows. A covered window drawn by the graphics card can capture black; the screenshot then fails and asks for
restore_window: true. A screenshot of an app that has stopped responding can wait until the call times out. A clipboard paste is refused when the clipboard holds data Glueprint could not save and put back. - Linux. Apps written in a scripting language (Python, for example) are identified by the name they report, so the blocklist, which matches programs, may not catch a password manager written that way. Flatpak apps are not listed. On X11, synthetic input looks exactly like yours to every app. Wayland has no screenshots, synthetic keystrokes, scrolling, drags or clicks at a point. Ctrl+A can select all through AT-SPI when the focused field supports text selection.
- Covered windows on Linux. Without a compositor, a screenshot is refused while another window covers the app.
- Input held by another app on Linux. Synthetic input is refused while another app holds the keyboard or pointer, such as an open menu, a screen locker or a password dialog.
- Modifier keys on Linux. Synthetic input is refused while you hold a modifier key the action did not press (
window_not_focused). Keystroke typing is refused under Shift Lock or Sticky Keys. - Content from another app on Linux. Actions are refused in a window that shows content from an unrelated app; observation omits that content.
- Shortcuts on Linux. Supported Ctrl and Alt letter shortcuts work under non-Latin layouts. System-wide shortcuts, including Super shortcuts, are still refused.
- Portal and mobile cards. The web portal and the mobile app show the computer-use card in the generic style, with its tool name
gp_computer_use. Both offer Always even for an app Glueprint cannot identify. That approval grants this session only. - Agents that give up waiting. Some agents stop waiting after a fixed time. Cancelling the call or closing its connection before approval prevents it from running. The card is withdrawn only when no waiting calls remain. If the agent stops waiting without cancelling or closing its connection, the card stays open and approval runs the action.
- Other Spaces on macOS. Computer use reaches a window on another Space only by switching your screen to it; there is no background control. If When switching to an application, switch to a Space with open windows for the application is off in System Settings > Desktop & Dock, activation may not reach the window and the agent gets
activation_failed. A window on another Space is listed only when macOS reports its Space; otherwiserestore_window: truestill brings the app forward. - Not included. Full-screen screenshots, launching or activating apps on Windows and Linux, opening documents, per-agent or per-project approval rules, the CLI daemon, and controlling another computer’s desktop.
Troubleshooting
The agent sees an error code and recovery steps. What each code means, and what to do:
| Code | What it means | What to do |
|---|---|---|
computer_use_disabled | Computer use is off, or the session started before you turned it on. | Turn it on in Settings > Computer use, then start a new session. |
computer_use_unavailable | This host cannot run computer use: the CLI daemon, an unsupported platform, or a spec interview. | Use the desktop app; for a spec interview, start a regular session. |
permission_denied | macOS lacks Accessibility or Screen Recording, a Windows target has a known higher integrity level than Glueprint, or Linux accessibility is off. | gp_computer_status shows which permission is missing. Grant it (relaunch after Screen Recording), run the Windows target at the same or a lower integrity level, or turn accessibility on. |
app_not_found | No running app matches the app value, or (for gp_computer_activate_app) no installed app does either. | Use the exact value from gp_computer_list_apps. To start an app, use gp_computer_activate_app with its bundle id or name on macOS, or open it yourself; for a website, target the browser. |
app_ambiguous | Several running apps match. | Use pid:N or the exact bundle id or executable. |
app_blocked | The app is blocked or the window shows the Glueprint portal. An incomplete check can also refuse a known browser, an Electron app on Glueprint’s allowlist, or any window with web content. On Linux, a window showing an unrelated app’s content is also refused. | Do this step yourself, or switch that browser window to another page. |
grant_denied | You denied the card (with a five-minute pause on new cards), or the session was not active when it needed to ask. | Tell the agent how to proceed; it will not ask again on its own. |
cancelled | You pressed Stop, revoked the access or interrupted the turn. | The agent tells you what it was doing; say whether to continue. |
window_not_found | The window is not one of the app’s windows, or the app has none. | Run gp_computer_list_windows; open a window if there is none, or bring the app forward with gp_computer_activate_app. |
window_on_other_space | The window is on another Space (macOS), for example because Glueprint is full screen. Nothing was sent. | Let the agent call gp_computer_activate_app or retry with restore_window: true, which switches your screen to that Space; or switch to the app yourself. |
activation_failed | Glueprint asked macOS to bring the app (or launch it) forward, but the requested window did not reach the current Space or stayed behind another of the app’s windows, a launched app never became the active app or opened its window on another Space, or the launch failed. | Switch to the app yourself, or turn on When switching to an application, switch to a Space with open windows for the application in System Settings > Desktop & Dock. |
window_not_focused | Synthetic input needed the window focused. On Linux, a held modifier or another app holding the keyboard can also cause this. Nothing was sent, unless the message says input may have landed. | Retry once with restore_window: true, bring the app forward, release held modifiers or dismiss the other app’s keyboard grab. |
window_stale | The capture is missing, expired or evicted, or snapshot_id does not match it. Resizing beyond 0.5 points (pixels on Windows) in either dimension also invalidates it. Only a window without a native id is invalidated by a changed title or an origin coordinate moving more than 0.5 points (pixels on Windows). | Read the state with screenshot: true and use coordinates from that image. |
element_not_found | An element number is stale. | Read the state again and use a number from the fresh tree. |
element_not_clickable | The element has no usable frame. | Pick a nearby parent or child element. |
action_not_supported | The element does not offer that action, a macOS shortcut acts outside the app, the clipboard route is unavailable, or you were holding a modifier key (Windows). | Use an action the element lists, gp_computer_set_value, or gp_computer_type_text. |
value_not_settable | The element’s value cannot be set directly. | Pick a settable text element, or focus it and type. |
unsupported_capability | This platform or session cannot do it, such as screenshots on Wayland or before macOS 14, or gp_computer_activate_app on Windows and Linux. | Check gp_computer_status and use a supported alternative. |
screenshot_failed | The window could not be captured; the tree is still returned. | Use screenshot: false, or restore_window: true for a covered or minimized window. |
accessibility_error | The app did not answer accessibility requests (it may be busy or hung), or the helper could not start. | Retry later; check gp_computer_status. |
action_timeout | The call reached a time limit. | Read the state before retrying. A call still in progress gets a cancellation request. Glueprint ends the helper only if cleanup takes too long. If ended, a replacement starts on the next call. |
invalid_argument | The tool arguments were wrong (for example coordinates without a window or snapshot_id), or a Windows or Linux shortcut acts outside the app. | Fix the arguments as the message says. |
provider_incompatible | The helper does not match this Glueprint build. | Restart Glueprint to finish an update. |
protocol_error | An internal error, or the reply was lost and the call may have run. | Read the state once; if it happens again, report it. |
The tools do not appear. Turn computer use on, then start a new session: the tool list is fixed when a session starts. Check that you are in the desktop app, not a CLI daemon host.
No card appears, but the agent got grant_denied. You denied that app in the last five minutes, or the session was not active when it needed to ask. Ask the agent to try again after the pause, or press Stop on the chip, which ends the pause and makes the session ask again.
No card appears and the agent just works. Don’t ask is on, or the app is on Always allowed apps. Check Settings > Computer use.
Screenshots fail on macOS. Grant Screen Recording in Settings > Computer use > Readiness, then quit and relaunch Glueprint.
The status bar chip will not go away. It stays while any session has access. Press Stop all, or revoke the access under Settings > Computer use > Active access.