Computer Use

Foxl can use the apps on your computer for you. It looks at an app's window, presses its buttons, types and scrolls, and tells you what it did. This is called computer use.

It works on a Mac, on Windows, and on Linux, in an X11 session and on GNOME in a Wayland session (the Ubuntu and Fedora default). It is on from the start.

1. Give it what your system needsCopy Anchor Link

Allow computer use in Settings > Computer use is on from the start; turn it off there to stop Foxl using apps. Then give it what your system needs:

MacCopy Anchor Link

1. Under macOS permissions, press Open settings. System Settings opens with a small panel attached to its window: drag Foxl Computer Use from the panel into the list above it, or turn on its switch if it is already in the list. The panel then moves on to the other permission.
2. When both are on, Foxl comes back to the front and both rows say Granted. You do not need to quit Foxl.

macOS asks for two permissions because Foxl needs to see the app's window (Screen Recording) and to click and type in it (Accessibility). Both go to Foxl Computer Use, the part of Foxl that does this. If you allowed Foxl or node-arm64 in an earlier version, allow Foxl Computer Use once: those permissions do not carry over.

WindowsCopy Anchor Link

Nothing else to allow: Windows needs no extra permission for this. Under Windows desktop, App windows and App controls say whether Foxl can use apps right now. They need Windows unlocked, with Foxl running in the same desktop session.

LinuxCopy Anchor Link

Linux has no per-app permission for this. Under Linux desktop, Settings checks two things:

• Desktop session: X11 or Wayland. On Wayland Foxl reads and presses an app's controls and types into its fields through accessibility, without screenshots; clicking at a point and pressing keys are not available there. It is checked on GNOME; other Wayland desktops are not yet.
• Accessibility bus (AT-SPI): how Foxl presses buttons and types without moving your pointer. Every Linux desktop comes with it.
• Screenshots (Wayland only): a screenshot on Wayland shows your whole screen, other windows and notifications included, because Wayland gives an app no single window to take. Foxl takes one only after you allow screenshots once, with Allow here or Open on the card in the chat, and never while a terminal, a password manager or a window that shows passwords is open (a minimized one counts as open). On a desktop with an older screenshot portal (xdg-desktop-portal before 1.15, as on Ubuntu 22.04), the desktop asks you to share every screenshot, so Foxl takes none there and reads and presses controls instead.

On X11, without a compositor running, Foxl can only take a screenshot of a window that nothing covers.

2. Pick a model that can seeCopy Anchor Link

Each step shows the model a picture of the app, so the model must be able to read images. If the model you picked cannot, Foxl tells you so in the chat, and you can switch models.

Claude models do this best. Claude Sonnet 5.5 on Amazon Bedrock did the example below in about a minute, and Claude Haiku 5.5 on OpenRouter did it too. Free OpenRouter models can work, but they are often busy, and each step uses one of your free requests for the day.

3. Ask FoxlCopy Anchor Link

Start a new chat and say what you want done, and in which app. For example:

• "Use Calculator to add an 18% tip to $186.40 and split it four ways."
• "Type a short thank-you note into the document open in TextEdit."

Foxl uses the app without asking first. To be asked the first time each chat uses an app, turn on Settings > Safety > Ask before Foxl uses an app for the first time. Foxl then asks like this:

• Allow: this chat may use the app.
• Always allow: Foxl stops asking for this app. You can remove it later under Always-allowed apps in Settings > Computer use, or in Settings > Safety, which lists the same apps right below Ask before Foxl uses an app for the first time.
• Deny: Foxl does not use the app.

With that switch on, if the chat runs from the iPhone, iPad or Android app, or from app.foxl.ai, the step shows there with the same answers as on the computer, and says so once it was answered on the computer. To let only Foxl on your computer allow it, turn on Only devices set up at this computer can allow in Settings > Safety: the step then shows on a phone as waiting for approval on your computer, with Deny, a phone you set up can allow it as well (see Allow from your phone), and app.foxl.ai in a wider browser window does not show it. A computer-use step does not wait for later: Deny, or no answer in time, means it does not run.

Then you can watch it work. A small pointer of Foxl's own moves to each button Foxl presses. On a Mac it is gray while you are in that window, and wears its chat's own soft color while you are elsewhere, so two chats using apps at once have two colors:

To stop at any time, press Stop on the Foxl is using bar. The iPhone, iPad and Android apps and app.foxl.ai show the same bar with Stop while your computer uses an app, and when a schedule, a trigger, a channel thread or the Heartbeat starts using one while you are away from your Mac, your phone gets a Foxl is using notification (under Scheduled tasks); a Windows or Linux computer cannot tell whether you are there, so your phone is always told.

Allow from your phoneCopy Anchor Link

With Only devices set up at this computer can allow off (the default), any phone or browser signed in to your account can allow a step, and a saved sign-in Foxl asks about before typing it, so someone who has your signed-in phone, or your session, can too. Turn it on in Settings > Safety to keep that to the computer and the phones you set up here; it can be turned off again only in Foxl on the computer. With it on, set a phone up once and it answers a computer-use step with the same Deny, Allow and Always allow as the computer. This works in the iPhone, iPad and Android apps (Android 9 or later) and in app.foxl.ai in a phone's browser, with Foxl 0.9.1 or later on the computer; until the computer has it, the phone says to update Foxl there. A step that asks before it deletes, sends or confirms (Settings > Safety) needs Foxl 0.10.3 or later on both the computer and the phone; with an older Foxl on the computer, only the computer can allow it, and the phone says to update Foxl there.

1. On the phone, tap Approve from this phone under a step waiting for approval, or, while your computer is connected, Set up beside Approve computer use from this phone in Settings > Desktop. The phone shows a six-digit code.
2. In Foxl on your computer, a card asks Let a phone approve computer use? and shows a code too. Press Accept only if it is the same code. The request waits five minutes.
3. The phone says it can now approve computer use.

From then on, in the iPhone, iPad and Android apps, every Allow and Always allow first asks for Face ID, Touch ID, a fingerprint, or the phone's passcode or screen lock; if the phone cannot confirm it is you, nothing is sent. A phone's browser asks for nothing first, so anyone using that browser can approve, and the card on your computer says so before you accept it. Deny works from any phone without setting it up.

The phones you accepted are listed under Devices that can approve in Settings > Computer use on the computer, and removing one there stops it at once. Stop in the phone's Settings > Desktop removes that phone too. A phone is accepted in Foxl on your computer, not from the phone; what that keeps out, and what it does not, is under Security.

What works on each systemCopy Anchor Link

MacCopy Anchor Link

• Works: apps built with Apple's own controls or SwiftUI (Calculator and TextEdit, for example), web pages shown inside apps, and apps built on Electron (Slack, VS Code, Notion and Discord, for example), in the background and without moving your pointer. Foxl sees only that app's window, reads it as text, presses buttons, fills fields, picks options, checks boxes, opens menus and moves sliders. Where an app describes nothing at a spot (a game, a canvas, or an app built with Tk, as many Python tools are), Foxl clicks, right-clicks, drags and scrolls there with your pointer for a moment, as described under Good to know.
• Password fields: Foxl types into them without reading the password, and takes no screenshot while one has the keyboard focus. A Mac app's own password field cannot be checked afterwards, and Foxl says so; one on a web page is checked by how many characters it holds.
• Does not work: the pages of Chrome and other Chromium browsers (use the Chrome extension), dragging and dropping content an app describes, a double-click inside a web page, and the scroll wheel in some web pages shown inside apps. In an Electron app, the first text typed right after a click into a field can be lost, so check what Foxl typed there.

WindowsCopy Anchor Link

• Works: Windows apps that describe their windows (Calculator and Notepad, for example), in the background and without moving your pointer, and the pages of Microsoft Edge once its window is showing. Foxl sees only that app's window, clicks, opens context menus, selects text, moves sliders, scrolls and types. A shortcut that needs the keyboard focus brings the app forward for a moment.
• Password fields: Foxl never reads or types into them.
• Does not work: reading a window as text (that is Mac and Linux only), a page in Edge while another window fully covers it (it describes nothing then, so bring it into view), the pages of many apps built on Electron, minimized, protected and administrator windows, and shortcuts with the Windows key. A page that just opened describes its controls a second or two later.

Linux with X11Copy Anchor Link

• Works: apps built with GTK (GNOME and Xfce apps, for example), and the Chromium and Electron apps Foxl opens itself, which it starts with their accessibility turned on. Foxl sees only that app's window, reads it as text and presses its controls, and types into the fields of GTK apps. A key, or text with a new line, brings the app forward for a moment.
• Password fields: Foxl types into them without reading the password, and checks them by how many characters they hold.
• Does not work: a Chromium or Electron app that was already open before Foxl opened it (quit it, then let Foxl open it), many apps built with Qt, dragging and dropping content, a middle click, and, without a compositor, a screenshot of a window that something covers.

Linux with Wayland (GNOME)Copy Anchor Link

• Works: the apps Foxl opens, built with GTK, Qt, Chromium or Electron, which it starts with their accessibility turned on. Foxl reads them as text, presses their controls and types into their fields; in Chromium and Electron apps it types with your desktop's remote control, which your desktop asks you to allow once, and only while that app's window is in front.
• Screenshots: the whole screen, only after you allow screenshots once, and never while a terminal, a password manager or a window that shows passwords is open.
• Does not work: clicking, dragging or scrolling at a point and pressing keys (Wayland lets no app do that in another app's window), Wayland desktops other than GNOME (not checked yet), and a desktop whose screenshot service asks about every screenshot (Foxl then takes none).

Good to knowCopy Anchor Link

• Your mouse stays yours. Wherever an app describes its controls, Foxl works without your pointer, so you can keep working. Some key presses and shortcuts bring the app forward for a moment, and your window is put back right after. On Windows and Linux, shortcuts use Ctrl.
• On a Mac, Foxl borrows your pointer only where an app gives it no other way. When nothing at a spot is described (a game, a canvas), or the app is built with Tk, whose buttons ignore a press through accessibility, the bar says Foxl is using your pointer in that app, and Foxl clicks, drags or scrolls there with your pointer, only while you are not using the mouse or the keyboard, and never in a password field. It stops and hands the pointer back the moment you move the mouse or press a key (a drag you take over is let go where it began, and your pointer stays where you moved it), and otherwise puts the pointer and the app you were in back right after. Turn it off in Settings > Safety > Use your pointer when an app can only be used that way.
• On a Mac, on Windows and on X11, Foxl only looks at that app's window, never your whole screen. On Wayland a screenshot is the whole screen, and Foxl takes one only as described under Linux.
• Typing a card number, a Korean resident registration number or a US Social Security number asks first, even in an app you always allow: the prompt names what it found (a card or SSN by its last four digits only), Allow covers that one step, and there is no Always allow for it.
• A run that someone else's mail or message started asks before each step that acts in an app, even in an app you always allow: a click, typing or a key press in a trigger's run or a check-in's waits for your OK, unless you choose Full access on that run's row in Settings > Safety (see By where a run comes from). Opening the app and looking at it do not ask.
• Some apps are never used:
  • On a Mac: Terminal and other terminal apps, Script Editor, Automator, Shortcuts, System Settings, Keychain Access, Passwords, and Foxl itself.
  • On Windows: terminals and script runners (Command Prompt, PowerShell, Windows Terminal, WSL and the like), File Explorer, Settings and Control Panel, Task Manager and Registry Editor, password managers, apps running as administrator, and Foxl itself. Foxl also never reads or types into a password field.
  • On Linux: terminals, run dialogs, the system settings, password managers, and Foxl itself.
• It can read an app's controls as text. On a Mac and on Linux, Foxl can read the window as a list of its buttons, fields and menus and press, fill or pick one by name, checking each result, without a screenshot for every step. It refuses a control that changed since it read the window instead of guessing (one row's Delete is never taken for the next row's). It types into a password field without reading the password back or repeating it in its results (see What works on each system). The first time Foxl uses an app on your computer, a card asks once: Ask before Foxl deletes, sends, confirms or pays in your apps? Choose Ask me first or Just do it, on your computer or your phone. Until you answer, pressing something that deletes, sends, confirms or pays runs like any other step. Change your answer later in Settings > Safety > Ask before deleting, sending, confirming or paying.
• Apps that describe their windows work best. Foxl works through each system's accessibility interface. Mac apps like Calculator and TextEdit do well. On a Mac, Foxl can also press the buttons and other controls inside an app built on Electron (Slack, VS Code, Notion and Discord, for example), even in a window that was never in front: while it operates one, it asks the app to describe its whole window, which adds two to three seconds to the first step of each turn, and turns that off again when the turn ends or Foxl quits. An app that VoiceOver or another assistive app is already using is left as it is. Chrome and the other Chromium browsers, and on Windows many apps built on Electron, still describe little of their pages, so there Foxl can do less; on Windows a page in Microsoft Edge works once its window is showing. On Linux, apps built with GTK (GNOME and Xfce apps, for example) work. Apps built with Qt, Chromium or Electron describe their windows only when started with their accessibility turned on: Foxl starts the Chromium and Electron apps it opens that way, and in a Wayland session Qt apps too, so it can read and press their controls. An app that was already open keeps how it was started, and Foxl refuses to click in one that describes nothing and says so. In a Wayland session Foxl types into Chromium and Electron apps with your desktop's remote control, which your desktop asks you to allow once, and only while that app's window is in front. For websites, use the Chrome extension.

More detail, with a video of a real run and the models we tried: Computer use on Mac.