Skip to content
The AI Agent Factory

7.5 Prefer connectors to browsers, and browsers to screen control

Approved by
Panaversity ·
7 min read

In everyday life. To pay a bill, you can use the company's app, log in on its website, or hand someone your unlocked laptop.

An AI Worker can reach another system, such as an accounting system, in three ways.

A connector is a link to one system that offers named actions through the system's own interface. Each named action is a tool. Reading and writing are separate tools. Each call to a tool can be approved and recorded one by one.

A browser uses web pages the way a person does. It can reach any site. If it is signed in, its session carries the person's full rights: everything that person may do on the site. And its clicks are harder to review than named tool calls.

Screen control uses desktop apps by looking at the screen, moving the pointer and typing. Anthropic calls it computer use. It reaches anything visible, including windows you did not mean to show.

Order them by how narrow the reach is, how clear the record is, and how the permissions are enforced. Connectors usually come first, but not always. A connector with every tool switched on can reach more than a browser signed in to an account with few rights. So use the narrowest well-controlled path. That is usually a connector, then the browser. Use screen control only for a trusted app that has no other way in, with sensitive windows closed.

Browser and screen actions skip a connector's own limits, such as which tools are on and which ask first. The application still checks the rights of the signed-in account. But those are a person's rights, usually wider than the worker needs. So rebuild the limits around them yourself, with approval for each app or site and a record of what was done.

The title reads "Choose the narrowest path with effective controls." The line below it reads "Usually: connector first, browser next, screen control last." A table compares three routes in four rows: reach, action record, permissions, and how to use it well. Connector, usually first: it reaches the named actions the connector exposes. Its action record is structured tool calls, and logging and approval depend on setup. Its permissions are tool restrictions plus the connected account's rights. To use it well, enable only the tools and data the task needs. Browser, when a connector cannot do the job: it reaches allowed sites and accessible signed-in sessions. Its record is page interactions and clicks, so inspect the available logs. Account and browser controls apply, but connector limits do not. To use it well, restrict sites, use a limited account and require needed approvals. Screen control, when no narrower route works: it reaches accessible apps, windows and desktop content. Its record is clicks, typing and screen observations, which are often harder to audit. Account, operating-system and app controls apply, but connector limits do not. To use it well, use trusted apps, close sensitive windows and record actions. A gold note says to compare the actual setup, because a broadly privileged connector can reach more than a tightly restricted browser account. A dark bar reads: check reach, action records and enforceable permissions before choosing.

Figure 7.5. Three ways to reach a system, compared by reach, record and permissions.

When the worker picks the route. In a conversation that routes itself (1.2), you may not choose the route at all. You ask for a result. The worker may use a connector, open a browser when the connector cannot do a step, and then reach for screen control. Each route has its own limits. A brief can narrow the envelope, never widen it (7.1). But a line in a brief does not switch on a control the product lacks. So before the task, check every route the worker could reach against the envelope, not only the one you expect it to use. Turn off the routes the job does not need, and set the rest to ask before any action that changes or sends something. A connector that can send mail is too much for a draft-only job, unless sending is blocked.

Check yourself

Question 1 / 9

0 answered

In what order should a worker usually reach a system?

Owner
Panaversity