Computer Use: Let Claude Control Your Screen
Every workflow you have tried so far relies on Claude working inside its own environment: reading files, running commands, browsing the web. Computer Use removes that boundary. With it, Claude can open your applications, move the cursor, click buttons, type into fields, and navigate your desktop the same way you would.
This is the most tangible Digital FTE experience in the course. You will watch Claude take screenshots of your screen, figure out where to click, and execute multi-step tasks across native applications. It is impressive. It is also powerful enough to warrant serious thought about safety.
Requirements
Before you begin, confirm you have all three:
| Requirement | Details |
|---|---|
| Hardware | Mac with Apple Silicon (M1, M2, M3, M4, or later) |
| Plan | Claude Pro or Max subscription (Team and Enterprise do not have access yet) |
| App | Claude Desktop, latest version from claude.com/download |
Computer Use is macOS-only as of March 2026. Windows support is listed as "coming soon." If you are on Windows or Linux, read through this lesson for the concepts and safety framework. The permission tiers and tool hierarchy apply to every platform once support arrives.
Step 1: Enable Computer Use
- Open the Claude Desktop app
- Go to Settings > Desktop app > General
- Find the Computer use toggle and turn it on
The toggle will not take effect until you grant two macOS system permissions. The Settings page shows the status of each one.
Step 2: Grant macOS Permissions
Claude needs two permissions to interact with your screen:
Accessibility (click, type, scroll)
This permission lets Claude simulate keyboard and mouse input.
- When prompted, click the badge in Claude Settings to open System Settings > Privacy & Security > Accessibility
- Find Claude in the list
- Toggle it on
- You may need to enter your Mac password or use Touch ID to confirm
Screen Recording (see your screen)
This permission lets Claude take screenshots to understand what is on screen.
- Click the badge in Claude Settings to open System Settings > Privacy & Security > Screen Recording
- Find Claude in the list
- Toggle it on
- Restart Claude Desktop if prompted
Both permissions must be active. If either is denied, Computer Use stays disabled even if the toggle is on. You can check the status in Settings > Desktop app > General at any time.
Step 3: Your First Screen Task
Open a new Cowork session and type:
"Open the Calculator app, multiply 847 by 23, and tell me the result."
Watch what happens. Claude will:
- Take a screenshot of your current desktop to understand the layout
- Open Calculator (using Spotlight or the Applications folder)
- Click the number buttons to enter 847 x 23
- Read the result from another screenshot
- Report the answer back in the chat
You will see Claude's actions appear in the session as it works. Each screenshot and click is logged so you can follow along.
Claude did not use an API. It did not run a shell command. It looked at your screen, decided where to click, clicked there, and read what changed. This is the same process you use when operating an unfamiliar app for the first time: look, click, observe, repeat.
How Claude Chooses Its Tools
Computer Use is powerful but slow. Claude follows a priority hierarchy, always choosing the most precise tool available:
| Priority | Tool | Speed | When Claude uses it |
|---|---|---|---|
| 1 | Connectors (Gmail, Slack, Google Drive) | Fastest | Service has a direct integration |
| 2 | Bash (shell commands) | Fast | Task can be done via command line |
| 3 | Browser (Claude in Chrome) | Medium | Task requires web interaction |
| 4 | Computer Use (screen control) | Slowest | Nothing else can reach the app |
This means if you ask Claude to send an email, it will use the Gmail connector, not open the Mail app and click "Compose." Computer Use is the fallback for applications that have no connector, no CLI, and no browser interface: native desktop apps, hardware control panels, the iOS Simulator, proprietary tools without an API.
The rule of thumb: if there is a faster, more reliable path, Claude takes it. Computer Use handles everything else.
Per-App Permission Tiers
The first time Claude needs to interact with an application, a permission prompt appears in your session. You can click Allow for this session or Deny.
But not all apps get the same level of access. Claude assigns a fixed tier based on the app category:
| Tier | What Claude can do | Which apps |
|---|---|---|
| View only | See the app in screenshots, nothing else | Browsers, trading platforms |
| Click only | Click and scroll, but cannot type or use keyboard shortcuts | Terminals, IDEs |
| Full control | Click, type, drag, and use keyboard shortcuts | Everything else |
These tiers are fixed by app category. You cannot upgrade a browser from view-only to full control, and you cannot downgrade a Notes app from full control to view-only. The design is intentional: browsers are view-only because Claude in Chrome handles web tasks more precisely. Terminals are click-only because Bash handles commands more reliably.
Applications like Terminal, Finder, and System Settings show an extra warning when Claude requests access. These apps can affect your entire system, so the prompt makes the scope of approval explicit.
Managing App Access
You have two controls in Settings > Desktop app > General:
Denied Apps
Add applications here to block Claude from accessing them entirely, without even prompting. Good candidates for your denied list:
- Password managers (1Password, Bitwarden)
- Banking apps
- Investment platforms
- Messaging apps with sensitive conversations
- Health or medical applications
Claude may still indirectly affect a denied app (for example, clicking a link that opens in a blocked browser), but it cannot interact with the denied app directly.
Window Hiding
While Claude is working, your other windows are hidden so it interacts only with the approved app. When Claude finishes, hidden windows are restored. You can turn this behavior off in Settings if you prefer.
Safety: What to Allow, What to Restrict
Computer Use runs on your actual desktop. Unlike the sandboxed Bash tool, there is no safety net. Anything visible on your screen is accessible to Claude through screenshots.
Claude is trained to avoid
- Stock trading or investment transactions
- Inputting sensitive data (passwords, credit card numbers)
- Gathering or scraping facial images
You should avoid delegating
| Category | Why |
|---|---|
| Financial accounts | Transactions are irreversible; one wrong click moves real money |
| Legal documents | Signing or submitting has binding consequences |
| Medical records | HIPAA and privacy implications |
| Apps with others' personal data | You are responsible for what Claude sees and does |
Before you enable Computer Use
Build your safety checklist:
- Close sensitive apps. If your banking app is open, Claude can see it in screenshots even if you deny direct access.
- Build your denied-apps list first. Decide what is off-limits before you start, not after Claude asks for access.
- Use session-scoped approvals. Permissions last for the current session only (or 30 minutes in Dispatch sessions). You approve fresh each time.
- Stay nearby for high-stakes tasks. You can stop Claude at any point by clicking the stop button or typing a correction.
Computer Use is a research preview. Anthropic states that safeguards "aren't perfect" and Claude may occasionally act outside the boundaries you expect. Treat it like a capable but new employee: supervise closely at first, expand trust gradually.
Try With AI
1. A Multi-Step Screen Task
Open Cowork and try this prompt:
"Open Safari, go to wikipedia.org, search for 'Apollo 11 moon landing', find the launch date in the article, and paste it into a new TextEdit document. Save the document as 'apollo-research.txt' on my Desktop."
What you are learning: Multi-app coordination through screen control. Claude must navigate Safari (view-only tier, so it reads but does not type in the URL bar directly), switch to TextEdit (full control), and save a file. Notice how Claude chains screenshots and clicks across two applications to complete a task that spans multiple tools.
2. Evaluate the Tool Hierarchy
Open Cowork and try this prompt:
"I want to check my most recent email subject line. You have the Gmail connector available. Walk me through which tool you would use and why. Then actually do it."
What you are learning: Tool selection logic. Claude should use the Gmail connector (priority 1) rather than opening a mail app via Computer Use (priority 4). If Claude reaches for screen control when a faster tool exists, that is a sign the task was ambiguous or the connector is not configured. This exercise builds your intuition for when Computer Use is the right choice vs. when it is overkill.
3. Build Your Safety Boundary
Open Cowork and try this prompt:
"I am about to enable Computer Use for the first time. Help me build a denied-apps list for my Mac. Ask me what categories of work I do (finance, health, legal, personal communication, development) and recommend which apps to deny based on my answers. For each recommendation, explain the specific risk if I left that app accessible."
What you are learning: Risk assessment for autonomous screen control. The denied-apps list is your primary safety boundary. By articulating why each app should be denied, you build a mental model of what "too much access" looks like. This skill transfers to any AI system that operates on your behalf, not just Computer Use.
What Happens Under the Hood
When Computer Use is active, Claude follows a loop:
- Screenshot: capture the current screen state
- Analyze: identify UI elements, text, and layout
- Decide: determine the next action (click, type, scroll, open app)
- Act: execute the action
- Screenshot again: verify the action produced the expected result
- Repeat until the task is complete or Claude needs your input
Each cycle takes a few seconds. Complex tasks may require dozens of cycles, which is why Computer Use is slower than connectors or Bash. The tradeoff is generality: Computer Use works with any app that has a visual interface, even apps Claude has never seen before.
When to Use Computer Use (and When Not To)
| Use Computer Use for | Use something else for |
|---|---|
| Native desktop apps with no API (Keynote, Preview, GarageBand) | Email, calendar, Slack (use connectors) |
| Testing in the iOS Simulator | Web browsing (use Claude in Chrome) |
| Proprietary enterprise tools with only a GUI | File operations (use Bash) |
| Filling out forms in apps that lack automation | Anything with a CLI or API |
| One-off GUI tasks you would normally do by hand | Recurring tasks (set up a connector or script instead) |
The best use of Computer Use is for tasks that live in the gap between "I could do this manually" and "there is no way to automate this." If an app has an API, a CLI, or a connector, those paths are faster and more reliable. Computer Use fills the gap for everything else.
What is Next
You have now seen Claude work inside its environment (files, commands, browser) and outside it (your screen, your apps). The next lessons explore how to organize this growing set of capabilities into persistent projects with memory, scheduling, and structured workflows.