Computer Use Agents

​

What is a Computer Use Agent?

You might’ve heard of Claude Computer Use or OpenAI’s Computer Using Agent.

These are powerful tools that can convert natural language into actions on the computer. However, you’d otherwise need to write your own code to convert these actions into Playwright commands.

Stagehand not only handles the execution of Computer Use outputs, but also lets you hot-swap between OpenAI and Anthropic models with one line of code.

​

How to use a Computer Use Agent in Stagehand

Stagehand lets you use Computer Use Agents with one line of code:

​

IMPORTANT! Configure your browser dimensions

Computer Use Agents will often return XY-coordinates to click on the screen, so you’ll need to configure your browser dimensions.

import { Stagehand } from "@browserbasehq/stagehand";

const stagehand = new Stagehand({
	env: "BROWSERBASE",
  	apiKey: process.env.BROWSERBASE_API_KEY /* API key for authentication */,
    projectId: process.env.BROWSERBASE_PROJECT_ID /* Project identifier */,
  
    browserbaseSessionCreateParams: {
      projectId: process.env.BROWSERBASE_PROJECT_ID!,
      browserSettings: {
		blockAds: true,
        viewport: {
          width: 1024,
          height: 768,
        },
      },
  	},
});

await stagehand.init();

​

Direct your Computer Use Agent

// Navigate to a website
await stagehand.page.goto("https://www.google.com");

const agent = stagehand.agent({
	// You can use either OpenAI or Anthropic
	provider: "openai",
	// The model to use (claude-3-7-sonnet-latest for Anthropic)
	model: "computer-use-preview",

	// Customize the system prompt
	instructions: `You are a helpful assistant that can use a web browser.
	Do not ask follow up questions, the user will trust your judgement.`,

	// Customize the API key
	options: {
		apiKey: process.env.OPENAI_API_KEY,
	},
});

// Execute the agent
await agent.execute("Apply for a library card at the San Francisco Public Library");