Skip to main content

Observe

See how to use observe() to discover actionable elements and analyze web page structure

​
Method Signatures

  • TypeScript
// String instruction only
await stagehand.observe(instruction: string): Promise<Action[]>

// String instruction with options
await stagehand.observe(instruction: string, options: ObserveOptions): Promise<Action[]>
ObserveOptions Interface:
interface ObserveOptions {
  model?: ModelConfiguration;
  timeout?: number;
  selector?: string;
  page?: PlaywrightPage | PuppeteerPage | PatchrightPage | Page;
}

// ModelConfiguration can be either a string or an object
type ModelConfiguration =
  | string  // Format: "provider/model" (e.g., "openai/gpt-4o", "anthropic/claude-3-5-sonnet-20241022")
  | {
      modelName: string;  // The model name
      apiKey?: string;    // Optional: API key override
      baseURL?: string;   // Optional: Base URL override
      // Additional provider-specific options
    }

​
Parameters

​
instruction
string
required
Natural language description of elements or actions to discover. If not provided, defaults to finding all interactive elements on the page.
​
model
ModelConfiguration
Configure the AI model to use for this observation. Can be either:
  • A string in the format "provider/model" (e.g., "openai/gpt-4o", "anthropic/claude-3-5-sonnet-20241022")
  • An object with detailed configuration
​
timeout
number
Maximum time in milliseconds to wait for the observation to complete. Default varies by configuration.
​
selector
string
Optional XPath selector to focus the observation on a specific part of the page. Useful for narrowing down the search area.
​
page
PlaywrightPage | PuppeteerPage | PatchrightPage | Page
Optional: Specify which page to perform the observation on. Supports multiple browser automation libraries:
  • Playwright: Native Playwright Page objects
  • Puppeteer: Puppeteer Page objects
  • Patchright: Patchright Page objects
  • Stagehand Page: Stagehand’s wrapped Page object
If not specified, defaults to the current “active” page in your Stagehand instance.

​
Returns Promise<Action[]>

Array of discovered actionable elements, ordered by relevance.
​
selector
string
XPath selector that precisely locates the element on the page.
​
description
string
Human-readable description of the element and its purpose.
​
method
string
Suggested interaction method for the element (e.g., "click", "fill", "type").
​
arguments
string[]
Additional parameters for the suggested action, if applicable.
Action Interface: 
interface Action {
  selector: string;        // XPath selector to locate element
  description: string;     // Human-readable description
  method?: string;         // Suggested action method
  arguments?: string[];    // Additional action parameters
}
Example Response: 
[
  {
    "selector": "/html/body/div[1]/header/nav/button[1]",
    "description": "Login button in the navigation bar",
    "method": "click",
    "arguments": []
  },
  {
    "selector": "/html/body/main/form/input[1]",
    "description": "Email input field in the login form",
    "method": "fill",
    "arguments": []
  }
]

​
Built-in Support

Iframe and Shadow DOM interactions are supported out of the box. Stagehand automatically handles iframe traversal and shadow DOM elements without requiring additional configuration or flags.

​
Code Examples

  • Basic Usage
  • Custom Model
  • Scoped
  • Multi-Page
  • Filter Results
import { Stagehand } from "@browserbasehq/stagehand";

// Initialize with Browserbase (API key and project ID from environment variables)
// Set BROWSERBASE_API_KEY and BROWSERBASE_PROJECT_ID in your environment
const stagehand = new Stagehand({ env: "BROWSERBASE" });
await stagehand.init();
const page = stagehand.context.pages()[0];

await page.goto("https://example.com");

// Basic element discovery
const buttons = await stagehand.observe("find all clickable buttons");
const formFields = await stagehand.observe("locate form input fields");

// Working with results
const [loginButton] = await stagehand.observe("find the login button");
if (loginButton) {
  console.log("Found:", loginButton.description);
  console.log("Selector:", loginButton.selector);
  await stagehand.act(loginButton); // Execute the action
}

​
Integration Patterns

// Observe → Act workflow
const actions = await stagehand.observe("find checkout elements");
for (const action of actions) {
  await stagehand.act(action);
  await page.waitForTimeout(1000);
}

// Observe → Extract workflow
const tables = await stagehand.observe("find data tables");
if (tables.length > 0) {
  const data = await stagehand.extract({
    instruction: "extract the table data",
    selector: tables[0].selector,
    schema: DataSchema
  });
}

// Element validation
const requiredElements = await stagehand.observe("find the login form");
if (requiredElements.length === 0) {
  throw new Error("Login form not found");
}

​
Error Types

The following errors may be thrown by the observe() method:
  • StagehandError - Base class for all Stagehand-specific errors
  • StagehandDomProcessError - Error occurred while processing the DOM
  • StagehandEvalError - Error occurred while evaluating JavaScript in the page context
  • StagehandIframeError - Unable to resolve iframe for the target element
  • ContentFrameNotFoundError - Unable to obtain content frame for the selector
  • XPathResolutionError - XPath does not resolve in the current page or frames
  • StagehandShadowRootMissingError - No shadow root present on the resolved host element
  • LLMResponseError - Error in LLM response processing
  • MissingLLMConfigurationError - No LLM API key or client configured
  • UnsupportedModelError - The specified model is not supported for this operation
  • InvalidAISDKModelFormatError - Model string does not follow the required provider/model format