observe() is used to get a list of actions that can be taken on the current page. It’s useful for adding context to your planning step, or if you unsure of what page you’re on.

  const observations = await page.observe();

observe() returns an array of objects, each with an XPath selector and short description.

If you are looking for a specific element, you can also pass in an instruction to observe via: observe({ instruction: "your instruction"}).

  const [{ selector, description }] = await page.observe({
    instruction: "Find the buttons on this page",
  });

Observe can also return a suggested action for the candidate element by setting the returnAction option to true. Here is a sample ObserveResult:

  {
    "description": "A brief description of the component",
    "method": 'click',
    "arguments": [],
    "selector": 'xpath=/html/body[1]/div[1]/main[1]/button[1]'
  }

Arguments: ObserveOptions

instruction
string

Provides instructions for the observation. Defaults to “Find actions that can be performed on this page.”

returnAction
boolean

Returns an observe result object that contains a suggested action for the candidate element. The suggestion includes method, and arguments (if any). Defaults to false.

onlyVisible
boolean

If true, returns only visible elements. Uses DOM inspection instead of accessibility trees. Defaults to false.

useAccessibilityTree
boolean
deprecated

[Deprecated] Previously used for accessibility tree observation. Use onlyVisible: false instead.

modelName
AvailableModel

Specifies the model to use

modelClientOptions
object

Configuration options for the model client

useVision
boolean | 'fallback'
deprecated

[Deprecated] Previously used to control vision-based processing. Vision processing is now always enabled.

domSettleTimeoutMs
number

Timeout in milliseconds for waiting for the DOM to settle

Returns: Promise<ObserveResult[]>

Each ObserveResult object contains a selector and description.

selector
string
required

A string representing the element selector

description
string
required

A string describing the possible action

If the returnAction option is set to true, the following fields will be included in the result.

method
string

The method to call on the element

arguments
object

The arguments to pass to the method