Browser Actions

Browser actions are the building blocks for web automation.

Overview

Available actions:

• Navigation
• Click interactions
• Text input
• Content extraction
• Screenshots
• Scrolling
• Waiting

Action Types

Navigate

Navigate to a URL:

{
  "action": "navigate",
  "url": "https://example.com"
}

Click

Click an element:

{
  "action": "click",
  "selector": "#submit-button"
}

Options:

{
  "action": "click",
  "selector": ".menu-item",
  "button": "left",
  "click_count": 2,
  "delay": 100
}

Type

Enter text:

{
  "action": "type",
  "selector": "#email-input",
  "text": "user@example.com"
}

With options:

{
  "action": "type",
  "selector": "#search",
  "text": "search query",
  "delay": 50,
  "clear": true
}

Press

Press keyboard keys:

{
  "action": "press",
  "key": "Enter"
}

Common keys:

• Enter, Tab, Escape
• ArrowUp, ArrowDown, ArrowLeft, ArrowRight
• Backspace, Delete
• Modifiers: Shift+Enter, Control+a

Extract

Extract content from page:

{
  "action": "extract",
  "selector": ".product-price",
  "attribute": "text"
}

Extract types:

{
  "action": "extract",
  "selector": "img.hero",
  "attribute": "src"
}

Multiple elements:

{
  "action": "extract",
  "selector": ".item-title",
  "multiple": true
}

Screenshot

Capture screenshot:

{
  "action": "screenshot"
}

With options:

{
  "action": "screenshot",
  "selector": "#main-content",
  "full_page": false,
  "format": "png"
}

Scroll

Scroll the page:

{
  "action": "scroll",
  "direction": "down",
  "amount": 500
}

Scroll to element:

{
  "action": "scroll",
  "selector": "#footer",
  "behavior": "smooth"
}

Wait

Wait for conditions:

{
  "action": "wait",
  "selector": ".loading",
  "state": "hidden"
}

Wait types:

// Wait for element
{
  "action": "wait",
  "selector": "#content",
  "state": "visible",
  "timeout": 10000
}

// Wait for time
{
  "action": "wait",
  "time": 2000
}

// Wait for navigation
{
  "action": "wait",
  "type": "navigation"
}

Executing Actions

Via API

curl -X POST https://your-domain.com/api/agents/{id}/browser/{sessionId}/execute \
  -H "Content-Type: application/json" \
  -d '{
    "action": "click",
    "selector": "#login-button"
  }'

Response:

{
  "success": true,
  "result": null,
  "duration_ms": 150
}

Via Workflow

{
  "nodes": [
    {
      "id": "nav-1",
      "type": "navigate",
      "data": { "url": "https://example.com" }
    },
    {
      "id": "click-1",
      "type": "click",
      "data": { "selector": "#login" }
    },
    {
      "id": "type-1",
      "type": "type",
      "data": {
        "selector": "#email",
        "text": "{{credentials.email}}"
      }
    }
  ]
}

Selectors

CSS Selectors

// By ID
{ "selector": "#my-element" }

// By class
{ "selector": ".my-class" }

// By attribute
{ "selector": "[data-testid='submit']" }

// By tag
{ "selector": "button" }

// Combined
{ "selector": "form#login input[type='email']" }

XPath Selectors

{
  "selector": "//button[contains(text(), 'Submit')]",
  "selector_type": "xpath"
}

Text Selectors

{
  "selector": "text=Click here",
  "selector_type": "text"
}

Action Chaining

Execute multiple actions:

curl -X POST .../browser/{sessionId}/execute \
  -d '{
    "actions": [
      { "action": "navigate", "url": "https://example.com" },
      { "action": "wait", "selector": "#content" },
      { "action": "click", "selector": "#menu" },
      { "action": "screenshot" }
    ]
  }'

Error Handling

Common Errors

Error	Cause	Solution
ElementNotFound	Selector doesn't match	Verify selector, wait for element
Timeout	Action took too long	Increase timeout
NavigationFailed	Page failed to load	Check URL, retry
ClickIntercepted	Element blocked	Wait, scroll into view

Retry Logic

{
  "action": "click",
  "selector": "#button",
  "retry": {
    "attempts": 3,
    "delay": 1000
  }
}

Advanced Actions

Hover

{
  "action": "hover",
  "selector": ".dropdown-trigger"
}

Select

Select dropdown option:

{
  "action": "select",
  "selector": "#country",
  "value": "US"
}

Upload

Upload file:

{
  "action": "upload",
  "selector": "input[type='file']",
  "file_path": "/path/to/file.pdf"
}

Evaluate

Execute JavaScript:

{
  "action": "evaluate",
  "script": "document.querySelector('#count').textContent"
}

Best Practices

1. Use Stable Selectors

Prefer:

• IDs and data attributes
• Semantic selectors

Avoid:

• Dynamic classes
• Position-based selectors

2. Wait for Elements

Always wait before interacting:

[
  { "action": "wait", "selector": "#form" },
  { "action": "type", "selector": "#email", "text": "..." }
]

3. Handle Dynamic Content

Wait for content to load:

{
  "action": "wait",
  "selector": ".loading",
  "state": "hidden"
}

4. Set Appropriate Timeouts

Adjust for slow pages:

{
  "action": "wait",
  "selector": "#data-table",
  "timeout": 30000
}

5. Validate Results

Check action results:

{
  "action": "extract",
  "selector": ".success-message",
  "validate": true
}

API Reference

See Browser API for complete endpoint documentation.