Browser Automation with ko-browser#

The CLI uses Chrome/Chromium via CDP directly. Install via go install github.com/libi/ko-browser/cmd/kbr@latest or build from source. Run kbr install to verify Chrome is available, or kbr install --with-deps to auto-install it. Existing Chrome, Brave, and Chromium installations are detected automatically.

Installation#

bash

# Install kbr binary directly (no CGO, no external dependencies)
go install github.com/libi/ko-browser/cmd/kbr@latest

# Or build from source
git clone https://github.com/libi/ko-browser.git
cd ko-browser
go build -o kbr ./cmd/kbr/
mv kbr /usr/local/bin/

# Verify browser dependency
kbr install

# Auto-install Chrome if missing
kbr install --with-deps

OCR is optional. The default install has zero CGO dependencies. To enable kbr snapshot --ocr, rebuild with -tags=ocr (requires Tesseract):

bash

# Install Tesseract first: brew install tesseract (macOS) / apt install libtesseract-dev (Linux)
CGO_ENABLED=1 go install -tags=ocr github.com/libi/ko-browser/cmd/kbr@latest

Manual browser installation by OS:

bash

# macOS
brew install --cask google-chrome

# Linux (Debian/Ubuntu)
sudo apt-get install -y chromium-browser

# Linux (Alpine)
apk add chromium

# Windows
choco install googlechrome

Or download from: https://www.google.com/chrome/

Core Workflow#

Every browser automation follows this pattern:

1. Navigate: kbr open <url>
2. Snapshot: kbr snapshot -i (get element IDs like 1, 2, 3)
3. Interact: Use numeric IDs to click, fill, select
4. Re-snapshot: After navigation or DOM changes, get fresh IDs

bash

kbr open https://example.com/form
kbr snapshot -i
# Output:
# 1: textbox "Email"
# 2: textbox "Password"
# 3: button "Submit"

kbr fill 1 "user@example.com"
kbr fill 2 "password123"
kbr click 3
kbr wait load
kbr snapshot -i  # Check result

Command Chaining#

Commands can be chained with && in a single shell invocation. The browser persists between commands via a background daemon, so chaining is safe and more efficient than separate calls.

bash

# Chain open + wait + snapshot in one call
kbr open https://example.com && kbr wait load && kbr snapshot -i

# Chain multiple interactions
kbr fill 1 "user@example.com" && kbr fill 2 "password123" && kbr click 3

# Navigate and capture
kbr open https://example.com && kbr wait load && kbr screenshot page.png

When to chain: Use && when you don't need to read the output of an intermediate command before proceeding (e.g., open + wait + screenshot). Run commands separately when you need to parse the output first (e.g., snapshot to discover IDs, then interact using those IDs).

Handling Authentication#

When automating a site that requires login, choose the approach that fits:

Option 1: Persistent profile (simplest for recurring tasks)

bash

# First run: login manually or via automation
kbr --profile ~/.myapp open https://app.example.com/login
# ... fill credentials, submit ...

# All future runs: already authenticated
kbr --profile ~/.myapp open https://app.example.com/dashboard

Option 2: State file (manual save/load)

bash

# After logging in:
kbr state export ./auth.json

# In a future session:
kbr state import ./auth.json
kbr open https://app.example.com/dashboard

Option 3: Auth vault (credentials stored, login by name)

bash

kbr auth save mysite --url https://app.example.com/login --username user --password pass
kbr auth login mysite

auth login navigates to the URL and waits for login form selectors to appear before filling/clicking.

Option 4: Session name (isolate parallel tasks)

bash

kbr --session myapp open https://app.example.com/login
# ... login flow ...
kbr close  # Session cleaned up

# Use named sessions for parallel automation
kbr --session agent1 open https://site-a.com
kbr --session agent2 open https://site-b.com

Essential Commands#

bash

# Navigation
kbr open <url>                        # Navigate to URL
kbr close                             # Close browser session (alias: stop)
kbr restart                           # Restart browser session
kbr back                              # Go back in history
kbr forward                           # Go forward in history
kbr reload                            # Reload current page

# Snapshot
kbr snapshot -i                       # Interactive elements with IDs (recommended)
kbr snapshot -c                       # Compact mode: omit unnamed wrappers
kbr snapshot --selector "#content"    # Scope to CSS selector
kbr snapshot --depth 3                # Limit tree depth
kbr snapshot --ocr                    # Enable OCR for unnamed images
kbr snapshot --cursor                 # Mark [cursor] on focused element

# Interaction (use numeric IDs from snapshot)
kbr click 1                           # Click element
kbr click --new-tab 1                 # Click and open in new tab
kbr dblclick 1                        # Double-click element
kbr fill 2 "text"                     # Clear and type text
kbr type 2 "text"                     # Type without clearing (append)
kbr select 1 "option"                 # Select dropdown option
kbr check 1                           # Check checkbox
kbr uncheck 1                         # Uncheck checkbox
kbr press Enter                       # Press key (Enter, Tab, Control+a, Meta+c)
kbr keyboard type "text"              # Type at current focus (no element ID)
kbr keyboard inserttext "text"        # Insert without key events
kbr scroll down 500                   # Scroll page (up/down/left/right)
kbr scrollintoview 5                  # Scroll element into viewport
kbr hover 1                           # Hover over element
kbr focus 1                           # Focus element
kbr drag 1 5                          # Drag element 1 to element 5

# Mouse (low-level coordinate-based)
kbr mouse move 100 200                # Move to coordinates
kbr mouse click 100 200               # Click at coordinates
kbr mouse click --button right 100 200  # Right-click
kbr mouse down 100 200                # Press button
kbr mouse up 100 200                  # Release button
kbr mouse wheel 100 200 300           # Scroll wheel

# Get information
kbr get text 1                        # Get element text
kbr get html 1                        # Get element inner HTML
kbr get value 1                       # Get form element value
kbr get attr 1 href                   # Get element attribute
kbr get url                           # Get current URL
kbr get title                         # Get page title
kbr get count "li.item"               # Count elements matching CSS
kbr get box 1                         # Get element bounding box
kbr get styles 1                      # Get computed styles
kbr get cdp-url                       # Get CDP WebSocket URL

# State queries
kbr is visible 1                      # Returns true/false
kbr is enabled 1                      # Returns true/false
kbr is checked 1                      # Returns true/false

# Wait
kbr wait time 2s                      # Wait for duration
kbr wait load                         # Wait for page fully loaded
kbr wait selector "#modal"            # Wait for element to appear
kbr wait hidden ".spinner"            # Wait for element to disappear
kbr wait url "*login*"                # Wait for URL pattern (* wildcard)
kbr wait text "Welcome"               # Wait for text to appear
kbr wait func "window.ready"          # Wait for JS expression to be truthy
kbr wait download ./downloads         # Wait for download to complete

# Downloads & Uploads
kbr upload 3 ./file.pdf               # Upload file to input element
kbr download 5 ./output               # Click element to trigger download

# Network
kbr network start-logging             # Start recording requests
kbr network requests                  # List recorded requests
kbr network route "*.ads.*" --action block   # Block matching requests
kbr network route "*.js" --action continue   # Monitor without blocking
kbr network unroute "*.ads.*"         # Remove route rule
kbr network clear                     # Clear recorded requests

# Viewport & Device Emulation
kbr set viewport 1920 1080            # Set viewport size
kbr set viewport 1920 1080 2          # 2x retina
kbr set device "iPhone 12"            # Emulate device (viewport + UA)
kbr set geo 37.7749 -122.4194         # Override geolocation
kbr set offline true                  # Enable offline mode
kbr set headers X-Token=abc           # Set extra HTTP headers
kbr set credentials admin pass        # Set HTTP Basic Auth
kbr set media prefers-color-scheme=dark  # Emulate CSS media feature
kbr set colorscheme dark              # Shortcut for dark mode

# Capture
kbr screenshot page.png               # Screenshot to file
kbr screenshot -f full.png            # Full page screenshot
kbr screenshot -a annotated.png       # Annotated with element IDs
kbr screenshot --element 5 el.png     # Capture specific element
kbr screenshot -q 80 page.jpeg        # JPEG with quality
kbr pdf output.pdf                    # Save as PDF
kbr pdf -l landscape.pdf              # Landscape PDF

# Clipboard
kbr clipboard read                    # Read text from clipboard
kbr clipboard write "Hello"           # Write text to clipboard
kbr clipboard copy                    # Copy current selection
kbr clipboard paste                   # Paste into focused element

# Console & Errors
kbr console start                     # Start collecting console messages
kbr console messages                  # Get collected messages
kbr console messages --level error    # Filter by level
kbr console clear                     # Clear messages
kbr errors list                       # List page JS errors
kbr errors clear                      # Clear errors

# JavaScript execution
kbr eval "document.title"             # Evaluate JS expression
kbr eval "document.querySelectorAll('a').length"

# Diff (compare page states)
kbr diff snapshot                     # Compare current vs last snapshot
kbr diff snapshot -b before.txt       # Compare current vs saved file
kbr diff screenshot -b before.png     # Visual pixel diff
kbr diff url <url1> <url2>            # Compare two pages

# Tabs
kbr tab list                          # List all open tabs
kbr tab new https://other.com         # Open new tab
kbr tab switch 1                      # Switch to tab by index
kbr tab close 1                       # Close tab by index

# Sessions
kbr session                           # Show current session name
kbr session list                      # List all active sessions

# Cookies
kbr cookies get                       # Get all cookies
kbr cookies set token abc --domain example.com --secure --http-only
kbr cookies delete token              # Delete cookie by name
kbr cookies clear                     # Clear all cookies

# Storage (localStorage / sessionStorage)
kbr storage list                      # List all entries
kbr storage get theme                 # Get value
kbr storage set theme dark            # Set value
kbr storage delete theme              # Delete key
kbr storage clear                     # Clear all
kbr storage get key --type session    # Use sessionStorage

# State import/export
kbr state export auth.json            # Export cookies + localStorage
kbr state import auth.json            # Import from file

# Auth vault
kbr auth save mysite --url <url> --username <user> --password <pass>
kbr auth login mysite                 # Login using saved profile
kbr auth list                         # List saved profiles
kbr auth show mysite                  # Show profile details
kbr auth delete mysite                # Delete profile

# Debugging
kbr highlight 1                       # Highlight element with red border
kbr inspect                           # Open DevTools (headed mode only)

# Tracing & Profiling
kbr trace start                       # Start Chrome trace
kbr trace stop trace.json             # Stop and save
kbr profiler start                    # Start CPU profiling
kbr profiler stop profile.json        # Stop and save
kbr record start                      # Record screenshot sequence
kbr record stop                       # Stop recording

# Confirmation (when --confirm-actions is set)
kbr confirm                           # Approve pending action
kbr deny                              # Reject pending action

# Find elements (semantic locators, alternative to snapshot IDs)
kbr find role button --name "Submit"  # By ARIA role + name
kbr find text "Sign In"               # By accessible name
kbr find label "Email"                # By label text
kbr find placeholder "Search"         # By placeholder
kbr find alt "logo"                   # By image alt text
kbr find title "info"                 # By title attribute
kbr find testid "submit-btn"          # By data-testid
kbr find first "div.item"             # First CSS match
kbr find last "div.item"              # Last CSS match
kbr find nth 2 "div.item"            # Nth CSS match

ID Lifecycle (Important)#

IDs ( 1, 2, 3, ...) are invalidated when the page changes. Always re-snapshot after:

• Clicking links or buttons that navigate
• Form submissions
• Dynamic content loading (dropdowns, modals)

bash

kbr click 5                           # Navigates to new page
kbr snapshot -i                       # MUST re-snapshot
kbr click 1                           # Use new IDs

IDs are strictly sequential integers starting from 1, assigned via depth-first traversal of the accessibility tree. They are temporary — never cache IDs across snapshots.

Snapshot Format#

The snapshot is a text representation of the browser's Accessibility Tree. Each line represents one element:

text

<indent><id>: <role> "<name>" [value="<value>"] [<state1> <state2> ...]

Example:

text

Page: "Login - GitHub"

1: link "GitHub"
2: heading "Sign in to GitHub"
3: textbox "Username or email" required
4: textbox "Password" required
5: link "Forgot password?"
6: button "Sign in"
7: link "Create an account"

Format details:

• Page: "title" — root node, not interactive, no ID assigned
• 2-space indentation indicates tree depth
• Names are truncated at 80 characters with ...
• Values shown only when different from name (e.g., value="admin")
• Common states: focused, disabled, checked, expanded, collapsed, selected, required, readonly, multiline

Why numeric IDs: Token-efficient for LLMs. 5: button "Submit" uses fewer tokens than @e5 button "Submit" or [5] button "Submit". LLMs reference elements with just a number: kbr click 5.

Common Patterns#

Form Submission#

bash

kbr open https://example.com/signup
kbr snapshot -i
kbr fill 1 "Jane Doe"
kbr fill 2 "jane@example.com"
kbr select 3 "California"
kbr check 4
kbr click 5
kbr wait load

Authentication with Auth Vault#

bash

# Save credentials once
kbr auth save github --url https://github.com/login --username user --password pass

# Login using saved profile (LLM never sees password)
kbr auth login github

# List/show/delete profiles
kbr auth list
kbr auth show github
kbr auth delete github

Authentication with State Persistence#

bash

# Login once and save state
kbr open https://app.example.com/login
kbr snapshot -i
kbr fill 1 "$USERNAME"
kbr fill 2 "$PASSWORD"
kbr click 3
kbr wait url "*dashboard*"
kbr state export auth.json

# Reuse in future sessions
kbr state import auth.json
kbr open https://app.example.com/dashboard

Data Extraction#

bash

kbr open https://example.com/products
kbr snapshot -i
kbr get text 5                        # Get specific element text
kbr get title                         # Get page title
kbr get url                           # Get current URL

# JSON output for parsing
kbr snapshot -i --json
kbr get text 1 --json

Parallel Sessions#

bash

kbr --session site1 open https://site-a.com
kbr --session site2 open https://site-b.com

kbr --session site1 snapshot -i
kbr --session site2 snapshot -i

kbr session list

Viewport & Responsive Testing#

bash

# Desktop
kbr set viewport 1920 1080
kbr screenshot desktop.png

# Mobile
kbr set viewport 375 812
kbr screenshot mobile.png

# Retina: same CSS layout, higher pixel density
kbr set viewport 1920 1080 2
kbr screenshot retina.png

# Device emulation (viewport + user agent)
kbr set device "iPhone 12"
kbr screenshot device.png

Visual Browser (Debugging)#

bash

kbr --headed open https://example.com
kbr highlight 1                       # Highlight element
kbr inspect                           # Open Chrome DevTools
kbr record start                      # Record session
kbr profiler start                    # Start profiling
kbr profiler stop trace.json          # Stop and save

Local Files#

bash

kbr --allow-file-access open file:///path/to/page.html
kbr snapshot -i
kbr screenshot output.png

Color Scheme (Dark Mode)#

bash

kbr set colorscheme dark
kbr screenshot dark.png

# Or via media feature emulation
kbr set media prefers-color-scheme=dark

Annotated Screenshots (Vision Mode)#

Use --annotate ( -a) to take a screenshot with numbered labels overlaid on interactive elements. This helps when:

• The page has unlabeled icon buttons or visual-only elements
• You need to verify visual layout or styling
• Canvas or chart elements are present (invisible to text snapshots)

bash

kbr screenshot -a annotated.png

Diffing (Verifying Changes)#

Use diff snapshot after performing an action to verify it had the intended effect:

bash

# Typical workflow: snapshot -> action -> diff
kbr snapshot -i                       # Take baseline snapshot
kbr click 2                           # Perform action
kbr diff snapshot                     # See what changed

For visual regression or monitoring:

bash

# Save a baseline, then compare later
kbr screenshot baseline.png
# ... changes happen ...
kbr diff screenshot -b baseline.png

# Compare two URLs
kbr diff url https://staging.example.com https://prod.example.com

diff snapshot output uses + for additions and - for removals, similar to git diff. diff screenshot produces a diff image with changed pixels highlighted.

Timeouts and Slow Pages#

The default timeout is 30 seconds (configurable with --timeout). For slow pages, use explicit waits:

bash

# Wait for network to settle
kbr wait load

# Wait for a specific element
kbr wait selector "#content"

# Wait for a URL pattern (useful after redirects)
kbr wait url "*dashboard*"

# Wait for a JS condition
kbr wait func "document.readyState === 'complete'"

# Wait a fixed duration as a last resort
kbr wait time 5s

When dealing with slow sites, use kbr wait load after kbr open to ensure the page is fully loaded before taking a snapshot.

Session Management and Cleanup#

When running multiple agents concurrently, use named sessions to avoid conflicts:

bash

# Each agent gets its own isolated session
kbr --session agent1 open site-a.com
kbr --session agent2 open site-b.com

# Check active sessions
kbr session list

Always close your session when done to avoid leaked daemon processes:

bash

kbr close                             # Close default session
kbr --session agent1 close            # Close specific session

Security#

Content Boundaries#

Enable --content-boundaries to wrap page-sourced output in markers that help LLMs distinguish tool output from untrusted page content:

bash

kbr --content-boundaries snapshot
# Output wrapped in:
# <ko-browser-content>
# [accessibility tree]
# </ko-browser-content>

Configuration File#

Create ko-browser.json in the project root for persistent settings:

json

{
  "headed": true,
  "proxy": "http://localhost:8080",
  "profile": "./browser-data"
}

Priority (lowest to highest): config file < env vars < CLI flags. Use --config <path> for a custom config location.