Troubleshooting Ostler

Symptoms, causes and fixes for Ostler. Start with ostler doctor.

Start with ostler doctor. It launches Chrome, round-trips a page and, if memory is on, prints a memory report and checks Qdrant. The report says in plain words what is stored where and what leaves the machine:

Memory: on, Qdrant at https://… (tenant "default")
  sealing:    ON … (or OFF, with a warning)
  key:        <path to memory.key> (owner-only, good)   | from OSTLER_MEMORY_KEY | not created yet
  embeddings: cloud | local (ollama at …) | local dev-hash
  isolation:  shared collections | one set of collections per tenant
  transport:  https (encrypted) | http to this machine | http to a remote host: REFUSED / ALLOWED
  qdrant:     reachable, collections ready. OK

With nothing configured it prints Memory: in-process, this session only and the rest still works. It never creates the sealing key. Every failing check prints a fix: line with a concrete command. ostler doctor --fix applies only the safe ones: it creates the config directory and tightens an over-open key file to owner-only. It never creates keys, installs software, edits client configs or touches the network. ostler doctor --cdp-url http://127.0.0.1:9222 also tests attaching to a Chrome you started yourself.

When not to use Ostler#

Ostler is a form-filling browser for an agent working on your behalf, on one machine. It is the wrong tool when:

• The site walls off automation. CAPTCHA, Cloudflare/Turnstile, "verify you are human" pages and search-engine bot checks are reported with a CHALLENGE: line and handed to a person. Ostler never solves or bypasses them, and it is not built to avoid detection: there is no stealth patching, fingerprint spoofing, residential proxy or IP rotation. A site that blocks headless or datacenter Chrome will block Ostler. --headed plus a persistent --profile-dir lets you pass a check once; it does not make a bot wall go away.
• You need scale. Each server drives one local Chrome (tabs share its cookies and profile). It is not a remote or cloud browser farm, not a crawler, and not meant for bulk scraping or high-volume account creation. More sessions means more servers, each with its own profile and its own memory use (see Benchmarks).
• The flow needs an identity Ostler does not have. There is no mailbox, phone number, SMS/OTP receiver or authenticator: one-time codes, email-link verification, passkeys, hardware keys and 2FA prompts need a person. Passwords and card numbers are never read back or remembered; sensitive fields are refused and the agent is told to ask you. It does not create accounts on your behalf out of thin air.
• The page is outside what the injected runtime can see. Cross-origin iframes (many hosted card fields), closed shadow roots, canvas-drawn UIs, native OS dialogs and file pickers, and desktop apps are not reachable. A control that stays disabled until a person scrolls or hovers is reported as disabled rather than worked around.
• A wrong submit is expensive and nobody is checking. Ostler verifies every value by reading it back and refuses to call a submit a success without evidence, but it cannot know whether a form means what it says. For payments, legal acceptance and anything irreversible, run with --require-confirm-submit and review the result yourself.

When Ostler cannot finish, it says which field and why (rejected value with the page's own message, hidden or disabled control, key not found) and points at screenshot for the element; it does not report a partial fill as done.

Searching the web#

Ostler drives a real Chrome, but from a fresh, headless session many sites treat it as a bot. What to expect (it varies by network and over time):

• Google, DuckDuckGo, Brave, Startpage, Reddit, news sites such as the NYT: often a bot wall or CAPTCHA. Ostler reports it and hands off to a person; it never tries to solve or get around it.
• Bing: usually loads (but it may ignore first= / "Page 2" and serve page 1 again to a headless session), but may return results unrelated to your query ("Some results have been removed"). When a search-results URL's visible text matches fewer than about half of the query's words, navigate adds Warning: results do not seem to match the query "...". Treat the results as unreliable and try another engine or --headed.

What helps, honestly:

1. Run --headed with a persistent --profile-dir, one profile per server (Chrome locks a profile while it is in use). A person clears a CAPTCHA or signs in once in the visible window; cookies persist for later runs.
2. When the agent hits a wall it calls wait_for_human: it pauses, you solve the page in the window, and it resumes from the same page.
3. For pure discovery (finding which pages to read), use a search API; it is more reliable than scraping a results page. Ostler is best at reading the pages you already have URLs for and at filling forms.

Do not spoof the user agent or try to hide automation; it is unreliable and not something Ostler supports.

If it still fails, open an issue with the output of ostler doctor and your OS and Chrome version.

View this page’s source on GitHub