Verified form filling with Ostler

Learn how Ostler matches fields, handles dependent controls, reads values back and distinguishes filling from submission outcomes.

Ostler's form engine is built so the model states intent once and the engine does the mechanics: it resolves each key to a field, fills in order, lets dependent controls appear, reads every value back, and reports what actually happened.

The three-call flow#

1. form_inspect returns typed fields: kind, label, required, options, pattern, current value, and any error. It also lists submit controls and says what memory knows about this form.
2. form_fill takes {label | name | ref: value} for the whole form.
3. form_submit clicks the submit or next control and reports the outcome.

Dependent dropdowns, multi-step wizards, input masks, comboboxes, shadow DOM and same-origin iframes are handled inside these calls. See the tools reference for parameters.

How a key becomes a field#

Keys are resolved in this order: exact ref, then name or id, then label, then fuzzy label. A key can match by ref, name, label or fuzzy label, and keys are resolved against every visible field once, so an ambiguous key cannot win by elimination.

• Ambiguity is reported, never guessed. If two fields match equally well, the report says ambiguous and asks for the ref. email finds "E-mail address", because a key equal to a run of the label's words written together matches, but two equally plausible fields are still refused.
• A key that matches nothing is UNMATCHED. It is not dropped silently, and the fill report only counts as OK if no key was unmatched.
• Forms re-mounted by a framework are followed.

How a value is normalised#

The engine converts the value for the field before writing it:

Fields are filled in document order and the form is re-inspected after every field, so a dependent control (State after Country) appears before it is needed.

Verification: read it back#

After writing a field, the engine reads the value back from the live page and compares it by field type.

• A value the page rewrote or cleared fails. A datetime-local must read back as the requested instant.
• A combobox must show the option you asked for. A longer option that merely contains it is a mismatch ("United States" versus "United States Minor Outlying Islands"), and the in-page picker only picks an exact, normalised option label, listing the options otherwise.
• Values the page legitimately reformats, such as (555) 123-4567, are reported as reformatted, not as errors. Only exact matches and genuine reformatting pass.
• form_fill reports unsettled (a count) when the page was still busy at the end of the fill, so the verdicts can be treated with care.

Typed input is tried first, because it fires the same events a user would. If the page rejects it (masked or framework-controlled inputs), the engine retries with the native value setter plus events. Fills also send one harmless keyup, because some date pickers learn the text only from that event.

Verified, then submitted empty#

Just before clicking submit, form_submit re-reads the fields it verified and reports any the page cleared or rewrote in between as CHANGED BEFORE SUBMIT. This was added after a date picker was reported verified and then submitted empty.

Submit results#

With several forms on a page, form_submit with no form or control submits the form form_fill last filled; if none was filled it refuses and lists the forms rather than guessing the first.

A thank-you page that still has an optional input (a newsletter box, a search) counts as Submitted, not Advanced, unless something on it is required, it has a step counter, or it has a "next" button.

Errors you can only see#

Fields marked invalid only by colour (a red border, outline or glow, or red message text with no ARIA or class) and floating toasts or banners are reported as field and page errors. An error that was already on a field before the fill is flagged as possibly stale. An error message belongs to the field it follows, not to every field sharing a wrapper, and Bootstrap 3 .help-block is help text unless the group is .has-error.

Pages without a <form>#

Only real action buttons are submit candidates. The header's "Search" button, anything inside header, nav or role=search, and type-less buttons whose label says nothing about submitting ("Show tips") do not count.

Dates and pickers#

Calendar date pickers on read-only inputs are handled heuristically: when a text field is read-only and the value is a date, the engine opens the popup, navigates months and clicks the day. It fails with a reason, not silently, when no popup or navigation is found. The heuristic looks for a month header, previous and next controls and day cells. Dropdown-only and year-only pickers are not handled.

Seeing the page#

Two tools help when a form is not obvious:

• look returns a screenshot with every clickable element boxed and numbered, and a legend such as e6 button "Send" -> submits the form.
• snapshot with a heading's ref reads just that section.

A CAPTCHA, hCaptcha, Turnstile or "verify you are human" wall is reported with a CHALLENGE: line at the top of the result. The agent should stop retrying and ask you; with --headed, wait_for_human waits while you complete it. Ostler never solves or bypasses these (ADR-0009).

What it does not handle#

• Cross-origin iframes and closed shadow roots are invisible to the injected runtime.
• 2FA, CAPTCHA solving and anti-bot evasion are out of scope.
• Fill on very large forms (thousands of fields) is slow, because every key re-reads the form.
• It has been evaluated on local fixtures, not on the open web. See Benchmarks.

When not to use Ostler#

Ostler is a form-filling browser for an agent working on your behalf, on one machine. It is the wrong tool when:

• The site walls off automation. CAPTCHA, Cloudflare/Turnstile, "verify you are human" pages and search-engine bot checks are reported with a CHALLENGE: line and handed to a person. Ostler never solves or bypasses them, and it is not built to avoid detection: there is no stealth patching, fingerprint spoofing, residential proxy or IP rotation. A site that blocks headless or datacenter Chrome will block Ostler. --headed plus a persistent --profile-dir lets you pass a check once; it does not make a bot wall go away.
• You need scale. Each server drives one local Chrome (tabs share its cookies and profile). It is not a remote or cloud browser farm, not a crawler, and not meant for bulk scraping or high-volume account creation. More sessions means more servers, each with its own profile and its own memory use (see Benchmarks).
• The flow needs an identity Ostler does not have. There is no mailbox, phone number, SMS/OTP receiver or authenticator: one-time codes, email-link verification, passkeys, hardware keys and 2FA prompts need a person. Passwords and card numbers are never read back or remembered; sensitive fields are refused and the agent is told to ask you. It does not create accounts on your behalf out of thin air.
• The page is outside what the injected runtime can see. Cross-origin iframes (many hosted card fields), closed shadow roots, canvas-drawn UIs, native OS dialogs and file pickers, and desktop apps are not reachable. A control that stays disabled until a person scrolls or hovers is reported as disabled rather than worked around.
• A wrong submit is expensive and nobody is checking. Ostler verifies every value by reading it back and refuses to call a submit a success without evidence, but it cannot know whether a form means what it says. For payments, legal acceptance and anything irreversible, run with --require-confirm-submit and review the result yourself.

When Ostler cannot finish, it says which field and why (rejected value with the page's own message, hidden or disabled control, key not found) and points at screenshot for the element; it does not report a partial fill as done.

Design records#

The reasoning, with the alternatives that were rejected, is in ADR-0004 (injected runtime) and ADR-0005 (form model and verification). A summary is on the Architecture page.

View this page’s source on GitHub