Cloudflare Browser Run Crawl Events: Queue-Based Orchestration Without Polling
QUICK ANSWER
Cloudflare added Browser Run crawl event subscriptions on September 25, 2026. An account-level Browser Run subscription can publish crawl.started, crawl.updated, and crawl.finished lifecycle events to a Cloudflare Queue, allowing downstream systems to react to progress instead of repeatedly polling a crawl job. Cloudflare's event schema exposes a jobId for correlation; crawl.updated identifies a URL plus crawlStatus and HTTP status; crawl.finished includes terminal job status and total, completed, errored, and skipped counts. Cloudflare Queues uses at-least-once delivery by default, so consumers should be idempotent and able to process the same message more than once. Treat lifecycle events as orchestration signals, preserve the crawl result as the detailed source of truth, and do not equate a finished job with every URL having completed successfully.
A Browser Run crawl integration repeatedly polls job status when the application only needs lifecycle transitions or downstream triggers
A queue consumer creates duplicate records, notifications, exports, or follow-up jobs when the same event is delivered more than once
A dashboard marks an entire crawl successful as soon as crawl.finished arrives without checking completed, errored, and skipped counts
An application cannot correlate crawl.updated URL events with the crawl job that created them
A team wants event-driven crawl automation but has not defined retry, deduplication, dead-letter, or observability behavior
Possible causes
Browser Run crawl lifecycle events are now available through account-level Cloudflare Queues subscriptions, so polling is no longer the only progress-notification pattern
Cloudflare Queues provides at-least-once delivery by default, which means a message can occasionally be delivered more than once
The Browser Run schemas expose different payloads for started, per-URL updated, and finished events; treating them as interchangeable loses state and error detail
A finished crawl can contain a mix of completed, errored, and skipped work, so terminal job state is not the same as all-URL success
Without a stable correlation and idempotency strategy, downstream actions can be duplicated when retries or repeated deliveries occur
HOW TO FIX IT
Work from the safest step to the harder repair.
Step 1. Inventory where the current Browser Run integration polls crawl status and separate genuine read-after-write needs from simple lifecycle notification.
Step 2. Create or select the Cloudflare Queue that should receive Browser Run lifecycle events and scope the consumer to the smallest downstream responsibility needed.
Step 3. Create the Browser Run event subscription for crawl.started, crawl.updated, and crawl.finished using the Cloudflare dashboard or the documented Wrangler subscription flow.
Step 4. Use payload.jobId as the primary crawl correlation key so started, per-URL updated, and finished events stay attached to the same job.
Step 5. For crawl.updated, treat the URL, crawlStatus, and HTTP status as a per-resource transition rather than an entire-job result.
Step 6. For crawl.finished, inspect jobStatus together with total, completed, errored, and skipped counts before presenting the crawl as complete.
Step 7. Make every consumer idempotent. Cloudflare documents at-least-once Queue delivery, so use a durable deduplication or operation key before creating downstream side effects.
Step 8. Acknowledge messages only after the intended durable work succeeds. Configure bounded retries and a dead-letter path for messages that cannot be processed safely.
Step 9. Keep the queue event payload separate from the detailed crawl result. When downstream work needs page content or complete crawl output, retrieve it through the supported Browser Run result path.
Step 10. Log correlation identifiers, event type, schema version, timestamps, and safe status fields while avoiding unnecessary sensitive crawl content.
Step 11. Test duplicate delivery, consumer restarts, crawls with errored or skipped URLs, queue retry behavior, and a dead-letter scenario before replacing a stable polling implementation.
Step 12. After rollout, compare queue event state with crawl-job records and retain a controlled recovery path for a missed downstream action.
Need the actual code? Go to GenesisCodeDoctor.com to search the Code Store or request code for the exact platform, error, and repair you are working on.
1. Do you only need to react when crawl state changes? Prefer lifecycle events over repeated status polling.
2. Do you need full crawled content or detailed result data? Use the event as a trigger and retrieve the supported crawl result.
3. Can the downstream side effect safely run twice? If no, add durable idempotency before enabling the consumer.
4. Does crawl.finished report errored or skipped URLs? Reconcile those items before declaring the workflow complete.
5. Does the consumer fail repeatedly? Use bounded retry and dead-letter handling rather than allowing uncontrolled repeated side effects.
REPAIR FLOW
A visual path from symptom to verified production.
Observe→
Protect→
Isolate→
Repair→
Test→
Publish→
Verify live
BEFORE YOU PASTE CODE
Protect the working site first.
Design the Queue consumer for duplicate delivery and make durable side effects idempotent before acknowledging the message.
Validate event type and schema fields defensively; reject or quarantine malformed or unrecognized payloads instead of guessing missing values.
Keep Browser Run account credentials and detailed crawl-result access server-side and out of routine event logs.
Test mixed-success crawls so a finished event cannot accidentally collapse per-URL errors into a green overall status.
STOP AND GET HELP WHEN
Do not turn a repair into a larger outage.
Do not assume a Cloudflare Queue message is delivered exactly once; Cloudflare documents at-least-once delivery by default.
Do not mark every crawled URL successful merely because crawl.finished was emitted; reconcile completed, errored, and skipped counts.
Do not use a URL by itself as the durable job key when multiple crawl jobs can include the same URL.
Do not trigger non-idempotent billing, email, deployment, or destructive operations directly from an event without a deduplication boundary.
Do not remove an existing polling path until event subscription, retries, consumer observability, and recovery have been verified.
HOW GENESIS HANDLES IT
Diagnose the exact failure before choosing a repair.
Genesis separates the visible symptom from the underlying technical cause. Run the supported diagnostic first, review the evidence, and then use a matching repair only when the failure is actually verified.
Platform scope
Cloudflare Browser Run · Cloudflare Queues · Wrangler · crawl API · event subscriptions
Which Browser Run crawl events can I subscribe to?
Cloudflare currently documents crawl.started, crawl.updated, and crawl.finished for the Browser Run event source.
Do Browser Run crawl events eliminate every reason to poll?
No. They remove routine lifecycle polling when an event-driven trigger is enough, but detailed crawl output can still require the crawl result path.
Can a Cloudflare Queue message arrive more than once?
Yes. Cloudflare documents at-least-once delivery by default and says messages may occasionally be delivered more than once, so consumers should be idempotent.
Does crawl.finished mean every URL succeeded?
No. Cloudflare's finished-event schema includes separate completed, errored, and skipped counts.
Start with a free diagnostic. If Genesis verifies a problem and a compatible treatment exists, continue to the matching Code Store product or repair path. If you cannot find the exact code you need, request it at GenesisCodeDoctor.com rather than forcing a generic snippet into the wrong platform.
