DevToolsOpen SourceAutomationSaaS

RunLoop — Version control for record-once scraping rules

Turn browser actions into replayable, versioned, rollback-ready scraping tasks

Problem
Writing a scraper is easy; maintaining one is not. A page redesign breaks every selector, and when the author leaves, nobody dares touch the script. AegisCrawler takes the 'record once, run forever' approach with a YAML DSL, real-browser replay validation, and immutable versions, but the full platform is heavy for small teams that mainly need versioned, regression-tested capture rules.
Solution
A lightweight open-source core plus a hosted panel: a Chrome extension records actions into YAML rules, each save creates an immutable version, one-click replay validates it, and failures produce screenshots and diffs against the previous version. Rules become scheduled tasks with leasing, retries, and a dead-letter queue, focused on rule regression and version comparison rather than a full-blown suite.
Users
Small data-collection teams, indie developers, and growth or ops engineers who maintain scrapers long term.
MVP
Chrome extension that records into YAML; local or hosted immutable version storage; real-browser replay validation; failure screenshots and version-to-version diff; simple scheduling with retries.
Revenue
Open-source core with a hosted subscription (scheduling, alerts, historical replay); usage-based enterprise tier.
Why now
AegisCrawler proves the record-as-rule approach, but the full platform is heavy for individuals and small teams. Extracting version management and regression validation into a focused tool is a few weeks of work and sits squarely on the long-running pain of scraper maintenance.

Source discussions · 1

  1. V2EX[程序员] 开源了一个"录一次、永久跑"的浏览器采集平台 AegisCrawler(Go + TS, GPL)
← Back to list