Per0x1de-1337 / selfmend

What is selfmend?

selfmend is a light-weight self-healing Python scraper on Playwright that mends broken selectors when sites tweak CSS classes or schemas. Feed it a YAML spec describing fields like "title, price in pounds," and it scrapes lists or pages with stealth browsing, human-like mouse moves, and automatic fixes via LLM on the accessibility tree—validated by regex, type, and range checks before caching winners in SQLite. CLI commands like `selfmend init-interactive` let you point-and-click to build specs without writing YAML, or use `selfmend init` for LLM-drafted ones from plain English.

Why is it gaining traction?

Unlike brittle scrapers that die on class renames, selfmend self-heals on the fly, tracks selector success rates with `selfmend stats`, detects schema drift between runs, and replays failed heals offline. The interactive browser picker auto-detects repeating rows for lists, while stealth patches and bezier mouse curves dodge bot detection—users get reliable data without monthly fire drills. Low OpenAI costs via gpt-4o-mini make it practical for daily pipelines.

Who should use this?

Data engineers maintaining product catalogs or job listings that break on site updates. Indie devs scraping e-commerce for price trackers or research tools. Teams with 10+ scrapers tired of YAML tweaks and silent failures in JSON outputs.

Verdict

Try it for light-weight self-healing on Playwright jobs—CLI shines, README covers setup fast—but with 19 stars and 1.0% credibility, it's early alpha: no tests exposed, expect rough edges. Solid for prototypes, watch for stability.

(198 words)