Loading...
Loading...
A web research tool built inside GrowxLabs to discover companies, crawl websites, extract useful information and keep the evidence behind every finding.
GrowX Crawl is a local-first web crawling, scraping, research and intelligence system built inside GrowxLabs. It supports fast asynchronous HTTP crawling, Playwright/Chromium browser rendering, single-page scraping, 50–100+ page multi-page crawling, configurable crawl depth, 100+ URL batch jobs, CSS/XPath extraction, sitemap and internal-link discovery, caching, proxies, robots handling, screenshots, PDFs, structured company and decision-maker extraction, SEO/AEO/GEO analysis, competitor research, evidence collection, scoring, monitoring and Agent Runtime orchestration.
It uses automatic HTTP-to-browser escalation for JavaScript-heavy sites and includes a separate AI Lab access-resilience layer for detecting WAFs, bot protection, Cloudflare challenges, CAPTCHAs and rate limits, with controlled bypass/resilience testing limited to systems GrowxLabs owns or is authorized to test. The complete system can run locally on a laptop through either CLI commands or the Agent Runtime.
| Specification | Value |
|---|---|
| Runtime | Local-first |
| Control | CLI + Agent Runtime |
| Website Crawl | 50–100+ pages |
| Batch Processing | 100+ URLs |
| Dynamic Websites | Supported |
| Browser Rendering | Playwright / Chromium |
| Sitemap Discovery | Supported |
| Internal Link Mapping | Supported |
| SEO Audit | 0–100 |
| AEO Audit | 0–100 |
| GEO Audit | 0–100 |
| Screenshots | Supported |
| PDF Capture | Supported |
| Website Monitoring | Supported |
| Reports | JSON / CSV / XLSX / PDF |
| Local Dashboard | Supported |
System architecture detailing the dual-engine crawl pipeline, extraction layers, and evidence storage.
GrowXLabs designs, architects, and ships custom digital platforms and software systems.