#web-scraping Repositories
382 repositories total
Amazon Nova Act is an AWS service for building and deploying highly reliable AI agents that automate UI-based workflows at scale.
A Python 3 compatible version of goose http://goose3.readthedocs.io/en/latest/index.html
Access your meal tracking data stored in MyFitnessPal programatically
Invisible Playwright MCP: Playwright MCP server undetected by anti-bots and CAPTCHAs: AI agent browses the web using anti-detect stealth Firefox, Python, undetected browser automation, scraping, and computer use.
โญ๏ธ A cute and willful Himalaya album audio unlimited downloader O(โฉ_โฉ)O
A tool for rotating a proxy pool by searching for open socks5 proxies using fofa
Hong Kong Fire (Tai Po, 11/26): A major blaze swept through Wang Fuk Court
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Electron-based multi-platform live streaming auto-recording software, supports Douyin/Kuaishou with multiple link format
A programmable, embeddable web browser driver compatible with the Selenium WebDriver spec -- headless, WebKit-based, pure Java
Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.
Undetected NodeJS version of the Playwright testing and automation library.
A guide to choosing fingerprint browsers (anti-association browsers / anti-detect browsers / Antidetect Browsers) and residential proxy IPs, plus a roundup of open-source resources: pricing plans for 27 leading fingerprint browsers and 28 proxy IP providers, covering Chromium fingerprint customization, Camoufox, Patchright, anti-detect automation, and fingerprint detection tools.
Integrates existing Xiangse Guguan sources on the web, the crawler automatically updates book sources weekly
Selenium that anti-bots canโt see, so no CAPTCHAs: same API, anti-detection stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
The Weibo Hot Search list records popular searches on Weibo starting from November 24, 2020. Data is fetched hourly and archived daily.
Use Cloudflare to create HTTP pass-through proxies for unique IP rotation, similar to fireprox
Stack blocks visually to build complex browser workflows and execute them via API
Scrape XBMC and Kodi movie metadeta and automatically rename files for Japanese Adult Videos (JAV), American Adult DVDs, and American Adult Webcontent
Qinglong panel, join, Banu Hot Pot, FAW-Volkswagen, Haidilao, Watsons, Bawang Tea Master, Huazhu Club, Dongcheng Club, Wanda Hotels, Yihetang Automatic Check-in. Monitoring inventory and price changes at the DJI store
Records the Douyin hot list, star list, live list, music list, and brand list, updated every hour, archived by day.
Residential, datacenter, ISP & mobile proxies with 90M+ IPs in 200+ countries โ plus Scraper & SERP APIs. Code examples in Python, Node.js, PHP, Go, Java, C#, Ruby and curl.
LittleCrawler (ๅฐ็ฌ่ซ) supports scraping public information on the Xiaohongshu (xhs), Zhihu (zh), and XianYu (xy) platforms.
Turn any web app into an API. Chrome extension captures browser traffic, auto-generates schemas, lets AI replay APIs directly. No official API needed.