Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Proxy_pool | 19,442 | 4 months ago | 273 | mit | Python | |||||
Python ProxyPool for web spider | ||||||||||
Proxypool | 5,154 | 4 months ago | 40 | mit | Python | |||||
An Efficient ProxyPool with Getter, Tester and Server | ||||||||||
Scylla | 3,819 | 5 | 5 months ago | 30 | August 25, 2019 | 42 | apache-2.0 | Python | ||
Intelligent proxy pool for Humans™ to extract content from the internet and build your own Large Language Models in this new AI era | ||||||||||
Proxybroker | 3,561 | 9 | 3 | 5 months ago | 12 | March 12, 2019 | 101 | apache-2.0 | Python | |
Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS :performing_arts: | ||||||||||
Crawler User Agents | 1,045 | 5 | 8 | 3 months ago | 118 | November 20, 2023 | 7 | mit | Python | |
Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star: | ||||||||||
Scrapyrt | 793 | 5 | 6 months ago | 7 | September 20, 2023 | 31 | bsd-3-clause | Python | ||
HTTP API for Scrapy spiders | ||||||||||
Staticgen | 668 | 3 years ago | 3 | September 23, 2020 | 13 | mit | Go | |||
Static website generator that lets you use HTTP servers and frameworks you already know | ||||||||||
Http Status Check | 587 | 4 | 1 | 4 months ago | 22 | June 14, 2023 | mit | PHP | ||
CLI tool to crawl a website and check HTTP status codes | ||||||||||
Player | 483 | 2 | 3 months ago | 86 | November 28, 2023 | 15 | mit | PHP | ||
Blackfire Player is a powerful Web Crawling, Web Testing, and Web Scraper application. It provides a nice DSL to crawl HTTP services, assert responses, and extract data from HTML/XML/JSON responses. | ||||||||||
Sitemap Generator | 331 | 74 | 27 | a year ago | 61 | September 11, 2020 | 32 | mit | JavaScript | |
Easily create XML sitemaps for your website. |