Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Xidel | 611 | 5 months ago | 18 | gpl-3.0 | Pascal | |||||
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents. | ||||||||||
Ultimate Web Scraper | 400 | a year ago | PHP | |||||||
A PHP library/toolkit designed to handle all of your web scraping needs under a MIT or LGPL license. Also has web server and WebSocket server classes for building custom servers. | ||||||||||
Cheers | 194 | 3 | 7 years ago | 7 | October 27, 2015 | mit | HTML | |||
Scrape a website efficiently, block by block, page by page. Based on cheerio and curl. | ||||||||||
Grawler | 128 | 3 years ago | 1 | mit | PHP | |||||
Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file. | ||||||||||
Argus | 67 | 2 years ago | 3 | gpl-3.0 | Python | |||||
ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9 | ||||||||||
Secraper | 60 | 4 years ago | gpl-3.0 | Shell | ||||||
Search engine scraper tool with BASH script. | ||||||||||
Shopee Scrape | 44 | 3 years ago | 4 | PHP | ||||||
Shopee Scrape is a tool that functions to collect data - the data needed, such as finding data from photos, prices, names, store locations and others. | ||||||||||
Caterpillar | 39 | 8 years ago | other | PHP | ||||||
Caterpillar is a PHP library intended for website crawling and screen scraping. It handles parallel requests using the curl_multi functions. | ||||||||||
Web Scraper | 37 | 7 years ago | 1 | January 06, 2017 | 2 | PHP | ||||
A web scraper php class using PHP cURL to scrap web page. By which you can scrap web page by cURL get, post methods also by which you can scrap web page content from a asp.net based websites with form post. | ||||||||||
Video Crawler | 26 | 4 years ago | 3 | mit | Scala | |||||
Crawl websites for videos from Youtube, Vimeo, Soundcloud, etc |