Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Request Crawler Open Source Projects

Open source projects categorized as Request Crawler
Categories > Data Processing > Crawler
Categories > Networking > Request
Decryptlogin ⭐ 2,375
DecryptLogin: APIs for loginning some websites by using requests.
dependent packages 4total releases 49most recent commit 2 years agopypi DecryptLogin} Downloads
Grab ⭐ 2,292
Web Scraping Framework
dependent packages 3total releases 120most recent commit 9 months agopypi grab} Downloads
Geziyor ⭐ 1,892
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
dependent packages 11total releases 6most recent commit a year ago
Go_spider ⭐ 1,629
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.
dependent packages 2total releases 1most recent commit 6 years ago
Python Crawler ⭐ 1,576
从头开始 系统化的 学习如何写Python爬虫。 Python版本 3.6
most recent commit 2 years ago
Bilili ⭐ 999
🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器
total releases 68most recent commit 3 months ago
Prerender Node ⭐ 916
Express middleware for prerendering javascript-rendered pages on the fly for SEO
dependent packages 38total releases 71most recent commit a year agonpm prerender-node} Downloads
Course Crawler ⭐ 633
🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。
most recent commit 2 years ago
Easy Scraping Tutorial ⭐ 618
Simple but useful Python web scraping tutorial code.
most recent commit 3 years ago
Python Fxxk Spider ⭐ 571
收集各种免费的 Python 爬虫项目
most recent commit 4 months ago
Bolt ⭐ 482
CSRF Scanner
most recent commit 4 months ago
Python3webcrawler ⭐ 347
🌈Python3网络爬虫实战：QQ音乐歌曲、京东商品信息、房天下、破解有道翻译、构建代理池、豆瓣读
most recent commit a year ago
Curl Easy ⭐ 282
cURL wrapper for PHP. Supports parallel and non-blocking requests. For high speed crawling, see stil/curl-robot
dependent packages 8total releases 9most recent commit 5 years agopackagist stil/curl-easy} Downloads
Sasila ⭐ 264
一个灵活、友好的爬虫框架
total releases 17most recent commit 4 years ago
Scrapy Deltafetch ⭐ 232
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
total releases 9most recent commit 2 years ago
Tiktok Downloader ⭐ 223
Tiktok Downloader/Scraper using requests & bs4
most recent commit 5 months ago
Infinitycrawler ⭐ 221
A simple but powerful web crawler library for .NET
total releases 7most recent commit 7 months ago
Qqzone_crawler ⭐ 179
QQ 空间动态爬虫，利用cookie登录获取所有可访问好友空间的动态保存到本地
most recent commit 5 years ago
Spider_reverse ⭐ 178
爬虫逆向案例，已完成：震坤行 | 网易易盾 | 微信小程序反编译逆向(百达星系) | 同花顺 | rpc解密 | 加速乐 | 极验滑块验证码 | 巨量算数 | Boss直聘 | 企查查 | 中国五矿 | qq音乐 | 产业政策大数据平台 | 企知道 | 雪球网(acw_sc__v2) | 1688 | 七麦数据 | whggzy | 企名科技 | mohurd | 艺恩数据 | 欧科云链
most recent commit 5 months ago
Estela ⭐ 142
estela, an elastic web scraping cluster 🕸
most recent commit 3 months ago
Not Your Average Web Crawler ⭐ 130
A web crawler (for bug hunting) that gathers more than you can imagine.
total releases 36most recent commit 2 years ago
Zhihu Spider ⭐ 128
一个获取知乎用户主页信息的多线程Python爬虫程序。
most recent commit 5 years ago
Reqman ⭐ 124
Reqman is a tool that can quickly help back-end engineers with api testing, as well as a nodejs-based crawler tool.
dependent packages 1total releases 6most recent commit 5 years agonpm reqman} Downloads
The 10 Most Depended On Request Crawler Open Source Projects
Prerender Node ⭐ 916
Express middleware for prerendering javascript-rendered pages on the fly for SEO
dependent packages 38total releases 71latest release May 22, 2023most recent commit a year agonpm prerender-node} Downloads
X Ray Crawler ⭐ 39
Friendly web crawler for x-ray
dependent packages 19total releases 7latest release August 26, 2020most recent commit 3 years agonpm x-ray-crawler} Downloads
Bear.resource ⭐ 45
A hypermedia framework for an object as a service
dependent packages 16total releases 125latest release August 22, 2023most recent commit 3 months agopackagist bear/resource} Downloads
Node Webcheck ⭐ 14
A node.js module to analyse websites for SEO, validation and code-quality
dependent packages 11total releases 10latest release January 19, 2019most recent commit 5 years agonpm webcheck} Downloads
Seenreq ⭐ 32
Generate an object for testing if a request is sent, request is Mikeal's request.
dependent packages 11total releases 26latest release April 15, 2019most recent commit 5 years agonpm seenreq} Downloads
Geziyor ⭐ 1,892
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
dependent packages 11total releases 6latest release April 19, 2021most recent commit a year ago
Curl Easy ⭐ 282
cURL wrapper for PHP. Supports parallel and non-blocking requests. For high speed crawling, see stil/curl-robot
dependent packages 8total releases 9latest release May 20, 2017most recent commit 5 years agopackagist stil/curl-easy} Downloads
Googlebot ⭐ 19
Express middleware that returns the resulting html after executing javascript, allowing crawlers to read on the page
dependent packages 5total releases 3latest release February 25, 2014most recent commit 10 years agonpm googlebot} Downloads
Express Nobots ⭐ 8
Keep Bots And Crawlers Away From Your Express App
dependent packages 4total releases 6latest release December 11, 2019most recent commit 4 years agonpm express-nobots} Downloads
The 10 Latest Releases In Request Crawler Open Source Projects
Escargot ⭐ 41
A web crawler or spider library based on Symfony components
dependent packages 2total releases 37latest release November 09, 2023most recent commit 5 months agopackagist terminal42/escargot} Downloads
Bear.resource ⭐ 45
A hypermedia framework for an object as a service
dependent packages 16total releases 125latest release August 22, 2023most recent commit 3 months agopackagist bear/resource} Downloads
Prerender Node ⭐ 916
Express middleware for prerendering javascript-rendered pages on the fly for SEO
dependent packages 38total releases 71latest release May 22, 2023most recent commit a year agonpm prerender-node} Downloads
Fuckvkeypad ⭐ 51
가상키보드(vKeypad) 우회도구
total releases 2latest release April 10, 2023most recent commit a year ago
Marmot ⭐ 18
💐Marmot A Golang HTTP Download
dependent packages 2total releases 13latest release November 25, 2022most recent commit a year ago
Decryptlogin ⭐ 2,375
DecryptLogin: APIs for loginning some websites by using requests.
dependent packages 4total releases 49latest release August 29, 2022most recent commit 2 years agopypi DecryptLogin} Downloads
Infinitycrawler ⭐ 221
A simple but powerful web crawler library for .NET
total releases 7latest release August 10, 2022most recent commit 7 months ago
Local_ua ⭐ 7
Random request headers for local versions of browsers
total releases 9latest release June 21, 2022most recent commit 2 years ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.