Awesome Open Source
Awesome Open Source

awesome_crawl Awesome


爬虫 说明 技术栈
腾讯新闻 采集所有腾讯新闻的链接和新闻详情 scrapy,mongo,redis
知乎话题 从话题广场出发,先采集子话题ID,再采集ID下所有问题 scrapy,mongo,redis
微博粉丝 采集大V的所有粉丝 scrapy,mongo,redis
Tumblr爬虫 下载指定Tumblr博主的资源 requests,concurrent
妹子图爬虫 go语言下载妹子图 goroutine,goquery
全站爬虫 分布式全站爬虫——以"搜狗电视剧"为例 scrapy,redis,分布式

如果能帮助你,那就最好了。欢迎关注公众号:


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
python (55,393
golang (3,955
awesome (1,412
redis (976
websockets (317
scrapy (130
weibo (64
zhihu (56
tumblr (28
douyu (26
tencent (25
crawl (19