Awesome Open Source

Programming Languages

Search results for kafka spider

11 search results found

Scrapy Cluster ⭐ 1,137

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Netdiscovery ⭐ 557

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

Spiderman ⭐ 498

基于 scrapy-redis 的通用分布式爬虫框架

Apiproject ⭐ 159

[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+ modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+live

Scrapy_demo ⭐ 150

all kinds of scrapy demo

Scrapy Distributed ⭐ 40

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Scrapy Kafka Demo ⭐ 14

Scrapy and Kafka

Requests_html_spider ⭐ 7

requests升级版requests-html 爬虫编写及通用爬虫模块搭建

基于django和scrapy的采集系统

Scrapy Cluster ⭐ 5

modified by http://github.com/istresearch/scrapy-cluster.git to make it appropriate for project.

Related Searches

Java Kafka (3,237)

Python Spider (2,155)

Kafka Zookeeper (1,229)

Docker Kafka (1,191)

Python Kafka (1,053)

Spider Scrapy (982)

Spark Kafka (979)

Scala Kafka (969)

Golang Kafka (919)

Apache Kafka (836)

1-11 of 11 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.