Awesome Open Source

Programming Languages

Search results for kafka parquet

20 search results found

Bigdata Playground ⭐ 154

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Eel Sdk ⭐ 140

Big Data Toolkit for the JVM

kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)

Etl Light ⭐ 38

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Paraflow ⭐ 36

A real-time analytical system for ID-associated data

Minipipe ⭐ 30

Minipipe: a minimal end-to-end data pipeline

Kafka Parquet Writer ⭐ 26

This project provides a compenent that reads logs from Kafka and writes it as parquet file on HDFS.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Kafka Connect Oss ⭐ 21

Kafka Connect suite of connectors for OSS

Cda Client ⭐ 19

Cloud Data Access client

Data Generator ⭐ 13

This repo is for generating data from existing dataset to a file or producing dataset rows as message to kafka in a streaming manner.

Yet Another Avro CLI Tool

Telecom Streaming ⭐ 9

Telecom scenarios implemented with streaming techniques

Flink10_learn ⭐ 9

flink 10 自我学习笔记和代码

Random Datagen ⭐ 7

A generator of Random Data to HDFS, HBase, Hive, Kafka, Kudu, Ozone, SolR in CDP (Cloudera Data Platform)

Kafka Connect S3 Parquet ⭐ 7

Drillbook ⭐ 6

The Official Source Repository for Learning Apache Drill (O'Reilly, 2018)

Bigdata Platform ⭐ 6

End to end big data project, that aims to show how to implement different big data layers, from the infrastructure layer to the end user one. [HADOOP][Spark][Kafka][Cassandra][Ansible][Jupyter

Kafka Replicator ⭐ 5

Kafka replicator is a tool used to mirror and backup Kafka topics across regions

Avroparquet ⭐ 5

AVRO / Parquet Demo Code

Related Searches

Java Kafka (3,237)

Kafka Zookeeper (1,229)

Docker Kafka (1,106)

Python Kafka (1,053)

Spark Kafka (1,006)

Scala Kafka (969)

Golang Kafka (919)

Apache Kafka (836)

Stream Kafka (790)

Shell Kafka (642)

1-20 of 20 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.