Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for spark avro
avro
x
spark
x
49 search results found
Adam
⭐
966
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
Kafka Storm Starter
⭐
729
[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
Devops Python Tools
⭐
709
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Sparklearning
⭐
573
Learning Apache spark,including code and data .Most part can run local.
Data Engineering Interview Questions
⭐
554
More than 2000+ Data engineer interview questions.
Spark Avro
⭐
535
Avro Data Source for Apache Spark
Shc
⭐
484
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Marmaray
⭐
444
Generic Data Ingestion & Dispersal Library for Hadoop
Sparkling
⭐
423
A Clojure library for Apache Spark: fast, fully-features, and developer friendly
Iceberg
⭐
409
Iceberg is a table format for large, slow-moving tabular data
Abris
⭐
215
Avro SerDe for Apache Spark structured APIs.
Rumble
⭐
194
⛈️ RumbleDB 1.21.0 "Hawthorn blossom" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Spark Bigquery
⭐
149
Google BigQuery support for Spark, SQL, and DataFrames
Schemer
⭐
89
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Spark Structured Streaming
⭐
66
Spark structured streaming with Kafka data source and writing to Cassandra
Avro Parquet Spark Example
⭐
61
An example of using Avro and Parquet in Spark SQL
Tablasco
⭐
52
Tablasco is a JUnit rule for comparing tables and Spark module for comparing large data sets
Spark Compaction
⭐
52
File compaction tool that runs on top of the Spark framework.
Spark Mail
⭐
45
Tutorial on parsing Enron email to Avro and then explore the email set using Spark.
Kafka Spark Hbase Example
⭐
39
Etl Light
⭐
38
A light Kafka to HDFS/S3 ETL library based on Apache Spark
Simplesparkavroapp
⭐
32
Simple Spark app that reads and writes Avro data
Framework Of Bigdata
⭐
30
大数据面试题,从0到1走向架构师之路。Flink、Spark、Hive、HBase、Hadoop、K
Kafka Compose
⭐
28
🎼 Docker compose files for various kafka stacks
Daflow
⭐
24
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Scala Kafka Twitter
⭐
23
Example integration of Kafka, Avro & Spark-Streaming on live Twitter feed
Darwin
⭐
22
Avro Schema Evolution made easy
Sparkonalog
⭐
19
Examples of Integrating Spark Streaming, Flume, and HBase to solve Streaming problems
Spark
⭐
18
SPARQL client API and a high-speed protocol implementation
Spark Kafka Avro
⭐
17
POC: Spark consumer for bottledwater-pg Kafka Avro topics
Confluent Spark Avro
⭐
17
Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.
Spark Bigquery
⭐
15
Google BigQuery data source for Apache Spark
Spark Dbf
⭐
14
Spark SQL DBF Library
Cmsspark
⭐
12
General purpose framework to run CMS experiment workflows on HDFS/Spark platform
Camus Compressor
⭐
12
Camus Compressor merges files created by Camus and saves them in a compressed format.
Sparkavro
⭐
11
Load Avro data into Spark with sparklyr
Spark Streaming Kafka Example
⭐
11
An example project using Spark Streaming with Kafka message and Avro serialization
Amazon S3 Tagging Spark Util
⭐
10
Confluent Platform Spark Streaming
⭐
10
Working example of consuming Avro data from Kafka with Spark Streaming
Spark Utils
⭐
10
Practical utilities for spark applications
Spark Avro Example
⭐
9
Example of using Avro with Apache Spark
Scabillmatch
⭐
9
Policy diffusion in the US legislature
Query
⭐
7
big data query console command and script for scala
Big_data_training
⭐
6
Avrotoparquet
⭐
6
Command line converter for Apache Avro to Apache Parquet file formats
Hdinsight Kafka Spark
⭐
6
Tutorial for Avro with Kafka and Spark on HDInsight
Hudi
⭐
5
Upserts And Incremental Processing on Big Data
Bullet Record
⭐
5
The generic AVRO record container for plugging in your data into Bullet
Avroparquet
⭐
5
AVRO / Parquet Demo Code
Related Searches
Scala Spark (3,279)
Python Spark (2,053)
Java Spark (1,587)
Jupyter Notebook Spark (1,268)
Apache Spark (1,207)
Spark Hadoop (1,188)
Spark Kafka (985)
Spark Streaming (817)
Spark Pyspark (812)
Docker Spark (683)
1-49 of 49 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.