Awesome Open Source

Programming Languages

Search results for scala avro

85 search results found

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

Kafka Storm Starter ⭐ 729

[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Avro schema generation and serialization / deserialization for Scala

Sparklearning ⭐ 573

Learning Apache spark,including code and data .Most part can run local.

Spark Avro ⭐ 535

Avro Data Source for Apache Spark

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.

MySQL binary log consumer with the ability to act on changed rows and publish changes to different systems with emphasis on Apache Kafka.

Mu Haskell ⭐ 311

Mu (μ) is a purely functional framework for building micro services.

Avro SerDe for Apache Spark structured APIs.

Trifecta ⭐ 211

Trifecta is a web-based and CLI tool that simplifies inspecting Kafka messages and Zookeeper data. Additionally, the CLI tool provides the capability to import/export data to/from ElasticSearch and MongoDB.

Avrohugger ⭐ 198

Generate Scala case class definitions from Avro schemas

Magnolify ⭐ 155

A collection of Magnolia add-on modules

Bigdata Playground ⭐ 154

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Scala library to eliminate boilerplate

Spark Bigquery ⭐ 149

Google BigQuery support for Spark, SQL, and DataFrames

Eel Sdk ⭐ 140

Big Data Toolkit for the JVM

Sbt Avrohugger ⭐ 129

sbt plugin for generating Scala sources for Apache Avro schemas and protocols.

Kafka Serialization ⭐ 121

Lego bricks to build Apache Kafka serializers and deserializers

Scalavro ⭐ 119

A reflection-based Avro library in Scala.

Avro2tf ⭐ 117

Avro2TF is designed to fill the gap of making users' training data ready to be consumed by deep learning training frameworks.

Kafka Connect Twitter ⭐ 75

Kafka Connect Sink/Source for Twitter

Shapeless Datatype ⭐ 67

Shapeless utilities for common data types

Spark Structured Streaming ⭐ 66

Spark structured streaming with Kafka data source and writing to Cassandra

Avro Parquet Spark Example ⭐ 61

An example of using Avro and Parquet in Spark SQL

Avro Scala Macro Annotations ⭐ 57

Compile-time tools for working with Avros in Scala

Scalable Consistency Adjustable Data Storage

Kafka Scala Examples ⭐ 42

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Etl Light ⭐ 38

A light Kafka to HDFS/S3 ETL library based on Apache Spark

Kafka Message Scheduler ⭐ 32

Scheduler for low-frequency and long-term scheduling of delayed messages to Kafka topics.

Simplesparkavroapp ⭐ 32

Simple Spark app that reads and writes Avro data

Parquet Extra ⭐ 29

A collection of Apache Parquet add-on modules

Sbt Avro ⭐ 28

sbt plugin for compiling Avro schemas, similar to sbt-protobuf

Affinity ⭐ 24

Library and a Framework for building fast, scalable, fault-tolerant Data APIs based on Akka, Avro, ZooKeeper and Kafka

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Salat Avro ⭐ 23

Fast bi-directional Scala case class to Avro serialization

Avro Sql ⭐ 23

Use SQL to transform your avro schema/records

Avro Schema Evolution made easy

Sbt Avro4s ⭐ 20

Sbt plugin for avro4s

Data Model Generator ⭐ 20

Data model generator based on Scala case classes

Dendrites ⭐ 20

Scala library for Reactive streaming Microservices, CQRS, Event Sourcing, Event Logging, & message-driven apps.

Spark Kafka Avro ⭐ 17

POC: Spark consumer for bottledwater-pg Kafka Avro topics

Confluent Spark Avro ⭐ 17

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

Salesforce2hadoop ⭐ 16

Import Salesforce data into Hadoop HDFS in Avro format

Sbt Avro ⭐ 15

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

Spark Bigquery ⭐ 15

Google BigQuery data source for Apache Spark

Spark Dbf ⭐ 14

Spark SQL DBF Library

Gradle Avrohugger Plugin ⭐ 12

Gradle plugin for generating scala case classes from apache avro schemas, datafiles and protocols

Kinesis Tee ⭐ 12

Unix tee, but for Kinesis streams

Spark Streaming Kafka Example ⭐ 11

An example project using Spark Streaming with Kafka message and Avro serialization

Confluent Platform Spark Streaming ⭐ 10

Working example of consuming Avro data from Kafka with Spark Streaming

Zio Kafka Streams ⭐ 10

Kafka Streams for Scala with ZIO

Amazon S3 Tagging Spark Util ⭐ 10

A small project to allow publishing data to Apache Kafka, Apache Pulsar or any other target system

Typebook ⭐ 10

An Avro Schema Registry with MySQL backend.

Scala case classes over 0mq, salat-avro for serialization, akka & lift actor integration

Scabillmatch ⭐ 9

Policy diffusion in the US legislature

Cassandra Summit Demo ⭐ 9

Hadoop integration demo for the Cassandra Summit

Spark Avro Example ⭐ 9

Example of using Avro with Apache Spark

Avrohugger Maven Plugin ⭐ 9

Maven plugin for generating Scala case classes and ADTs from Apache Avro schemas, datafiles, and protocols

Some Avro operations in Scala

Cloudera Cca175 ⭐ 8

CCA Spark and Hadoop Developer Certification

Cloud Storage Extension ⭐ 8

Exasol Cloud Storage Extension for accessing formatted data Avro, Orc and Parquet, on public cloud storage systems

Copy_to_postgres ⭐ 7

Command line tool for fast importing to Postgres. Reads Avro, Json, Csv and provides support for simple mappings.

big data query console command and script for scala

Avro4s Schema Registry Kafka Streams Demo ⭐ 7

Demo application for Avro4s/Schema Registry/Kafka Streams integration

Metrifier ⭐ 7

Comparing HTTP endpoints against mu-rpc services.

Flink Avro State Serialization ⭐ 7

Wsdl2avro ⭐ 7

WSDL2Avro is a Scala library that lets you convert datatypes from a SOAP WSDL to Avro Schemas.

Scio Contrib ⭐ 7

Community-supported add-ons for Scio

Avro Codegen ⭐ 7

Scala code generator for Avro schemas.

Serialization for Scala.

Avro Scala Demo ⭐ 6

Demonstration of using avro4s library in Scala to provide basic Avro serialization

Hdinsight Kafka Spark ⭐ 6

Tutorial for Avro with Kafka and Spark on HDInsight

Kudu From Avro ⭐ 6

A small Command Line tool to create an Kudu table from an Avro schema or from SQL script

Kafka Avro Generator ⭐ 6

Kafkaavroexamples ⭐ 6

Set of repositories where we use Apache Avro with Apache Kafka (producer/consumer/stream/scheme registry)

Blockchain Importer ⭐ 6

Functional, typesafe, well-tested and composable streaming blockchain importer, drop-in solution for Kafka ecosystem.

Akkastreams Kafka Example ⭐ 6

An example project using Akka Streams with Kafka and a Schema Registry to read and write messages in Avro format.

Kafka Connect Opentsdb ⭐ 5

Demo Segmenter ⭐ 5

Demo Kafka Streams Application segmenting events into sessions

Avro2caseclass ⭐ 5

Generate Scala case class definitions from Avro schemas

Freestyle Rpc Examples ⭐ 5

freestyle-rpc sample demos

Scala/Java library to conveniently interact with Avro files stored in Hadoop HDFS.

Related Searches

Scala Sbt (4,178)

Scala Spark (3,279)

Scala Akka (2,120)

Java Scala (1,794)

Scala Play Framework (1,309)

Plugin Scala (1,079)

Scala Kafka (969)

Scala Functional Programming (942)

Scala Scalajs (887)

Docker Scala (728)

1-85 of 85 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.