Awesome Open Source

Programming Languages

The Top 23 Scala Hadoop Open Source Projects

Open source projects categorized as Scala Hadoop

Categories > Data Processing > Hadoop

Categories > Programming Languages > Scala

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46most recent commit 3 months ago

Bigdata Notes ⭐ 14,872

大数据入门指南 ⭐

most recent commit 3 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54most recent commit 23 days ago

It_book ⭐ 8,543

本项目收藏这些年来看过或者听过的一些不错的常用的上千本书籍，没准你想找的书就在这里呢，包含了互联网行

most recent commit 2 years ago

Bigdl ⭐ 4,728

Accelerate LLM with low-bit (FP4 / INT4 / FP8 / INT8) optimizations using bigdl-llm

dependent packages 10total releases 16most recent commit 3 months ago

Tensorflowonspark ⭐ 3,851

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

total releases 32most recent commit 9 months ago

Scalding ⭐ 3,433

A Scala API for Cascading

dependent packages 40total releases 43most recent commit a year ago

Bigdataguide ⭐ 2,355

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

most recent commit 4 months ago

Szt Bigdata ⭐ 2,055

深圳地铁大数据客流分析系统🚇🚄🌟

most recent commit 3 months ago

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11most recent commit 3 months ago

Docker Spark ⭐ 1,783

Apache Spark docker image

most recent commit 2 years ago

Movie_recommend ⭐ 1,441

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

most recent commit 5 years ago

Carbondata ⭐ 1,401

High performance data store solution

total releases 1most recent commit 4 months ago

Data Algorithms Book ⭐ 973

MapReduce, Spark, Java, and Scala for Data Algorithms Book

most recent commit 2 years ago

Livy is an open source REST interface for interacting with Apache Spark from anywhere

dependent packages 5total releases 4most recent commit 4 years ago

Scala extensions for the Kryo serialization library

dependent packages 80total releases 35most recent commit 6 months ago

Data Lineage Tracking And Visualization Solution

dependent packages 6total releases 14most recent commit 4 months ago

Spark Redshift ⭐ 514

Redshift data source for Apache Spark

dependent packages 1total releases 10most recent commit 4 years ago

A Scala productivity framework for Hadoop.

dependent packages 1total releases 37most recent commit 4 years ago

Former GraphX development repository. GraphX has been merged into Apache Spark; please submit pull requests there.

most recent commit a year ago

Sagemaker Spark ⭐ 285

A Spark library for Amazon SageMaker.

total releases 36most recent commit 7 months ago

Sparkonhbase ⭐ 277

most recent commit 3 years ago

Parquet4s ⭐ 267

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

dependent packages 6total releases 57most recent commit 3 months ago

The 10 Most Depended On Scala Hadoop Open Source Projects

Spark ⭐ 37,661

Apache Spark - A unified analytics engine for large-scale data processing

dependent packages 939total releases 46latest release May 09, 2021most recent commit 3 months ago

Deeplearning4j ⭐ 13,397

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

dependent packages 119total releases 54latest release August 10, 2022most recent commit 23 days ago

Scala extensions for the Kryo serialization library

dependent packages 80total releases 35latest release June 04, 2021most recent commit 6 months ago

Scalding ⭐ 3,433

A Scala API for Cascading

dependent packages 40total releases 43latest release September 14, 2016most recent commit a year ago

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11latest release November 03, 2023most recent commit 3 months ago

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

dependent packages 24total releases 65latest release October 16, 2023most recent commit 4 months ago

Eel Sdk ⭐ 140

Big Data Toolkit for the JVM

dependent packages 17total releases 103latest release February 11, 2019most recent commit 3 years ago

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

dependent packages 15total releases 25latest release September 14, 2023most recent commit 7 months ago

Incubator Wayang ⭐ 162

Apache Wayang(incubating) is the first cross-platform data processing system.

dependent packages 13total releases 2latest release September 03, 2023most recent commit 3 months ago

Spark Util ⭐ 16

low-level helpers for Apache Spark libraries and tests

dependent packages 12total releases 15latest release December 29, 2018most recent commit 5 years ago

The 10 Latest Releases In Scala Hadoop Open Source Projects

Parquet4s ⭐ 267

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

dependent packages 6total releases 57latest release November 12, 2023most recent commit 3 months ago

Kyuubi ⭐ 1,849

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

dependent packages 24total releases 11latest release November 03, 2023most recent commit 3 months ago

Smart Data Lake ⭐ 87

Smart Automation Tool for building modern Data Lakes and Data Pipelines

dependent packages 8total releases 26latest release October 25, 2023most recent commit 3 months ago

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

dependent packages 24total releases 65latest release October 16, 2023most recent commit 4 months ago

Sparkplugins ⭐ 70

Code and examples of how to write and deploy Apache Spark Plugins with Spark 3.x. Spark plugins allow runnig custom code on the executors as they are initialized. This also allows extending the Spark metrics systems with user-provided monitoring probes.

total releases 3latest release September 26, 2023most recent commit 5 months ago

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

dependent packages 15total releases 25latest release September 14, 2023most recent commit 7 months ago

Incubator Wayang ⭐ 162

Apache Wayang(incubating) is the first cross-platform data processing system.

dependent packages 13total releases 2latest release September 03, 2023most recent commit 3 months ago

Enceladus ⭐ 28

Dynamic Conformance Engine

dependent packages 4total releases 40latest release March 23, 2023most recent commit 3 months ago

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

total releases 27latest release November 17, 2022most recent commit 9 months ago

Sagemaker Spark ⭐ 285

A Spark library for Amazon SageMaker.

total releases 36latest release August 26, 2022most recent commit 7 months ago

Popular Data Processing Categories

Jupyter Notebook

Categories

Application Programming Interfaces

Artificial Intelligence

Cloud Computing

Command Line Interface

Computer Science

Configuration Management

Content Management

Data Processing

Integrated Development Environments

Learning Resources

Lists Of Projects

Machine Learning

Operating Systems

Package Managers

Programming Languages

Runtime Environments

Software Architecture

Software Development

Software Performance

Software Quality

Text Processing

User Interface Components

Version Control

Web User Interface

Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source. All rights reserved.