Awesome Open Source
Awesome Open Source
Combined Topics
spark-sql
x
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
The Top 270 Spark Sql Open Source Projects on Github
Categories
>
Data Processing
>
Spark Sql
Redash
⭐
19,708
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
Quill
⭐
1,947
Compile-time Language Integrated Queries for Scala
Spark
⭐
1,695
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Almond
⭐
1,400
A Scala kernel for Jupyter
Incubator Kyuubi
⭐
778
Apache Kyuubi is a distributed multi-tenant JDBC server for large-scale data processing and analytics, built on top of Apache Spark
Useractionanalyzeplatform
⭐
581
电商用户行为分析大数据平台
Sparta
⭐
521
Real Time Analytics and Data Pipelines based on Spark Streaming
Magellan
⭐
509
Geo Spatial Data Analytics on Spark
Sparklens
⭐
416
Qubole Sparklens tool for performance tuning Apache Spark
Learningspark
⭐
406
Scala examples for learning to use Spark
Learningsparkv2
⭐
346
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
Mastering Spark Sql Book
⭐
288
The Internals of Spark SQL
Spark Druid Olap
⭐
283
Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.
Iql
⭐
276
An ad hoc query service based on the spark sql engine.(基于spark sql引擎的即席查询服务)
Data Accelerator
⭐
257
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Cuelake
⭐
240
Use SQL to build ELT pipelines on a data lakehouse.
Spark Workshop
⭐
212
Apache Spark™ and Scala Workshops
Cc Pyspark
⭐
159
Process Common Crawl data with Python and Spark
Rasterframes
⭐
158
Geospatial Raster support for Spark DataFrames
Mcw Big Data And Visualization
⭐
157
MCW Big data and visualization
Bigdata Playground
⭐
154
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Spark Structured Streaming Examples
⭐
153
Spark Structured Streaming / Kafka / Cassandra / Elastic
Opaque Sql
⭐
148
An encrypted data analytics platform
Pyspark Cheatsheet
⭐
121
🐍 Quick reference guide to common patterns & functions in PySpark.
Spring Shiro Spark
⭐
112
Spring-Shiro-Spark是Spring-Boot Hibernate Spark Spark-SQL Shiro iView VueJs... ...的集成尝试
Pulsar Spark
⭐
74
When Apache Pulsar meets Apache Spark
Sparksql Protobuf
⭐
73
Read SparkSQL parquet file as RDD[Protobuf]
Cleanframes
⭐
70
type-class based data cleansing library for Apache Spark SQL
Dt Sql Parser
⭐
60
SQL Parsers for BigData, built with antlr4.
Spark Records
⭐
58
Bulletproof Apache Spark jobs with fast root cause analysis of failures.
Recommendmoteur
⭐
57
电影推荐系统、电影推荐引擎、使用Spark完成的电影推荐引擎
Spark
⭐
55
Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .
Spark Select
⭐
53
A library for Spark DataFrame using MinIO Select API
Awesome Pulsar
⭐
53
A curated list of Pulsar tools, integrations and resources.
Spark Google Spreadsheets
⭐
46
Google Spreadsheets datasource for SparkSQL and DataFrames
Geospark
⭐
44
bring sf to spark in production
Struct Type Encoder
⭐
41
Deriving Spark DataFrame schemas from case classes
Sparkoptics
⭐
40
Optics for Spark DataFrames
Paraflow
⭐
36
A real-time analytical system for ID-associated data
Demoapps
⭐
36
demo applications that show how to deploy offline feature engineering solutions to online in one minute with fedb and nativespark
Sparkdemo
⭐
34
spark全示例代码(java、scala) Spark most full instance code DEMO (java、scala)
Airbnb Spark Thrift
⭐
34
A library for loadling Thrift data into Spark SQL
Spark Twitter Sentiment Analysis
⭐
33
Sentiment Analysis of a Twitter Topic with Spark Structured Streaming
Spark_learning
⭐
32
尚硅谷大数据Spark-2019版最新 Spark 学习
Sope
⭐
31
Apache Spark ETL Utilities
Big_data
⭐
27
A collection of tutorials on Hadoop, MapReduce, Spark, Docker
Cloud Based Sql Engine Using Spark
⭐
25
Cloud-based SQL engine using SPARK where data is accessible as JDBC/ODBC data source via Spark ThriftServer.
Hadooplearning
⭐
25
全套大数据基础学习教程，包含最基础的centos、maven。大数据主要包含hdfs、mr、yarn、hbase、kafka、scala、sparkcore、sparkstreaming、sparksql。教程包含所有的源代码演示以及在线文档说明。
Spark Postgres
⭐
24
PostgreSQL and GreenPlum Data Source for Apache Spark
Datalake Etl Pipeline
⭐
24
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Movies Analytics In Spark And Scala
⭐
24
Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.
Recommendsystem
⭐
24
电影推荐系统
Spark Data Sources
⭐
22
Developing Spark External Data Sources using the V2 API
Awesome Sparklyr
⭐
22
An awesome sparklyr related package collection
Pre Lt Raster Frames
⭐
21
Spark DataFrames for earth observation data
Sparkplug
⭐
20
Spark package to "plug" holes in data using SQL based rules ⚡️ 🔌
Sparkprogramminginscala
⭐
18
Apache Spark Course Material
Spark Structured Streaming Examples
⭐
18
Spark structured streaming examples with using of version 2.4.4
Spark And Mllib Projects
⭐
18
This repository contains Spark, MLlib, PySpark and Dataframes projects
Bigdatatutorial
⭐
18
bigdatatutorial
Albis
⭐
17
Albis: High-Performance File Format for Big Data Systems
Resume Bjkonglu
⭐
17
记录Spark、Flink研究经验
Spark Ranger
⭐
17
ACL Management for Apache Spark SQL with Apache Ranger. This library has been contributed to https://github.com/apache/submarine as a sub-module, and that module can still be used individually. The project here will no longer be updated. If you have any questions please go to https://github.com/apache/submarine/tree/master/docs/submarine-security/spark-security/README.md to learn how to use and give feedback to the apache submarine community by following https://submarine.apache.org/community/contributors.html
Recsys_spark
⭐
16
Spark SQL 实现 ItemCF，UserCF，Swing，推荐系统，协同过滤
Litemall Dw
⭐
16
基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。
Spark2 Etl Examples
⭐
16
A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0
Tweet Analysis With Kafka And Spark
⭐
15
A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.
Spark Vcf
⭐
15
Spark VCF data source implementation for Dataframes
Qbeast Spark
⭐
14
Qbeast-spark: DataSource enabling multi-dimensional indexing and efficient data sampling. Big Data, free from the unnecessary!
Spark Fits
⭐
14
FITS data source for Spark SQL and DataFrames
Bdp
⭐
14
A Big Data Platform Prototype Project
Ox Clo
⭐
13
Materials for Oxford Software Engineering Programme CLO course
Wow Spark
⭐
13
🔆 spark自学手册，包含了例如spark core、spark sql、spark streaming、spark-kafka、delta-lake，以及scala基础练习，还有一些例如master、shuffle源码分析，总结及翻译。
Spark Druid Connector
⭐
12
A library for querying Druid data sources with Apache Spark
Hive Jdbc Proxy
⭐
11
Hive-JDBC-Proxy是一个高性能的HiveServer2和Spark ThriftServer的代理服务，具备负载均衡、基于规则转发Hive JDBC Client的请求给到HiveServer2和Spark ThriftServer的能力。
Spark Example
⭐
11
Spark1.6和spark2.2的示例，包含kafka,flume,structuredstreaming,jedis,elasticsearch,mysql,dataframe
Databricks Notebooks
⭐
11
Collection of Databricks and Jupyter Notebooks
Real Time Data Warehouse
⭐
10
Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi
Huemul Bigdatagovernance
⭐
10
Huemul BigDataGovernance, es una framework que trabaja sobre Spark, Hive y HDFS. Permite la implementación de una estrategia corporativa de dato único, basada en buenas prácticas de Gobierno de Datos. Permite implementar tablas con control de Primary Key y Foreing Key al insertar y actualizar datos utilizando la librería, Validación de nulos, largos de textos, máximos/mínimos de números y fechas, valores únicos y valores por default. También permite clasificar los campos en aplicabilidad de derechos ARCO para facilitar la implementación de leyes de protección de datos tipo GDPR, identificar los niveles de seguridad y si se está aplicando algún tipo de encriptación. Adicionalmente permite agregar reglas de validación más complejas sobre la misma tabla.
Java_spark
⭐
10
Spark 2.x 案例操作：Scala版本与 Java1.8lambda版代码示例。涵盖Spark核心技术操作SparkCore、SparkSql、SparkStreaming。同时提供了Spark高级性能优化、序列化、广播变量、数据倾斜、算子优化、JVM优化、troubleshooting、数据倾斜解决方案。是多年来根据工作积累整理出来！
Restaurantinspectionssparkmlnet
⭐
9
ETL & Data Enrichment with Spark.NET and ML.NET Automated (Auto) ML
Bigdata Examples
⭐
9
bigdata examples about spark and flink
Log Analysis System
⭐
8
基于Spark的行为日志分析系统
Apache Spark Etl Pipeline Example
⭐
8
Demonstration of using Apache Spark to build robust ETL pipelines while taking advantage of open source, general purpose cluster computing.
Db2 Event Store Akka Streams
⭐
7
Use Akka to implement a WebSockets endpoint and stream data to Db2 Event Store
Sparksql Dashboard
⭐
7
Provides a cherrypy dashboard to bind access to Spark SQL.
Cca175
⭐
7
R For Hpc And Big Data
⭐
7
Slides and lab material for the talk R for HPC and big data at http://rsummer.data-analysis.at
Sparkathon
⭐
7
A library having Java and Scala examples for Spark 2.x
Spark Machine Learning
⭐
7
spark机器学习
Score
⭐
7
ScORe - Programmatic Schema On Read for Spark SQL, powered by Taboola
Distributable_docker_sql_on_hadoop
⭐
6
Toy Hadoop cluster combining various SQL-on-Hadoop variants
Loganalysis
⭐
6
日志分析项目
Spark Slowly Changing Dimension
⭐
6
Spark implementation of Slowly Changing Dimension type 2
Spark Connector Test
⭐
6
A tutorial on how to use pulsar-spark-connector
Impala Spark Jdbc Kerberos
⭐
6
Spark2 Template
⭐
6
Intellij template to develop Apache Spark 2.x applications
Kotlin Apache Spark Example
⭐
5
kotlin-apache-spark-example
Bdm Project 2
⭐
5
Spark Streaming In Python
⭐
5
Apache Spark 3 - Structured Streaming Course Material
1-100 of 270 projects
Next >
Related Projects
Scala Spark Sql Projects (111)
Python Spark Sql Projects (50)
Java Spark Sql Projects (49)
Pyspark Spark Sql Projects (43)
Jupyter Notebook Spark Sql Projects (36)
Apache Spark Sql Projects (10)
Spark Streaming Sql Projects (2)
Advertising
📦 9
All Projects
Application Programming Interfaces
📦 120
Applications
📦 181
Artificial Intelligence
📦 72
Blockchain
📦 70
Build Tools
📦 111
Cloud Computing
📦 79
Code Quality
📦 28
Collaboration
📦 30
Command Line Interface
📦 48
Community
📦 81
Companies
📦 60
Compilers
📦 60
Computer Science
📦 74
Configuration Management
📦 39
Content Management
📦 167
Control Flow
📦 197
Data Formats
📦 77
Data Processing
📦 266
Data Storage
📦 132
Economics
📦 60
Frameworks
📦 198
Games
📦 122
Graphics
📦 103
Hardware
📦 148
Integrated Development Environments
📦 47
Learning Resources
📦 147
Legal
📦 28
Libraries
📦 119
Lists Of Projects
📦 21
Machine Learning
📦 336
Mapping
📦 61
Marketing
📦 15
Mathematics
📦 55
Media
📦 228
Messaging
📦 97
Networking
📦 304
Operating Systems
📦 84
Operations
📦 120
Package Managers
📦 52
Programming Languages
📦 229
Runtime Environments
📦 96
Science
📦 42
Security
📦 375
Social Media
📦 26
Software Architecture
📦 70
Software Development
📦 68
Software Performance
📦 57
Software Quality
📦 127
Text Editors
📦 45
Text Processing
📦 131
User Interface
📦 310
User Interface Components
📦 465
Version Control
📦 29
Virtualization
📦 68
Web Browsers
📦 38
Web Servers
📦 25
Web User Interface
📦 194
"GitHub" is a registered trademark of GitHub, Inc. Awesome Open Source is not affiliated with GitHub.
