Spark Redis

A connector for Spark that allows reading and writing to/from Redis cluster
Alternatives To Spark Redis
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Flink Learning13,198
a month agoapache-2.0Java
flink learning blog. 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例,还有 Flink 落地应用的大型项目案例(PVUV、日志存储、百亿数据实时去重、监控告警)分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
Dev Setup5,802
7 months ago34otherPython
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
Technical Books4,578
8 months agocc-by-sa-4.0HTML
😆 国内外互联网技术大牛们都写了哪些书籍:计算机基础、网络、前端、后端、数据库、架构、大数据、深度学习...
5 months ago85July 18, 20171apache-2.0Jsonnet
PipelineAI Kubeflow Distribution
Szt Bigdata1,702
4 months ago15otherScala
7 days ago40Shell
后端 (Java Golang)全栈知识架构体系总结
Spark Redis88516 months ago10March 09, 2020121bsd-3-clauseScala
A connector for Spark that allows reading and writing to/from Redis cluster
a year ago21Java
定期更新Hadoop生态圈中常用大数据组件文档 重心依次为: Flink Solr Sparksql ES Scala Kafka Hbase/phoenix Redis Kerberos (项目包含hadoop思维导图 印象笔记 Scala版本简单demo 常用工具类 去敏后的train code 持续更新!!!)
3 years agoScala
awesome 三维数据挖掘 数据分析 & 推荐
Freestyle615344 years ago19June 07, 201838apache-2.0Scala
A cohesive & pragmatic framework of FP centric Scala libraries
Alternatives To Spark Redis
Select To Compare

Alternative Project Comparisons

license Release CircleCI Maven Central Javadocs Discord Codecov


A library for reading and writing data in Redis using Apache Spark.

Spark-Redis provides access to all of Redis' data structures - String, Hash, List, Set and Sorted Set - from Spark as RDDs. It also supports reading and writing with DataFrames and Spark SQL syntax.

The library can be used both with Redis stand-alone as well as clustered databases. When used with Redis cluster, Spark-Redis is aware of its partitioning scheme and adjusts in response to resharding and node failure events.

Spark-Redis also supports Spark Streaming (DStreams) and Structured Streaming.

Version compatibility and branching

The library has several branches, each corresponds to a different supported Spark version. For example, 'branch-2.3' works with any Spark 2.3.x version. The master branch contains the recent development for the next release.

Spark-Redis Spark Redis Supported Scala Versions
master 3.2.x >=2.9.0 2.12
3.0 3.0.x >=2.9.0 2.12
2.4, 2.5, 2.6 2.4.x >=2.9.0 2.11, 2.12
2.3 2.3.x >=2.9.0 2.11
1.4 1.4.x 2.10

Known limitations

  • Java, Python and R API bindings are not provided at this time

Additional considerations

This library is a work in progress so the API may change before the official release.


Please make sure you use documentation from the correct branch (2.4, 2.3, etc).


You're encouraged to contribute to the Spark-Redis project.

There are two ways you can do so:

Submit Issues

If you encounter an issue while using the library, please report it via the project's issues tracker.

Author Pull Requests

Code contributions to the Spark-Redis project can be made using pull requests. To submit a pull request:

  1. Fork this project.
  2. Make and commit your changes.
  3. Submit your changes as a pull request.
Popular Redis Projects
Popular Spark Projects
Popular Data Storage Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Redis Cluster