Learning Spark
学习Spark的代码，关于Spark Core、Spark SQL、Spark Streaming、Spark MLLib
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Spark Testing Base | 1,418 | 25 | 51 | a month ago | 286 | March 05, 2020 | 97 | apache-2.0 | Scala | |
| Base classes to use when writing tests with Spark | ||||||||||
| Streaming Benchmarks | 560 | a year ago | 26 | apache-2.0 | Jupyter Notebook | |||||
| Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, ... | ||||||||||
| Spark Fast Tests | 379 | a month ago | 11 | February 15, 2021 | 27 | mit | Scala | |||
| Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit) | ||||||||||
| Glow | 220 | 1 | 2 months ago | 8 | June 23, 2020 | 54 | apache-2.0 | Scala | ||
| An open-source toolkit for large-scale genomic analysis | ||||||||||
| Example Spark | 197 | 6 years ago | 2 | other | Scala | |||||
| Spark, Spark Streaming and Spark SQL unit testing strategies | ||||||||||
| Kraps Rpc | 152 | 1 | 4 years ago | 1 | August 12, 2017 | 4 | apache-2.0 | Scala | ||
| A RPC framework leveraging Spark RPC module | ||||||||||
| Dl4clj | 101 | 5 years ago | 1 | bsd-2-clause | Clojure | |||||
| Spark Integration Tests | 55 | 8 years ago | 8 | apache-2.0 | Scala | |||||
| Integration tests for Spark | ||||||||||
| Tablasco | 52 | 1 | 2 months ago | 4 | March 12, 2020 | apache-2.0 | Java | |||
| Tablasco is a JUnit rule for comparing tables and Spark module for comparing large data sets | ||||||||||
| Etlflow | 43 | 3 | 2 months ago | 25 | March 26, 2022 | apache-2.0 | Scala | |||
| EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for running complex Auditable workflows which can interact with Google Cloud Platform, AWS, Kubernetes, Databases and more. | ||||||||||
Alternatives To Learning SparkSelect To Compare
Alternative Project Comparisons
Readme
Learning-Spark
学习Spark的代码，关于Spark Core、Spark SQL、Spark Streaming、Spark MLLib
说明
开发环境
- 基于Deepin Linux 15.9版本
- 基于Hadoop2.6、Spark2.4、Scala2.11、java8等
系列环境搭建相关文章，见下方
- 【向Linux迁移记录】Deepin下java、大数据开发环境配置【一】
- 【向Linux迁移记录】Deepin下Python开发环境搭建
- 【向Linux迁移记录】Deepin Linux下快速Hadoop完全分布式集群搭建
- 【向Linux迁移记录】基于Hadoop集群的Hive安装与配置详解
- 【向Linux迁移记录】Deepin Linux下Spark本地模式及基于Yarn的分布式集群环境搭建
- eclipse安装Scala IDE插件及An internal error occurred during: "Computing additional info"报错解决
- Deepin Linux 安装启动scala报错 java.lang.NumberFormatException: For input string: "0x100" 解决
- 更多内容见：【https://blog.csdn.net/lzw2016/ 】【josonle/Coding-Now 】
文件说明
- Spark_With_Scala_Testing 存放平时练习代码
- notes存放笔记
- LearningSpark(1)数据来源.md
- LearningSpark(2)spark-submit可选参数.md
- LearningSpark(3)RDD操作.md
- LearningSpark(4)Spark持久化操作
- LearningSpark(5)Spark共享变量.md
- LearningSpark(6)Spark内核架构剖析.md
- LearningSpark(7)SparkSQL之DataFrame学习（含Row）.md
- LearningSpark(8)RDD如何转化为DataFrame
- LearningSpark(9)SparkSQL数据来源
- RDD如何作为参数传给函数.md
- 判断RDD是否为空
- 高级排序和topN问题.md
- Spark1.x和2.x如何读取和写入csv文件
- Spark DataFrame如何更改列column的类型.md
- 使用JDBC将DataFrame写入mysql.md
- Scala 语法点
- 报错和问题归纳.md
待续
Popular Testing Projects
Popular Spark Projects
Popular Software Quality Categories
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Testing
Scala
Spark