Spark Data Sources
Developing Spark External Data Sources using the V2 API
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Iceberg | 5,179 | 3 months ago | 3 | October 29, 2022 | 1,485 | apache-2.0 | Java | |||
| Apache Iceberg | ||||||||||
| Nessie | 762 | 32 | 3 months ago | 40 | November 21, 2023 | 110 | apache-2.0 | Java | ||
| Nessie: Transactional Catalog for Data Lakes with Git-like semantics | ||||||||||
| Iceberg | 409 | 3 years ago | 27 | apache-2.0 | Java | |||||
| Iceberg is a table format for large, slow-moving tabular data | ||||||||||
| Connectors | 383 | 9 months ago | 5 | December 06, 2022 | apache-2.0 | Java | ||||
| This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake. | ||||||||||
| Parquet Index | 113 | 3 years ago | 16 | apache-2.0 | Scala | |||||
| Spark SQL index for Parquet tables | ||||||||||
| Tpch Spark | 91 | 3 months ago | 1 | mit | C | |||||
| TPC-H queries in Apache Spark SQL using native DataFrames API | ||||||||||
| Spark Dynamodb | 90 | 3 years ago | 12 | March 21, 2018 | 17 | apache-2.0 | Scala | |||
| DynamoDB data source for Apache Spark | ||||||||||
| Flowman | 85 | 24 | 4 months ago | 65 | October 16, 2023 | 55 | apache-2.0 | Scala | ||
| Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines. | ||||||||||
| Spark Llap | 82 | 4 years ago | 31 | apache-2.0 | Java | |||||
| Spark Acid | 79 | 3 years ago | 19 | apache-2.0 | Scala | |||||
| ACID Data Source for Apache Spark based on Hive ACID | ||||||||||
Alternatives To Spark Data SourcesSelect To Compare
Alternative Project Comparisons
Popular Spark Projects
Popular Table Projects
Popular Data Processing Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Java
Table
Spark