Awesome Open Source
Awesome Open Source
Combined Topics
parquet x
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210

The Top 34 Parquet Open Source Projects

Categories > Data Processing > Parquet
Gaffer ⭐ 1,600
A large-scale entity and relation database supporting aggregation of properties
Parquet Mr ⭐ 1,250
Apache Parquet
Petastorm ⭐ 1,055
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Quilt ⭐ 984
Quilt is a self-organizing data hub for S3
Parquet Format ⭐ 772
Apache Parquet
Iceberg ⭐ 387
Iceberg is a table format for large, slow-moving tabular data
Skale ⭐ 386
High performance distributed data processing engine
Devops Python Tools ⭐ 380
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Choetl ⭐ 356
ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml formatted files)
Parquet Cpp ⭐ 334
Apache Parquet
Oap ⭐ 327
Optimized Analytics Package for Spark* Platform
Pystore ⭐ 304
Fast data store for Pandas time-series data
Elasticsearch_loader ⭐ 292
A tool for batch loading data files (json, parquet, csv, tsv) into ElasticSearch
Ratatool ⭐ 275
A tool for data sampling, data generation, and data diffing
Parquet Dotnet ⭐ 267
🏐 Apache Parquet for modern .NET
Awkward 0.x ⭐ 214
Manipulate arrays of complex data structures as easily as Numpy.
Parquetjs ⭐ 195
fully asynchronous, pure JavaScript implementation of the Parquet file format
Bigdata Playground ⭐ 173
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Sqlite Parquet Vtable ⭐ 159
A SQLite vtable extension to read Parquet files
Parquet Rs ⭐ 144
Apache Parquet implementation in Rust
Eel Sdk ⭐ 140
Big Data Toolkit for the JVM
Kartothek ⭐ 138
A consistent table management library in python
Parquetviewer ⭐ 134
Simple windows desktop application for viewing & querying Apache Parquet files
Parquet4s ⭐ 117
Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.
Parquet Index ⭐ 107
Spark SQL index for Parquet tables
Parquet Go ⭐ 103
Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.
Amazon S3 Find And Forget ⭐ 101
Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)
Schemer ⭐ 95
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Sparksql Protobuf ⭐ 82
Read SparkSQL parquet file as RDD[Protobuf]
Gcs Tools ⭐ 56
GCS support for avro-tools, parquet-tools and protobuf
Rumble ⭐ 53
⛈️ Rumble 1.10.0 "Buttonwood"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Node Parquet ⭐ 45
NodeJS module to access apache parquet format files
Pucket ⭐ 29
Bucketing and partitioning system for Parquet
Parquet Generator ⭐ 16
Parquet file generator
1-34 of 34 projects
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210