Awesome Open Source
Awesome Open Source
Combined Topics
parquet x
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210

The Top 39 Parquet Open Source Projects

Categories > Data Processing > Parquet
Gaffer ⭐ 1,615
A large-scale entity and relation database supporting aggregation of properties
Drill ⭐ 1,532
Apache Drill is a distributed MPP query layer for self describing data
Parquet Mr ⭐ 1,318
Apache Parquet
Petastorm ⭐ 1,146
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Quilt ⭐ 1,036
Quilt is a self-organizing data hub for S3
Parquet Format ⭐ 833
Apache Parquet
Devops Python Tools ⭐ 439
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Iceberg ⭐ 398
Iceberg is a table format for large, slow-moving tabular data
Skale ⭐ 391
High performance distributed data processing engine
Choetl ⭐ 387
ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Oap ⭐ 355
Optimized Analytics Package for Spark* Platform
Parquet Cpp ⭐ 342
Apache Parquet
Pystore ⭐ 336
Fast data store for Pandas time-series data
Elasticsearch_loader ⭐ 306
A tool for batch loading data files (json, parquet, csv, tsv) into ElasticSearch
Roapi ⭐ 294
Create full-fledged APIs for static datasets without writing a single line of code.
Ratatool ⭐ 284
A tool for data sampling, data generation, and data diffing
Parquet Dotnet ⭐ 282
🏐 Apache Parquet for modern .NET
Vscode Data Preview ⭐ 266
Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
Awkward 0.x ⭐ 216
Manipulate arrays of complex data structures as easily as Numpy.
Parquetjs ⭐ 209
fully asynchronous, pure JavaScript implementation of the Parquet file format
Bigdata Playground ⭐ 180
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Sqlite Parquet Vtable ⭐ 171
A SQLite vtable extension to read Parquet files
Parquetviewer ⭐ 165
Simple windows desktop application for viewing & querying Apache Parquet files
Kartothek ⭐ 149
A consistent table management library in python
Kglab ⭐ 148
Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.
Parquet Rs ⭐ 145
Apache Parquet implementation in Rust
Eel Sdk ⭐ 140
Big Data Toolkit for the JVM
Parquet4s ⭐ 135
Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.
Parquet Go ⭐ 125
Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.
Amazon S3 Find And Forget ⭐ 122
Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)
Parquet Index ⭐ 113
Spark SQL index for Parquet tables
Bigdata File Viewer ⭐ 107
A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
Schemer ⭐ 99
Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Sparksql Protobuf ⭐ 84
Read SparkSQL parquet file as RDD[Protobuf]
Gcs Tools ⭐ 60
GCS support for avro-tools, parquet-tools and protobuf
Rumble ⭐ 60
⛈️ Rumble 1.12.0 "Ashoka Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Node Parquet ⭐ 47
NodeJS module to access apache parquet format files
Pucket ⭐ 29
Bucketing and partitioning system for Parquet
Parquet Generator ⭐ 18
Parquet file generator
1-39 of 39 projects
Advertising 📦 10
All Projects
Application Programming Interfaces 📦 124
Applications 📦 192
Artificial Intelligence 📦 78
Blockchain 📦 73
Build Tools 📦 113
Cloud Computing 📦 80
Code Quality 📦 28
Collaboration 📦 32
Command Line Interface 📦 49
Community 📦 83
Companies 📦 60
Compilers 📦 63
Computer Science 📦 80
Configuration Management 📦 42
Content Management 📦 175
Control Flow 📦 213
Data Formats 📦 78
Data Processing 📦 276
Data Storage 📦 135
Economics 📦 64
Frameworks 📦 215
Games 📦 129
Graphics 📦 110
Hardware 📦 152
Integrated Development Environments 📦 49
Learning Resources 📦 166
Legal 📦 29
Libraries 📦 129
Lists Of Projects 📦 22
Machine Learning 📦 347
Mapping 📦 64
Marketing 📦 15
Mathematics 📦 55
Media 📦 239
Messaging 📦 98
Networking 📦 315
Operating Systems 📦 89
Operations 📦 121
Package Managers 📦 55
Programming Languages 📦 245
Runtime Environments 📦 100
Science 📦 42
Security 📦 396
Social Media 📦 27
Software Architecture 📦 72
Software Development 📦 72
Software Performance 📦 58
Software Quality 📦 133
Text Editors 📦 49
Text Processing 📦 136
User Interface 📦 330
User Interface Components 📦 514
Version Control 📦 30
Virtualization 📦 71
Web Browsers 📦 42
Web Servers 📦 26
Web User Interface 📦 210