Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Data Processing Open Source Projects

Open source projects categorized as Python Data Processing
Categories > Data Processing > Data Processing
Dali ⭐ 4,770
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
dependent packages 4total releases 19most recent commit 3 months agopypi nvidia-dali-cuda110} Downloads
Pandera ⭐ 2,807
A light-weight, flexible, and expressive statistical data testing library
dependent packages 97total releases 79most recent commit 3 months agopypi pandera} Downloads
Texar ⭐ 2,008
Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow
total releases 5most recent commit 4 years ago
Bonobo ⭐ 1,548
Extract Transform Load for Python 3.5+
dependent packages 7total releases 38most recent commit a year agopypi bonobo} Downloads
Satpy ⭐ 980
Python package for earth-observing satellite data processing
dependent packages 14total releases 91most recent commit 3 months agopypi satpy} Downloads
Bytewax ⭐ 957
Python Stream Processing
dependent packages 3total releases 34most recent commit 3 months agopypi bytewax} Downloads
Texar Pytorch ⭐ 711
Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation. This is part of the CASL project: http://casl-project.ai/
dependent packages 3total releases 5most recent commit 2 years agopypi texar-pytorch} Downloads
Haupt ⭐ 451
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
dependent packages 1total releases 86most recent commit 15 days agopypi haupt} Downloads
Nonechucks ⭐ 315
Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!
total releases 18most recent commit 3 years ago
Lithops ⭐ 305
A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
dependent packages 3total releases 45most recent commit 2 days agopypi lithops} Downloads
Dolma ⭐ 302
Data and tools for generating and inspecting OLMo pre-training data.
total releases 10most recent commit 3 months ago
Fondant ⭐ 293
Production-ready data processing made easy and shareable
total releases 24most recent commit 3 months ago
Rapidtables ⭐ 284
Super fast list of dicts to pre-formatted tables conversion library for Python 2/3
dependent packages 7total releases 37most recent commit 4 years agopypi rapidtables} Downloads
Pysparkling ⭐ 253
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
dependent packages 1total releases 69most recent commit a year agopypi pysparkling} Downloads
Scramjet ⭐ 243
Public tracker for Scramjet Cloud Platform, a platform that bring data from many environments together.
dependent packages 2total releases 1most recent commit a year agonpm @winton/scramjet} Downloads
Machine Learning Notebooks ⭐ 241
Machine Learning notebooks for refreshing concepts.
most recent commit 3 years ago
50 Days Of Ml ⭐ 237
A day to day plan for this challenge (50 Days of Machine Learning) . Covers both theoretical and practical aspects
most recent commit 4 years ago
Padasip ⭐ 236
Python Adaptive Signal Processing
dependent packages 2total releases 13most recent commit a year agopypi padasip} Downloads
Vaspy ⭐ 220
Manipulating VASP files with Python.
total releases 21most recent commit 2 years ago
Forte ⭐ 215
Forte is a flexible and powerful ML workflow builder. This is part of the CASL project: http://casl-project.ai/
dependent packages 12total releases 14most recent commit a year agopypi forte} Downloads
Batchflow ⭐ 195
BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.
total releases 15most recent commit 3 months ago
Dataflows ⭐ 182
DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.
dependent packages 11total releases 127most recent commit 6 months agopypi dataflows} Downloads
Convtools Ita ⭐ 176
convtools is a python library to declaratively define conversions for processing collections, doing complex aggregations and joins.
most recent commit 3 years ago
The 10 Most Depended On Python Data Processing Open Source Projects
Pandera ⭐ 2,807
A light-weight, flexible, and expressive statistical data testing library
dependent packages 97total releases 79latest release December 08, 2023most recent commit 3 months agopypi pandera} Downloads
Satpy ⭐ 980
Python package for earth-observing satellite data processing
dependent packages 14total releases 91latest release November 29, 2023most recent commit 3 months agopypi satpy} Downloads
Forte ⭐ 215
Forte is a flexible and powerful ML workflow builder. This is part of the CASL project: http://casl-project.ai/
dependent packages 12total releases 14latest release June 29, 2022most recent commit a year agopypi forte} Downloads
Dataflows ⭐ 182
DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.
dependent packages 11total releases 127latest release October 11, 2023most recent commit 6 months agopypi dataflows} Downloads
Bonobo ⭐ 1,548
Extract Transform Load for Python 3.5+
dependent packages 7total releases 38latest release July 20, 2019most recent commit a year agopypi bonobo} Downloads
Rapidtables ⭐ 284
Super fast list of dicts to pre-formatted tables conversion library for Python 2/3
dependent packages 7total releases 37latest release November 17, 2019most recent commit 4 years agopypi rapidtables} Downloads
Salem ⭐ 161
Add geolocalised subsetting, masking, and plotting operations to xarray
dependent packages 5total releases 18latest release January 20, 2023most recent commit a year agopypi salem} Downloads
Invenio Stats ⭐ 8
Statistical data processing and querying for Invenio.
dependent packages 4total releases 27latest release November 09, 2023most recent commit 6 months agopypi invenio-stats} Downloads
Pipe21 ⭐ 10
Simple functional pipes
dependent packages 4total releases 45latest release November 30, 2023most recent commit 4 months agopypi pipe21} Downloads
The 10 Latest Releases In Python Data Processing Open Source Projects
Fondant ⭐ 293
Production-ready data processing made easy and shareable
total releases 24latest release December 12, 2023most recent commit 3 months ago
Haupt ⭐ 451
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
dependent packages 1total releases 86latest release December 10, 2023most recent commit 15 days agopypi haupt} Downloads
Pandera ⭐ 2,807
A light-weight, flexible, and expressive statistical data testing library
dependent packages 97total releases 79latest release December 08, 2023most recent commit 3 months agopypi pandera} Downloads
Pipe21 ⭐ 10
Simple functional pipes
dependent packages 4total releases 45latest release November 30, 2023most recent commit 4 months agopypi pipe21} Downloads
Satpy ⭐ 980
Python package for earth-observing satellite data processing
dependent packages 14total releases 91latest release November 29, 2023most recent commit 3 months agopypi satpy} Downloads
Libertem ⭐ 104
Open pixelated STEM framework
dependent packages 3total releases 25latest release November 09, 2023most recent commit 3 months agopypi libertem} Downloads
Invenio Stats ⭐ 8
Statistical data processing and querying for Invenio.
dependent packages 4total releases 27latest release November 09, 2023most recent commit 6 months agopypi invenio-stats} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.