Awesome Open Source

Programming Languages

Search results for python dask

116 search results found

Dask ⭐ 11,711

Parallel computing with task scheduling

cuDF - GPU DataFrame Library

The flexibility of Python with the scale and performance of modern SQL.

Xarray ⭐ 3,322

N-D labeled arrays and datasets in Python

Stumpy ⭐ 2,901

STUMPY is a powerful and scalable Python library for modern time series analysis

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.

Swifter ⭐ 2,407

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner

Fugue ⭐ 1,821

A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

Distributed ⭐ 1,514

A distributed task scheduler for Dask

Eliot ⭐ 1,074

Eliot: the logging system that tells you *why* it happened

Python package for earth-observing satellite data processing

Mlforecast ⭐ 635

Scalable machine 🤖 learning for time series forecasting.

Pystore ⭐ 404

Fast data store for Pandas time-series data

Dask Sql ⭐ 350

Distributed SQL Engine in Python using Dask

Datacompy ⭐ 339

Pandas and Spark DataFrame comparison for humans and more!

Pyresample ⭐ 320

Geospatial image resampling in Python

Hypergbm ⭐ 306

A full pipeline AutoML tool for tabular data

Library of derived climate variables, ie climate indicators, based on xarray.

🪴 Nebari - your open source data science platform

Paperboy ⭐ 231

A web frontend for scheduling Jupyter notebook reports

Dask Jobqueue ⭐ 229

Deploy Dask on job schedulers like PBS, SLURM, and SGE

Merlin Models is a collection of deep learning recommender system model reference implementations

Amazon Sagemaker Local Mode ⭐ 220

Amazon SageMaker Local Mode Examples

Climpred ⭐ 213

🌎 Verification of weather and climate forecasts 🌍

Universal Regridder for Geospatial Data

Stackstac ⭐ 212

Turn a STAC catalog into a dask-based xarray

Aicsimageio ⭐ 177

Image Reading, Metadata Conversion, and Image Writing for Microscopy Images in Python

Kartothek ⭐ 163

A consistent table management library in python

Geowombat ⭐ 154

GeoWombat: Utilities for geospatial data

On-Demand Earth System Data Cubes (ESDCs) in Python

Bumblebee ⭐ 124

🚕 A spreadsheet-like data preparation web app that works over Optimus (Pandas, Dask, cuDF, Dask-cuDF, Spark and Vaex)

Xgboost_ray ⭐ 121

Distributed XGBoost on Ray

Fast & furious GroupBy operations for dask.array

Xarray Beam ⭐ 110

Distributed Xarray with Apache Beam

Autoray ⭐ 106

Abstract your array operations.

Dask Ec2 ⭐ 104

Start a cluster in EC2 for dask.distributed

Geoxarray ⭐ 94

Geolocation utilities for xarray

Timeeval ⭐ 68

Evaluation Tool for Anomaly Detection Algorithms on Time Series

Summarise and explore Pandas DataFrames

Ncar Python Tutorial ⭐ 63

Numerical & Scientific Computing with Python Tutorial

Parse and read ctl and associated binary file commonly used by GrADS into xarray

Framequery ⭐ 59

SQL on dataframes - pandas and dask

Dask Rasterio ⭐ 58

Read and write rasters in parallel using Rasterio and Dask

Graphchain ⭐ 56

⚡️ An efficient cache for the execution of dask graphs.

Dask Awkward ⭐ 54

Native Dask collection for awkward arrays, and the library to use it.

Containerized distributed programming framework for Python

Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead

Read MITgcm mds binary files into xarray

Lazycluster ⭐ 43

🎛 Distributed machine learning made simple.

Big Data ⭐ 37

Python tools for big data

Dask Scaling Dataframe ⭐ 36

Python and Dask: Scaling the Dataframe

Dask Deltatable ⭐ 34

A Delta Lake reader for Dask

Madpy Dask ⭐ 34

MadPy Dask talk materials

oocgcm is a python library for the analysis of large gridded geophysical dataset.

Cesm Lens Aws ⭐ 29

Examples of analysis of CESM LENS data publicly available on Amazon S3 (us-west-2 region) using xarray and dask

Apache Mesos backend for Dask scheduling library

Arboreto ⭐ 27

A scalable python-based framework for gene regulatory network inference using tree-based ensemble regressors.

Gaia is a geospatial analysis library jointly developed by Kitware and Epidemico.

Daskperiment ⭐ 27

Reproducibility for Humans: A lightweight tool to perform reproducible machine learning experiment.

Distributed data processing routines for multidimensional photoemission spectroscopy (MPES)

Python For Hpc ⭐ 25

Repository for participants of the "Python for HPC" training

Mdio Python ⭐ 24

Cloud native, scalable storage engine for various types of energy data.

Adaptive Scheduler ⭐ 24

Run many functions (adaptively) on many cores (>10k) using mpi4py.futures, ipyparallel, loky, or dask-mpi. 🎉

Dask Snowflake ⭐ 24

Dask integration for Snowflake

Earth System Model Lab (esmlab). ⚠️⚠️ ESMLab functionality has been moved into <https://github.com/NCAR/geocat-comp>. ⚠️⚠️

Daskmaskrcnn ⭐ 22

Running Mask-RCNN on Dask with PyTorch

ByteHub: making feature stores simple

Synthetic Aperture Radar (SAR) Level-1 GRD python mapper for efficient xarray/dask based processing

Dask Histogram ⭐ 21

Histograms with task scheduling.

Geospatial Data Analysis Python ⭐ 21

This repo contain the most common tools used in geospatial analysis using python!

Awesome Pandas Alternatives ⭐ 19

Awesome list of alternative dataframe libraries in Python.

MerCat: python code for versatile k-mer counting and diversity estimation for database independent property analysis for meta -ome data

Coffea Casa ⭐ 16

Repository with configuration setup of a prototype of analysis facility - "coffea-casa"

A scalable, declarative, low-code framework for real-time and batch feature calculation/management (quant finance, anomaly/fraud detection, etc.), predictive ML training/inference and simulation. Built on top of Ray

Prefect Saturn ⭐ 15

Python client for using Prefect Cloud with Saturn Cloud

Lexcube: 3D Data Cube Visualization in Jupyter Notebooks

Stemgraphic ⭐ 14

stemgraphic python package for visualization of data and text

Pangeo Binder ⭐ 13

Pangeo + Binder (dev repo for a binder/pangeo fusion concept)

Dvc_dask_use_case ⭐ 13

A use case of a reproducible machine learning pipeline using Dask, DVC, and MLflow.

Codex Africanus ⭐ 13

Radio Astronomy Algorithms Library

Pdal Parallelizer ⭐ 12

A python app (cli/api) to parallelize your PDAL pipelines

Ngff Zarr ⭐ 12

A lean and kind Open Microscopy Environment (OME) Next Generation File Format (NGFF) Zarr implementation.

Dask Saturn ⭐ 12

Python library for interacting with Dask clusters in Saturn

Deep Learning Toolkit ⭐ 11

Deep Learning Toolkit for Splunk

Portfolio ⭐ 11

Self-taught data analyst.

Dask Memusage ⭐ 11

A low-impact profiler to figure out how much memory each task in Dask is using

Pywrangler ⭐ 11

Advanced data wrangling for python

Zcollection ⭐ 10

Python library allowing to manipulate data splited into a collection of groups stored in Zarr format.

Ncar Jobqueue ⭐ 10

Utilities for configuring dask-jobqueue with appropriate settings for NCAR clusters

Loop like a pro, make parameter studies fun.

Contentwise Impressions ⭐ 9

This repository contains the code used to run generate the data splits, run the hyperparameter tunings, and export the results presented in our article "ContentWise Impressions: An industrial dataset with impressions included"

Pulsarbat ⭐ 9

pulsarbat (PULSAR Baseband Analysis Tools) is a Python package for analysis of radio baseband observations of pulsars.

Examples of the Python programming language

Astrohack ⭐ 9

Climate_python_intro ⭐ 8

Dasf Core ⭐ 8

Framework for computing Machine Learning algorithms in Python using Dask and RAPIDS AI.

Image reading, metadata management, and image writing for Microscopy images in Python

Daskkubernetes ⭐ 8

A simple demonstration on how to use Dask and Kubernetes to score Deep Learning models in parallel

scientific functions to compute radar or geophysical parameters from satellite images over ocean

Old Dask Yarn ⭐ 7

Deprecated, please use https://github.com/jcrist/skein or https://github.com/dask/dask-yarn instead

Related Searches

Python Machine Learning (20,195)

Python Dataset (14,792)

Python Docker (14,113)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Jupyter Notebook (12,976)

Python Network (11,495)

Python Algorithms (10,033)

Python Pytorch (7,877)

Python Server (7,793)

1-100 of 116 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.