The Top 23 Python S3 Open Source Projects

Open source projects categorized as Python S3

Categories > Programming Languages > Python

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

dependent packages 11total releases 311most recent commit 3 months ago

Awesome Aws ⭐ 11,773

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

total releases 1most recent commit 6 months ago

S3cmd ⭐ 4,363

Official s3cmd repo -- Command line tool for managing Amazon S3 and CloudFront services

total releases 1most recent commit 3 months ago

Wal E ⭐ 3,408

Continuous Archiving for Postgres

total releases 33most recent commit 4 months ago

Smart_open ⭐ 3,065

Utils for streaming large files (S3, HDFS, gzip, bz2...)

most recent commit a month ago

Bucket Stream ⭐ 1,447

Find interesting Amazon S3 Buckets by watching certificate transparency logs.

most recent commit 4 years ago

S4cmd ⭐ 1,249

Super S3 command line tool

total releases 5most recent commit a year ago

Parliament ⭐ 937

AWS IAM linting library

dependent packages 2total releases 48most recent commit 9 months ago

Bareos ⭐ 908

Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.

dependent packages 2total releases 33most recent commit 3 months ago

Grafana Backup Tool ⭐ 751

A Python-based application to backup Grafana settings by using the Grafana API

total releases 20most recent commit 5 months ago

Dataengineeringproject ⭐ 644

Example end to end data engineering project.

most recent commit a year ago

Oio Sds ⭐ 634

High Performance Software-Defined Object Storage for Big Data and AI, that supports Amazon S3 and Openstack Swift

total releases 2most recent commit 3 months ago

Yas3fs ⭐ 605

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.

total releases 34most recent commit 3 years ago

Django S3direct ⭐ 602

Directly upload files to S3 compatible services with Django.

dependent packages 1total releases 78most recent commit 2 years ago

Goodreads_etl_pipeline ⭐ 593

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

most recent commit 4 years ago

Pypicloud ⭐ 476

S3-backed pypi server implementation

most recent commit a year ago

S3viewer ⭐ 413

Storage Explorer - Publicly open storage viewer (Amazon S3 Bucket, Azure Blob, FTP server, HTTP Index Of/)

most recent commit 6 months ago

S3monkey ⭐ 403

A Python library that allows you to interact with Amazon S3 Buckets as if they are your local filesystem.

total releases 2most recent commit a year ago

Django S3 Storage ⭐ 398

Django Amazon S3 file storage.

dependent packages 5total releases 37most recent commit 5 months ago

Awacs ⭐ 392

Python library for AWS Access Policy Language creation

dependent packages 24total releases 42most recent commit 3 months ago

Paaster ⭐ 387

Paaster is a secure and user-friendly pastebin application that prioritizes privacy and simplicity. With end-to-end encryption and paste history, Paaster ensures that your pasted code remains confidential and accessible.

most recent commit 3 months ago

Cloudpathlib ⭐ 348

Python pathlib-style classes for cloud storage services such as Amazon S3, Azure Blob Storage, and Google Cloud Storage.

dependent packages 20total releases 28most recent commit 4 months ago

Nodb ⭐ 330

NoDB isn't a database.. but it sort of looks like one.

total releases 7most recent commit 4 years ago

The 10 Most Depended On Python S3 Open Source Projects

S3transfer ⭐ 190

Amazon S3 Transfer Manager for Python

dependent packages 431total releases 40latest release December 21, 2023most recent commit 4 months ago

S3fs ⭐ 148

Amazon S3 filesystem for PyFilesystem2

dependent packages 56total releases 16latest release August 14, 2019most recent commit 3 months ago

Pathy ⭐ 169

simple, flexible, offline capable, cloud storage with a Python path-like interface

dependent packages 34total releases 29latest release October 22, 2023most recent commit 3 months ago

Datajoint Python ⭐ 158

Relational data pipelines for the science lab

dependent packages 26total releases 66latest release September 21, 2022most recent commit 4 months ago

Awacs ⭐ 392

Python library for AWS Access Policy Language creation

dependent packages 24total releases 42latest release August 13, 2023most recent commit 3 months ago

Yolov5 Pip ⭐ 273

Packaged version of ultralytics/yolov5 + many extra features

dependent packages 23total releases 69latest release October 16, 2023most recent commit 6 months ago

Cloud Volume ⭐ 120

Read and write Neuroglancer datasets programmatically.

dependent packages 23total releases 319latest release October 19, 2023most recent commit 4 months ago

Tiledb Py ⭐ 167

Python interface to the TileDB storage engine

dependent packages 23total releases 125latest release November 29, 2023most recent commit 3 months ago

Cloudpathlib ⭐ 348

Python pathlib-style classes for cloud storage services such as Amazon S3, Azure Blob Storage, and Google Cloud Storage.

dependent packages 20total releases 28latest release October 10, 2023most recent commit 4 months ago

Python Remotezip ⭐ 94

Python module to access single members of a zip archive without downloading the full content from a remote web server.

dependent packages 14total releases 8latest release January 20, 2023most recent commit 4 months ago

The 10 Latest Releases In Python S3 Open Source Projects

S3transfer ⭐ 190

Amazon S3 Transfer Manager for Python

dependent packages 431total releases 40latest release December 21, 2023most recent commit 4 months ago

S3path ⭐ 191

s3path is a pathlib extension for AWS S3 Service

dependent packages 13total releases 30latest release December 18, 2023most recent commit 3 months ago

Locopy ⭐ 99

locopy: Loading/Unloading to Redshift and Snowflake using Python.

total releases 21latest release December 08, 2023most recent commit 3 months ago

Airbyte ⭐ 12,918

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

dependent packages 11total releases 311latest release December 08, 2023most recent commit 3 months ago

Tiledb Py ⭐ 167

Python interface to the TileDB storage engine

dependent packages 23total releases 125latest release November 29, 2023most recent commit 3 months ago

S3shutil ⭐ 5

Pythonic copy, move and sync for Amazon S3

total releases 10latest release November 27, 2023most recent commit 4 months ago

Megfile ⭐ 99

Megvii FILE Library - Working with Files in Python same as the standard library

dependent packages 2total releases 68latest release November 27, 2023most recent commit 3 months ago

Pysimplestorageservice ⭐ 5

AWS S3 Starage Helper for Python

latest release November 24, 2023most recent commit 8 years ago

Bareos ⭐ 908

Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.

dependent packages 2total releases 33latest release November 21, 2023most recent commit 3 months ago

Synapse S3 Storage Provider ⭐ 98

Synapse storage provider to fetch and store media in Amazon S3

total releases 7latest release November 17, 2023most recent commit 5 months ago

Popular Programming Languages Categories