Scrapy S3pipeline

Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.
Alternatives To Scrapy S3pipeline
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Z3333943 years ago15March 06, 201814apache-2.0Python
Backup your ZFS snapshots to S3.
S3 Multipart149
8 years ago12apache-2.0Python
Utilities to do parallel upload/download with Amazon S3
Scrapy S3pipeline66
12 years ago8January 31, 20211mitPython
Scrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.
Kafka Connect S350
4 years ago13otherJava
A Kafka-Connect Sink for S3 with no Hadoop dependencies.
S3 Multipart Upload Javascript Browser21
6 years ago2JavaScript
AWS S3 Multipart Upload in Browser
S3 Stream Upload2140266 years ago13June 20, 20172mitJavaScript
A writable stream which uploads to Amazon S3 using the multipart file upload API.
Backuppc Archive S317
11 years agootherPython
Python script to archive BackupPC to Amazon S3
S3 Download Stream16449 years ago1January 31, 20151mitJavaScript
This node.js module helps with multipart downloads of objects from aws s3
Kafka S3 Consumer13
11 years ago1September 06, 2013otherGo
Consumes Kafka topics specified in the config, and outputs them in chunks as desired in an S3 Bucket. Keeps track of offsets via S3.
S3 Uploader12
9 years agoNovember 22, 2023Go
Concurrent streaming upload to Amazon S3
Alternatives To Scrapy S3pipeline
Select To Compare


Alternative Project Comparisons
Popular S3 Projects
Popular Chunk Projects
Popular Cloud Computing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Amazon Web Services
Pipeline
Scraper
S3
Chunk
Spider
Scrapy