S3 Streaming Upload

s3-streaming-upload is node.js library that listens to your stream and upload its data to Amazon S3 using ManagedUpload API.
Alternatives To S3 Streaming Upload
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
S3fs Fuse7,208
18 hours ago4March 09, 2022219gpl-2.0C++
FUSE-based file system backed by Amazon S3
3 months agoApril 04, 2020258apache-2.0Go
a high-performance, POSIX-ish Amazon S3 file system written in Go
3 months ago1February 27, 2018276gpl-2.0Python
Official s3cmd repo -- Command line tool for managing Amazon S3 and CloudFront services
Fake S32,905134102 months ago21August 15, 2018119Ruby
A lightweight server clone of Amazon S3 that simulates most of the commands supported by S3 with minimal dependencies
Mc2,46615152 days ago50April 22, 202138agpl-3.0Go
Simple | Fast tool to manage MinIO clusters :cloud:
S3 Uploads1,756439a day ago19July 30, 2021177PHP
The WordPress Plugin to Store Uploads on Amazon S3
Bucket Stream1,447
3 years ago4mitPython
Find interesting Amazon S3 Buckets by watching certificate transparency logs.
5 years ago529apache-2.0JavaScript
The official @github repository of the Trovebox frontend software. A photo sharing and photo management web interface for data stored "in the cloud" (i.e. Amazon S3, Rackspace CloudFiles, Google Storage).
Aws Toolkit Vscode1,119
7 hours ago341apache-2.0TypeScript
CodeWhisperer, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Exec, AWS resources
Amazon S3 Php Class98039102 years ago2April 27, 201568PHP
A standalone Amazon S3 (REST) client for PHP 5/CURL
Alternatives To S3 Streaming Upload
Select To Compare

Alternative Project Comparisons

s3-streaming-upload s3-streaming-upload

s3-streaming-upload is node.js library that listens to your stream and upload its data to Amazon S3 and OCI Bucket Store.

It is heavily inspired by knox-mpu, but unlike it, it does not buffer data to disk and is build on top of official AWS SDK instead of knox.


  • Version 0.3.2 NodeJS 12+ supported.

  • Version 0.3.x Change from Coffee-script to Javascript. NodeJS 6 and 8 supported.

  • Version 0.2.x using ManagedUpload API. NodeJS 0.10 and 0.12 supported.

  • Version 0.1.x using MultiPartUpload API. NodeJS 0.8 and 0.10 supported.


Installation is done via NPM, by running npm install s3-streaming-upload


  • Super easy to use
  • No need to know data size beforehand
  • Stream is buffered up to specified size (default 5MBs) and then uploaded to S3
  • Segments are not written to disk and memory is freed as soon as possible after upload
  • Uploading is asynchronous
  • You can react to upload status through events

Quick example

var Uploader = require('s3-streaming-upload').Uploader,
  upload = null,
  stream = require('fs').createReadStream('/etc/resolv.conf');

upload = new Uploader({
  // credentials to access AWS
  accessKey: process.env.AWS_S3_ACCESS_KEY,
  secretKey: process.env.AWS_S3_SECRET_KEY,
  bucket: process.env.AWS_S3_TEST_BUCKET,
  objectName: 'myUploadedFile',
  stream: stream,
  debug: true,

upload.send(function(err) {
  if (err) {
    console.error('Upload error' + err);

Setting up ACL

Pass it in objectParams to the Uploader:

upload = new Uploader({
  // credentials to access AWS
  accessKey: process.env.AWS_API_KEY,
  secretKey: process.env.AWS_SECRET,
  bucket: process.env.AWS_S3_TRAFFIC_BACKUP_BUCKET,
  objectName: 'myUploadedFile',
  stream: stream,
  objectParams: {
    ACL: 'public-read',

Example usage with Oracle Cloud (OCI) compatible S3 API

region = process.env.OCI_REGION;
tenancy = process.env.OCI_TENANCY;
// define custom service
service = new aws.S3({
  apiVersion: '2006-03-01',
  credentials: {
    accessKeyId: process.env.BUCKET_ACCESS_KEY,
    secretAccessKey: process.env.BUCKET_SECRET_KEY,
  params: { Bucket: process.env.BUCKET_NAME },
  endpoint: `${tenancy}.compat.objectstorage.${region}.oraclecloud.com`,
  region: region,
  signatureVersion: 'v4',
  s3ForcePathStyle: true,

uploader = new Uploader({
  accessKey: process.env.BUCKET_ACCESS_KEY,
  secretKey: process.env.BUCKET_SECRET_KEY,
  bucket: process.env.BUCKET_NAME,
  objectName: filename,
  stream: source,
  service: service,
  objectParams: {
    ContentType: 'text/csv',
  debug: true,
Popular S3 Projects
Popular Amazon Projects
Popular Cloud Computing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.