S3 Stream

Akka Streaming Client for S3 and Supporting Libraries
Alternatives To S3 Stream
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Smart_open2,8033201565 days ago54April 24, 202277mitPython
Utils for streaming large files (S3, HDFS, gzip, bz2...)
a year ago4November 19, 202121mitGo
StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
Afs226142 days ago71October 27, 20212apache-2.0Go
Abstract File Storage
Tus Ruby Server188
2 years ago5mitRuby
Ruby server for tus resumable upload protocol
S3 Streaming Upload11932102 years ago26January 21, 20212mitJavaScript
s3-streaming-upload is node.js library that listens to your stream and upload its data to Amazon S3 using ManagedUpload API.
Node.js Amazon S396
12 years ago7mitJavaScript
An S3 library for node.js. Supports automatic md5 of content, streaming, 100-continues, and very large files
4 years ago26apache-2.0Java
kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
S3 Sync7930147 years ago14October 29, 201514otherJavaScript
A streaming interface for uploading multiple files to S3.
Megfile782a day ago23April 26, 202214apache-2.0Python
Megvii FILE Library - Working with Files in Python
Media Services Application Mapper76
a month agon,ullapache-2.0JavaScript
Media Services Application Mapper is a browser-based tool that allows operators to visualize the structure and logical connections among AWS Media Services and supporting services in the cloud. The tool can be used as a top-down resource monitoring tool when integrated with CloudWatch.
Alternatives To S3 Stream
Select To Compare

Alternative Project Comparisons


This project has been merged into Alpakka. The version found here is not actively maintained.


Akka Streaming Client for S3 and Supporting Libraries



A library built around Akka-Http to stream an upload to S3. The initialization request will be sent, incoming stream will be chunked and uploaded, then the completion request is sent.


Add to your build.sbt:

resolvers += Resolver.jcenterRepo
libraryDependencies += "com.bluelabs" %% "s3-stream" % "0.0.3"

Then in your application:

  implicit val system = ActorSystem()
  implicit val mat: Materializer = ActorMaterializer()
  implicit val ec = mat.executionContext

  val creds = AWSCredentials("KEYGOESHERE", "SECRETGOESHERE")
  val stream: S3Stream = new S3Stream(creds)

  val input: Source[ByteString, Future[IOResult]] = StreamConverters.fromInputStream(...whatever) // Or something else to generate a stream of ByteStrings
  val sink: Sink[ByteString, Future[CompleteMultipartUploadResult]] = stream.multipartUpload(S3Location("bucketGoesHere", "keygoeshere"))

  val result: Future[CompleteMultipartUploadResult] = input.runWith(sink)


Sign Akka-HTTP requests using AWS credentials, following the AWS Signature v4 process.


Add to your build.sbt:

resolvers += Resolver.jcenterRepo
libraryDependencies += "com.bluelabs" %% "akka-http-aws" % "0.0.3"

Then in your application:

val credentials = AWSCredentials("KEYGOESHERE", "SECRETGOESHERE")
val signingKey = SigningKey(credentials, CredentialScope(LocalDate.now(), region, "s3"))
val req = HttpRequest(... with assorted params ...)
val signedRequest = Signer.signedRequest(req, signingKey)

Popular S3 Projects
Popular Streaming Projects
Popular Cloud Computing Categories

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.