SFTP Server designed to store data in Google Cloud Storage (GCS) Buckets
This is based upon atmoz/sftp project.
We need to setup an image (based on atomz/sftp) so that we can mount to Google Cloud Storage. That means just installing gcsfuse.
Find and build your own image using the
We use gcsfuse
--only-dir arguments to mount each SFTP users home directory to a single bucket. Inside the bucket, we create a directory for each user manually. (Not sure if using
--only-dir will work unless the directory already exists)
Sample Bucket Directory Structure:
bucket-name - /user1 - /user2
The mounting is done in
etc/sftp.d/mount_user_directories.sh. When deploying to Kubernetes, this script gets executed as a
We just need to ensure your GKE cluster is created with the OAuth scope https://www.googleapis.com/auth/devstorage.read_write, and everything else will be handled automatically. Alternatively, we can mount a file in Service Account JSON key.
For testing, you will need to have Minikube and Docker installed.
For deployment, you will need to have the gcloud SDK.
You can configure SFTP user accounts by adjusting what's in
When adding a new user, add a new line into
uid is a number (e.g. 1003) and
gid is a number (e.g. 1003).
And then add a new line into
etc/sftp.d/mount_user_directories.sh to monunt their
directory to a GCS bucket:
runuser -l partner1 -c \ 'export GOOGLE_APPLICATION_CREDENTIALS=/credentials/gcloud-key.json && \ gcsfuse -o nonempty --only-dir username bucket /home/username/ftp'
This command will mount the bucket as the given user. It also does some environment variable trickery.
⚠️ User passwords are committed to this repo as a demo. Not the best to commit them in practice.
To deploy to GKE follow these steps:
Follow these steps to run this locally with
eval $(minikube docker-env)
docker build --rm -t mikeghen/kube-sftp .
You'll need to adjust files in
etc so that it reflects the SFTP users you're planning to use. You'll also need a Service Account as well.
Then, you can run these commands to put these files on the cluster as secrets:
kubectl create secret generic users --from-file=users.conf=./etc/sftp/users.conf kubectl create secret generic sftp-gcloud-key --from-file=gcloud-key.json=./secrets/gcloud-key.json kubectl create configmap gcs-mounts --from-file=gcs-mounts.sh=./etc/sftp.d/gcs-mounts.sh
kubectl apply -f sftp.yaml
minikube service sftp --url
This will give you the IP and NodePort port.
ℹ️ We use NodePort 30022 for SFTP.
$ sftp -P 30022 [email protected] [email protected]'s password: sftp> pwd /directory