Warning: You must clone this repositiory with recursion in order to pull the submodules.
As infrastructure becomes more and more abstracted from the user, it becomes both easier to work with and harder to manage. Hypervisors, containers, orchestration platforms, etc. Cloud providers today manage to automate this complexity at scale for millions of enterprise customers. TKS is a collection of projects aiming to provide a similar experience with bare metal.
Each component of TKS is broken out into a dedicated subproject. The is that each component should be able to be used interchangeably with other platforms. For example, TKS-Deploy_Kubernetes_Apps is collection of Kubernetes manifests, Kustomizations, and Ansible projects that should allow you to deploy applications to any Kubernetes cluster.
Combining all of the components together will produce a platform that leverages:
|Proxmox VE||Type 1 Hypervisor|
|ZFS||Block storage for VMs and file storage for containers|
|Kubernetes||Container Orchestration Platform|
|HAProxy||Virtual Load Balancer for Kubernetes Control Plane nodes|
|Grafana Stack||Federated monitoring platform|
|Vault||Encrypted & decentralized secrets management|
|Gitlab||Source Control & Continuous Integration|
|Harbor||Container Image Registry|
When possible, automation is leveraged using common tooling like Terraform, Ansible, Cloud Init, and Kustomize. When configuration is necessary, options are exposed through environment variables and defaults are configured as appropriate.
TKS requires a server, some storage, and an understanding of how to network everything together. You don't need much compute, I ran all of this on a 2008 Mac Pro for years. You could even re-purpose this to run on a cloud platform like AWS. TKS-Bootstrap_Proxmox provides instructions for getting started with a Bootable USB Flash Drive or Dell iDRAC.
I'm able to use all of the tooling here from both MacOS and Arch Linux. It will probably work on Windows too. An understanding of how to use the following tools will be helpful, but hopefully not necessary with the documentation.
Clone this repository to retrieve the submodules below. This repository is treated like a Release and each Submodule should reflect the most current stable commit from each project. You can review the
master branch for each project for additional unstable updates if desired. Detailed instructions for how to use each project is located in the respective README.
./inventory.yml file at the root of this repository is used for Ansible in each of the submodules. Be sure to modify it as per your environment before starting.
|TKS-Bootstrap_Proxmox||* Prepare iDRAC or a Bootable USB Device
* Provision and Configure Proxmox VE
* Initialize Storage & Clustering
|TKS-Build_Template||* Build a VM template with Ansible|
|TKS-Bootstrap_Kubernetes||* Deploy HAProxy with Terraform to load balance K8s
* Deploy Kubernetes Cluster with Terraform
* Deploy Calico CNI Plugin
|TKS-Deploy_Kubernetes_Apps||* Deploy Kubernetes apps like MetalLB, Istio, etc.
* Deploy enterprise apps like Jira, OpenVPN, etc.
* Deploy homelab apps like Plex, ruTorrent, Sonarr, etc.
* Leverage Kustomize when possible, Ansible when not
* Support for Istio, External Secrets, resource management, etc.
* Lean on NFS & ZFS for Persistent Volumes
|TKS-Deploy_Harbor||* Deploy Harbor with Terraform
* Leverage LetsEncrypt to receive a valid SSL Certificate
* Integrate with Kubernetes to self host container images
|TKS-Deploy_Grafana||* Deploy Grafana with Terraform
* Configure Kubernetes to ship logs
* Configure other apps to ship logs
|TKS-Deploy_Vault||* Deploy Vault with Terraform
* Configure Vault to act as a secret store for Kubernetes
|TKS-Deploy_Argo||* Deploy Argo with Terraform
* Configure Argo to perform continuous delivery for Kubernetes
|TKS-Deploy_Gitlab||* Deploy Gitlab with Terraform
* Configure Gitlab to manage continuous integration for Kubernetes
Q: Where did the older Ansible/QEMU based project go?
I retired that project in favor of TKS. You can find the code here, however.
Q: Why did you choose Debian instead of X?
Q: Why did you expose configuration through environment variables?
IaC and CaC tooling usually expose configuration through variables files, so I understand why you might ask that. My goal in exposing configuration through environment variables was to better support CI/CD with this tooling.
Q: Why didn't you use X? Why aren't you using Y?
Consider opening an issue informing me why you think that.
Q: Why do you make things so complicated?
It's fun. TKS is developed as a hobby.
Q: I found an issue! How should I notify you?
Please file a GitHub issue under the respective subproject. Please do not email me for support until you have initiated the issue process on GitHub. Pull requests are also welcome and encouraged. :)
Q: What are some ways that I can contribute?