These docker images are tested by hundreds of tools and also used in the full functional test suites of various other GitHub repos.
See also the Kubernetes templates repo.
Overview - this repo contains:
These images are all available pre-built on My DockerHub - https://hub.docker.com/u/harisekhon/.
Quality and Testing - this repo has entire test suites run against it from various GitHub repositories to validate the docker images' functionality, branches vs tagged versions align, latest contains correct version from master branch, syntax checks covering all common build and file formats (Make/JSON/CSV/INI/XML/YAML configurations) etc.
These are reusable tests that can anybody can implement and can be found in my DevOps Python Tools and DevOps Bash Tools repos as well as the Advanced Nagios Plugins Collection which contains hundreds of technology specific API-level test programs to ensure the docker images are functioning as intended.
Continuous Integration in run on this and adjacent repos that form a bi-directional validation between these docker images and several other repositories full of hundreds of programs. All of this is intended to keep the quality of this repo as high as possible.
Cloud & Big Data Contractor, United Kingdom
(ex-Cloudera, former Hortonworks Consultant)
docker search harisekhon docker run harisekhon/nagios-plugins
To see more than the 25 DockerHub repos limited by
docker search (docker issue 23055) I wrote
dockerhub_search.py using the DockerHub API, available in my DevOps Python Tools github repo and as a pre-built docker image:
docker run harisekhon/pytools dockerhub_search.py harisekhon
There are lots of tagged versions of official software in my repos to allow development testing across multiple versions, usually more versions than available from the official repos (and new version updates available on request, just raise a GitHub issue).
DockerHub tags are not shown by
docker search (docker issue 17238) so I wrote
dockerhub_show_tags.py available in my DevOps Python Tools github repo and as a pre-built docker image - eg. to see an organized list of all CentOS tags:
docker run harisekhon/pytools dockerhub_show_tags.py centos
For service technologies like Hadoop, HBase, ZooKeeper etc for which you'll also want port mappings, each directory in the GitHub project contains both a standard
docker-compose configuration as well as a
make run shortcut (which doesn't require
docker-compose to be installed) - either way you don't have to remember all the command line switches and port number specifics:
cd zookeeper docker-compose up
or for technologies with interactive shells like Spark, ZooKeeper, HBase, Drill, Cassandra where you want to be dropped in to an interactive shell, use the
make run shortcut instead:
cd zookeeper make run
which is much easier to type and remember than the equivalent bigger commands like:
docker run -ti -p 2181:2181 harisekhon/zookeeper
and avoid this for more complex services like Hadoop / HBase:
docker run -ti -p 2181:2181 -p 8080:8080 -p 8085:8085 -p 9090:9090 -p 9095:9095 -p 16000:16000 -p 16010:16010 -p 16201:16201 -p 16301:16301 harisekhon/hbase
docker run -ti -p 8020:8020 -p 8032:8032 -p 8088:8088 -p 9000:9000 -p 10020:10020 -p 19888:19888 -p 50010:50010 -p 50020:50020 -p 50070:50070 -p 50075:50075 -p 50090:50090 harisekhon/hadoop
More specific information can be found in the readme page under each respective directory in the Dockerfiles git repo.
Repos suffixed with
-dev are the official technologies + development & debugging tools + my github repos with all dependencies pre-built.
Advanced Nagios Plugins Collection - 450+ nagios plugins for every Hadoop distribution and every major NoSQL technology - Hadoop, Redis, Elasticsearch, Solr, HBase, Cassandra & DataStax OpsCenter, MongoDB, MySQL, Kafka, Riak, Memcached, Couchbase, CouchDB, Mesos, Spark, Neo4j, Datameer, H2O, WanDisco, Yarn, HDFS, Impala, Apache Drill, Presto, ZooKeeper, Cloudera, Hortonworks, MapR, IBM BigInsights, Infrastructure - Linux, DNS, Whois, SSL Certs etc
harisekhon/tools - DevOps Tools superset of the below images, containing hundreds of programs:
harisekhon/pytools- DevOps Python Tools - 80+ DevOps CLI tools tools for AWS, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, Ambari, Blueprints, CloudFormation, Elasticsearch, Solr, Pig etc.
harisekhon/bash-tools- DevOps Bash Tools - 200+ DevOps CLI tools for AWS, GCP, Kubernetes, Hadoop, Hive, Impala, Kafka, Docker, LDAP, Git, Code & build linting, package management for Linux / Mac / Python / Perl / Ruby / NodeJS / Golang, and lots more random goodies
harisekhon/perl-tools- DevOps Perl Tools - 25+ DevOps CLI Tools - Log Anonymizer, Hadoop HDFS & Hive tools, Solr/SolrCloud CLI, SQL ReCaser (MySQL, PostgreSQL, AWS Redshift, Snowflake, Apache Drill, Hive, Impala, Cassandra CQL, Microsoft SQL Server, Oracle, Couchbase N1QL, Dockerfiles, Pig Latin, Neo4j, InfluxDB), Linux, Nginx stats & HTTP(S) URL watchers for load balanced web farms, Ambari FreeIPA Kerberos, Datameer etc.
all Github repos pre-built on major Linux distros with CLI programs located at
harisekhon/dev:centos/ - CentOS latest with Java JDK, Perl, Python, Jython, Ruby, Scala, Groovy, GCC, Maven, SBT, Gradle, Make, Expect, EPEL etc.
harisekhon/dev:alpine- Alpine latest with Java JDK, Perl, Python, Jython, Ruby, Scala, Groovy, GCC, Maven, SBT, Gradle, Make, Expect etc.
harisekhon/dev:debian- Debian latest with Java JDK, Perl, Python, Jython, Ruby, Scala, Groovy, GCC, Maven, SBT, Gradle, Make, Expect etc.
harisekhon/dev:fedora- Fedora latest with Java JDK, Perl, Python, Jython, Ruby, Scala, Groovy, GCC, Maven, SBT, Gradle, Make, Expect etc.
harisekhon/dev:ubuntu- Ubuntu latest with Java JDK, Perl, Python, Jython, Ruby, Scala, Groovy, GCC, Maven, SBT, Gradle, Make, Expect etc.
All builds use OpenJDK with
jdk numbered tags. See this article below for why it might be illegal to bundle Oracle Java (and why no Linux distributions do this either):
harisekhon/alpine-java- Alpine latest with Java 8
harisekhon/centos-java- CentOS latest combinations of Java 7 / 8 and Scala 2.10 / 2.11
harisekhon/debian-java- Debian latest with Java 7, 8
harisekhon/fedora-java- Fedora latest combinations of Java 7/8 and Scala 2.10/2.11
All images come pre-built on DockerHub but if you want to compile from source for any reason such as developing improvements, I've made this easy to do:
git clone https://github.com/harisekhon/Dockerfiles cd Dockerfiles
To build all Docker images, just run the
make command at the top level:
To build a specific Docker image, enter its directory and run
cd nagios-plugins make
You can also build a specific version by checking out the git branch for the version and running the build:
cd consul git checkout consul-0.9 make
or build all versions of a given software project like so:
cd hadoop make build-versions
See the top level
Makefile as well as the
Makefile.in which is sourced per project with any project specific overrides in the
Please raise tickets for issues and improvements at https://github.com/harisekhon/dockerfiles/issues
DevOps Bash Tools - 550+ DevOps Bash Scripts, Advanced
.gitconfig, CI configs & Utility Code Library - AWS, GCP, Kubernetes, Docker, Kafka, Hadoop, SQL, BigQuery, Hive, Impala, PostgreSQL, MySQL, LDAP, DockerHub, Jenkins, Spotify API & MP3 tools, Git tricks, GitHub API, GitLab API, BitBucket API, Code & build linting, package management for Linux / Mac / Python / Perl / Ruby / NodeJS / Golang, and lots more random goodies
SQL Scripts - 100+ SQL Scripts - PostgreSQL, MySQL, AWS Athena, Google BigQuery
Templates - dozens of Code & Config templates - AWS, GCP, Docker, Jenkins, Terraform, Vagrant, Puppet, Python, Bash, Go, Perl, Java, Scala, Groovy, Maven, SBT, Gradle, Make, GitHub Actions Workflows, CircleCI, Jenkinsfile, Makefile, Dockerfile, docker-compose.yml, M4 etc.
Kubernetes templates - Kubernetes YAML templates - Best Practices, Tips & Tricks are baked right into the templates for future deployments
DevOps Python Tools - 80+ DevOps CLI tools for AWS, Hadoop, HBase, Spark, Log Anonymizer, Ambari Blueprints, AWS CloudFormation, Linux, Docker, Spark Data Converters & Validators (Avro / Parquet / JSON / CSV / INI / XML / YAML), Elasticsearch, Solr, Travis CI, Pig, IPython
The Advanced Nagios Plugins Collection - 450+ programs for Nagios monitoring your Hadoop & NoSQL clusters. Covers every Hadoop vendor's management API and every major NoSQL technology (HBase, Cassandra, MongoDB, Elasticsearch, Solr, Riak, Redis etc.) as well as message queues (Kafka, RabbitMQ), continuous integration (Jenkins, Travis CI) and traditional infrastructure (SSL, Whois, DNS, Linux)
DevOps Perl Tools - 25+ DevOps CLI tools for Hadoop, HDFS, Hive, Solr/SolrCloud CLI, Log Anonymizer, Nginx stats & HTTP(S) URL watchers for load balanced web farms, Dockerfiles & SQL ReCaser (MySQL, PostgreSQL, AWS Redshift, Snowflake, Apache Drill, Hive, Impala, Cassandra CQL, Microsoft SQL Server, Oracle, Couchbase N1QL, Dockerfiles, Pig Latin, Neo4j, InfluxDB), Ambari FreeIPA Kerberos, Datameer, Linux...
HAProxy Configs - 80+ HAProxy Configs for Hadoop, Big Data, NoSQL, Docker, Elasticsearch, SolrCloud, HBase, Cloudera, Hortonworks, MapR, MySQL, PostgreSQL, Apache Drill, Hive, Presto, Impala, ZooKeeper, OpenTSDB, InfluxDB, Prometheus, Kibana, Graphite, SSH, RabbitMQ, Redis, Riak, Rancher etc.