Mtail

extract internal monitoring data from application logs for collection in a timeseries database
Alternatives To Mtail
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Bettercap14,547
6 days ago61April 21, 2021164otherGo
The Swiss Army knife for 802.11, BLE, IPv4 and IPv6 networks reconnaissance and MITM attacks.
Mtail3,632
12 days ago55May 27, 202150apache-2.0Go
extract internal monitoring data from application logs for collection in a timeseries database
Orange2,294
a month ago77mitLua
OpenResty/Nginx Gateway for API Monitoring and Management.
Zabbix Docker1,967
5 days ago2gpl-2.0Shell
Official Zabbix Dockerfiles
Promxy982
8 days ago93February 03, 202212mitGo
An aggregating proxy to enable HA prometheus
Chameleon593
2 months ago6agpl-3.0Dockerfile
19 Customizable honeypots for monitoring network traffic, bots activities and username\password credentials (DNS, HTTP Proxy, HTTP, HTTPS, SSH, POP3, IMAP, STMP, RDP, VNC, SMB, SOCKS5, Redis, TELNET, Postgres, MySQL, MSSQL, Elastic and ldap)
Dogvscat457
9 months ago10mitHCL
Sample Docker Swarm cluster stack of tools
Prometheus Boshrelease111
10 days ago1apache-2.0Shell
Prometheus BOSH Release
Gateway9713a month ago106April 16, 2021apache-2.0Go
A proxy to buffer and forward metrics, events, and traces.
Activedrouter97
6 years agootherGo
Simple http / https Reverse Proxy(In developing), monitoring, routing services.
Alternatives To Mtail
Select To Compare


Alternative Project Comparisons
Readme
mtail

mtail - extract internal monitoring data from application logs for collection into a timeseries database

ci GoDoc Go Report Card OSS-Fuzz CirrusCI Build Status codecov

mtail is a tool for extracting metrics from application logs to be exported into a timeseries database or timeseries calculator for alerting and dashboarding.

It fills a monitoring niche by being the glue between applications that do not export their own internal state (other than via logs) and existing monitoring systems, such that system operators do not need to patch those applications to instrument them or writing custom extraction code for every such application.

The extraction is controlled by mtail programs which define patterns and actions:

# simple line counter
counter lines_total
/$/ {
  lines_total++
}

Metrics are exported for scraping by a collector as JSON or Prometheus format over HTTP, or can be periodically sent to a collectd, StatsD, or Graphite collector socket.

Read the programming guide if you want to learn how to write mtail programs.

Ask general questions on the users mailing list: https://groups.google.com/g/mtail-users

Installation

There are various ways of installing mtail.

Precompiled binaries

Precompiled binaries for released versions are available in the Releases page on Github. Using the latest production release binary is the recommended way of installing mtail.

Windows, OSX and Linux binaries are available.

Building from source

The simplest way to get mtail is to go get it directly.

go get github.com/google/mtail/cmd/mtail

This assumes you have a working Go environment with a recent Go version. Usually mtail is tested to work with the last two minor versions (e.g. Go 1.12 and Go 1.11).

If you want to fetch everything, you need to turn on Go Modules to succeed because of the way Go Modules have changed the way go get treats source trees with no Go code at the top level.

GO111MODULE=on go get -u github.com/google/mtail
cd $GOPATH/src/github.com/google/mtail
make install

If you develop the compiler you will need some additional tools like goyacc to be able to rebuild the parser.

See the Build instructions for more details.

A Dockerfile is included in this repository for local development as an alternative to installing Go in your environment, and takes care of all the build dependency installation, if you don't care for that.

Deployment

mtail works best when it paired with a timeseries-based calculator and alerting tool, like Prometheus.

So what you do is you take the metrics from the log files and you bring them down to the monitoring system?

It deals with the instrumentation so the engineers don't have to! It has the extraction skills! It is good at dealing with log files!!

Read More

Full documentation at http://google.github.io/mtail/

Read more about writing mtail programs:

Read more about hacking on mtail

Read more about deploying mtail and your programs in a monitoring environment

After that, if you have any questions, please email (and optionally join) the mailing list: https://groups.google.com/forum/#!forum/mtail-users or file a new issue.

Popular Monitoring Projects
Popular Proxy Projects
Popular Operations Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Golang
Proxy
Virtual Machine
Metrics
Monitoring
Prometheus
Time Series
Bytecode
Instrumentation
Observability
Alerting
Timeseries Database