Currently I’m using the logging setup of Beaver shipping logs into an ELK stack, and metrics with collectd shipping metrics into a Graphite stack. Now that Elastic have Beats that do both logging and metrics, its worth exploring further.

Looking at the combination of,

• Filebeat - shipping logs, replacing beaver in the above configuration
• Topbeat - shipping cpu, memory, load metrics.
• Packetbeat - Network packet data.
• Metricbeat - In alpha, a targeted apache/redis/whatever metrics shipper.

If Beats do what they are attempting to do, it could mean the ability to drop that Graphite stack altogether and centralise all system data analysis, graphing, and alert triggering in ELK.

Obviously theres work around elasticsearch stack architecture and performance if you want to throw all the metrics at it too (a detailed topic in and of itself). But, it needs to be highly available anyway, and supporting one HA component rather than two could be a simplification worth making.

So, what do Beats look like in action. Maybe a simple docker test setup will help me, so I’ve started building a toy environment for playing around with beats.

`Currently I’m using the setup of,

• Logging - Beaver shipping logs into an ELK stack
• Metrics - collectd shipping metrics into a Graphite stack

Now that Elastic have Beats, the combination of,

• Filebeat - shipping logs, replacing beaver in the above configuration
• Topbeat - shipping cpu, memory, load metrics.
• Packetbeat - Network packet data.
• Metricbeat - In alpha, a targeted apache/redis/whatever metrics shipper.

If Beats do what they are attempting to do, it could mean the ability to drop that Graphite stack altogether and centralise all system data analysis, graphing, and alert triggering in ELK.

Obviously theres work around elasticsearch stack architecture and performance if you want to throw all the metrics at it too (a detailed topic in and of itself). But, it needs to be highly available anyway, and supporting one HA component rather than two could be a simplification worth making.

So, what do Beats look like in action. Maybe a simple docker test setup will help me, so I’ve started building a toy environment for playing around with beats.

`

The basic stuff seems ok, below is an example of the load usage.

Theres also a fair amount of downloadable community beats.

Fun to play around with just now, looking optimistic for future production use? Maybe…