Monitoring Java applications with ELK
Monitoring Java applications with ELK (Elasticsearch, Logstash and Kibana) will show you step-by-step how to properly centralize your Java application logs by sending all its log messages to a remote ELK server. Using this approach you can have all information generated by Java applications, running along multiple servers, in a centralized place. This way you can easily create dashboards and start to analyze your applications in a more high level and practical manner.
You know it’s sad but true
Let’s think about a very common scenario in many companies: many developed Java applications running across multiple application servers, each application performing many operations per day and logging thousands and thousands of lines that generally nobody checks unless some problem occurs along the applications. It sounds familiar, doesn’t it? The biggest issue here is that, unless we are debugging a production problem, the logs have no value at all. They are not telling us anything about aspects we must care about, such as business process performance. There’s gold within these logs!
How about building a better scenario?
Think about the sad story I just told you. Now imagine all your Java applications producing the same amount of logs but then sending them to a centralized place where all received data is accordingly analyzed, modified and finally presented in a real accessible way. Would you like to know how many payments did your system realize in the last minute, day or week? What about how many times a specific exception was thrown? The possibilities are infinite.
Let’s see how to achieve this desired scenario using the ELK stack.
Our proposed solution will combine one Java Application configured to use Logback (the successor of the famous Log4J), one specialized Log Appender class “LogstashTcpSocketAppender” (provided by the Logstash team) and one ELK server.
Tutorial – Monitoring Java applications with ELK
Step 1 – Setup the ELK stack
We have two detailed articles about how to setup the ELK stack on Ubuntu and Windows, please check them following the links bellow:
Step 2 – Configure Logstash to receive our logs
Within the ELK server, create a new configuration file, /etc/logstash/conf.d/logback-listener.conf for Ubuntu 16.04 and D:\ELK\logstash-2.3.4\conf.d\logback-listener.conf for Windows, inserting the following content: