centralized logging

Table of Contents
  1. Better centrally: Professional Logging
  2. centralized logging
  3. example
  4. For example, part 2
  5. Conclusion
  6. read on one side

The simple answer: centralized logging

Remedy for these problems is able to create a centralized logging, which throughout all parts of the application system, or - better yet - use the entire application landscape. The basic idea is simple: All logs come together in one place, are stored there as long as necessary and can be analyzed by the administrators, developers and possibly other parties simultaneously structured. To make this possible, a key logging software has to cope with the following challenges:

Central logging software can be against this background into two categories divided: Cloud-based and self-managed. The latter techniques can be subdivided further. Here there are a few built-in products, for example logfaces or ELK stack (Elasticsearch, Logstash, Kibana) of Elasticsearch.

In addition, there are tools that cover one or more of these areas of responsibility. They can be combined to full-fledged central logging techniques. As many companies express strategically for or against the use of cloud services, which facilitates the selection of an appropriate software.

For application systems in the (public) cloud the story is quickly told. A logging on the local hard disk of an unknown computer is not very helpful. The provision of a separate log server in the cloud contradicts the architectural decision of the actual system. Therefore, a thoughtful decision can only fall in favor of a cloud-logging technique in such a scenario. Prominent representatives are here loggly, Paper Trail or lied tries.

For custom engineered, mission-critical application systems, the situation often is different. They are predominantly operated its own technical infrastructure. In such an environment, it is advisable to use a low-maintenance self-managed software. It should be noted that the integration of cloud technologies is also possible for these application systems. The license costs and the dependence of the use of external services is to pursue but attentive.

To illustrate how even managed centralized logging is used in practice and connected, this article looks example the commercial product logfaces. It is easy to configure, integrate and set up simply. The manufacturer of this tool offers a time-limited trial version without registration, which is to try out the concepts described herein. Who satisfies the scope of logfaces can refer for production use licenses per installation or per site. Who wants more searches than simple logging operational application data or an open-source software, may not find a suitable offer with the ELK stack.

Centralized logging: tried and tested plus networking

Logfaces consists of three components: A dedicated log server takes the log events of the application components against a database they persisted, and a graphical client supports reading, searching and analyzing the log entries. The following figure shows a fictitious 3-layer application with load distribution, such as a centralized logging itself could be implemented.

All components of a distributed application transmitted their logs to a central location (Fig. 1). All components of a distributed application transmitted their logs to a central location (Fig. 1).

In the example scenario, the application clients access via a load balancer to the backend server. Here lies the computationally complex application core. Data storage is simple, so that the back-end servers to access a common database. In addition, the servers are stateless, and the load balancer can decide per request to which back-end server to point a client.

In the yellow rectangle at the bottom right of the log server is displayed. The green lines indicate the flow of log entries. Both the client and server components of the application send their log entries to the log server. As a rule, used log library is the interface between the application components and the log server. Most here will log4j or another member of the family log4x (z. B. log4net, slf4j) Logback or NLog. For the most common libraries provides logfaces output plug-ins. In log4x these are provided in the form of a so-called appender. The plug-ins provide each ensure that the events buffered and transmitted asynchronously. Breaks the connection to log server Appender tried the retransmission of events and updates them if necessary to a local file.

In addition to these libraries an interface exists based on the known from Unix syslog protocol. It can be used in the sample scenario to connect the database and the Load Balancer. As a result, all log entries are sorted chronologically in one place: the database of the log server.