Normally our server logs are like one
mentioned below and so if you use the config file mentioned in first post, then you
can see that each and every line of one log event is captured as separate event
which leads to confusion..!!
There is way to overcome this problem
in logstash by using filter called multiline. This filter will collapse
multiline messages into a single event. The multiline filter is for combining
multiple events from a single source into the same event. The goal of this
filter was to allow joining of multi-line messages from files into a single
event. For example - joining java exception and stack trace messages into a
single event.
General syntax of multiline filter is
filter
{
multiline {
type => "type"
pattern => "pattern, a regexp"
negate => boolean
what => "previous" or
"next"
}
}
Where, ‘regexp’ should match what you believe to be an indicator that the
field is part of a multi-line event. Here we can match logs that start with ^####
which will be common for all weblogic logs. ^ à indicates that logs
start with ####
The 'negate' can be "true" or "false" (defaults false).
input {
Save this file as sample.conf
Run logstash and feed your logs with sample logs, now you can see that all your java exception log entry is captured as single event. This simple multilane filter helps to solve the problem.
You can see that logs are captured as single event.
Please contact me incase of any doubts and in next post, we I will share about GROK filters which gives more flexibility in analyzing the logs..
You need to add the filter in between input and output like mentioned in
below config file.
stdin {
type => "stdin-type"
}
file {
type => "ADMdomainlog"
path => [ "D:/Logstash/Log/soa_domain.log"]
}
}
filter {
multiline {
type => "ADMdomainlog"
pattern => "^####"
negate => true
what => "previous"
}
}
output {
elasticsearch { embedded => true }
}
View in Kibana,
No comments:
Post a Comment