Simple Parser Rules - Wildcards in text markers?

Notes, tips, and other usefull things on how to use LogMX

Moderator: admin

Post Reply
tgco
Posts: 4
Joined: Tue Jan 31, 2017 5:50 am

Simple Parser Rules - Wildcards in text markers?

Post by tgco » Tue Jan 31, 2017 7:56 am

Is there a wildcard that can be put into the text entry for a simple parser structure? For example if you had a situation where a numeric value that was variable but not significant was in the middle of a text string you wanted to delimit with...

For example in a MIT kerberos KDC log file there is the portion of a REQ handling line that expresses encryption types identified that are available for negotiation and gives a count in the log statement like so (note the paren and bracket are literal to the log file output).

(3 etypes {18 17 23})

The question is can the 3 before etypes in this first one be wildcarded... for example

(\# etypes {\## \## \##})

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Simple Parser Rules - Wildcards in text markers?

Post by admin » Tue Jan 31, 2017 10:38 pm

Hello,

If, as you said, this piece of information to match is "variable but not significant", then it means you cannot use a LogMX "Simple Parser". You will have to use a LogMX "Log4j/Logback Pattern" Parser instead, or a "Regular Expression" Parser. You can read more here: http://logmx.com/features#feat1.
Using one of these Parsers, you will be able to ignore the "3" and any other type of data. If you give me the complete log entry (a few lines of your logs), and the mapping fields (this part is the date, this part is my "Level", this part is a custom field name "Client",...) then I will be pleased to give you a valid Parser for your logs :)

Xavier

tgco
Posts: 4
Joined: Tue Jan 31, 2017 5:50 am

Re: Simple Parser Rules - Wildcards in text markers?

Post by tgco » Thu Feb 02, 2017 6:13 am

Xavier,

Thank you so much for that offer, I dont mean to tie up your time.

For discussion's sake the log segments are kerberos KDC log segments, the significant lines are the AS_REP and TGS_REP lines (authentication service request and reply and ticket granting service request and reply).

Jan 07 09:18:53 kdc.example.com krb5kdc[11217](info): TGS_REQ (3 etypes {18 17 23}) 203.0.113.18: ISSUE: authtime 1483734563, etypes {rep=17 tkt=18 ses=18}, impala/host1.example.com@EXAMPLE.REALM for impala/host4.example.com@EXAMPLE.REALM

Jan 12 16:26:26 kdc.example.com krb5kdc[11217](info): AS_REQ (3 etypes {17 23 18}) 203.0.113.42: ISSUE: authtime 1488960386, etypes {rep=17 tkt=17 ses=17}, HTTP/host8.example.com@EXAMPLE.REALM for krbtgt/EXAMPLE.REALM@EXAMPLE.REALM

I'll take a crack at doing regex based filters for this and post an appropriate topic.

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Simple Parser Rules - Wildcards in text markers?

Post by admin » Thu Feb 02, 2017 11:32 pm

Hello,

I'm really not sure which fields you want to extract/parse, but here's an example of a possible Regex Parser for these logs:

Code: Select all

(\S+ \S+ \S+) (\S+?) .*?\[(.*?)\]\((.*?)\): \S+? \(.*?\) (.*)
For your convenience, I've exported my Parser in a file you can download here:
logmx.parsers.export
(437 Bytes) Downloaded 28 times
(to import it in LogMX, go to Options > Parsers > then click on the Import button at the bottom right-hand corner)

To show you an example of how to use user-defined fields, I've considered that "11217" is a "PID" in "krb5kdc[11217]". Feel free to modify this regex to your needs! To read more details on how to build LogMX Regex Parsers, you can have a look at the file "regex-parsers.html" in LogMX "help" directory.

Xavier

Post Reply