Log4j pattern parser not implementing %C{1} correctly

Notes, tips, and other usefull things on how to use LogMX

Moderator: admin

Post Reply
doahh
Posts: 1
Joined: Fri Nov 04, 2011 6:23 pm

Log4j pattern parser not implementing %C{1} correctly

Post by doahh »

Hello,

I have the following Log4j pattern parser:

Code: Select all

%d{HH:mm:ss.SSS} | [%-5p] | %C{1} | %m
and the following Parser test:

Code: Select all

18:05:07.338 | [DEBUG] | l10n.map.LocalizationMap | l10n: controllerAbstract.l10nAlertErrorTitle [Error message]
and I am finding that the {1} part of the %C{1} is not being read. The value outputted into the 'Emitter' field is ' l10n.map.LocalizationMap' no matter what the value of the number is. Is this normal behaviour for LogMX?

Great piece of software by the way, it really fills a void.
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Log4j pattern parser not implementing %C{1} correctly

Post by admin »

Hello,

Thanks for posting here.

It is indeed a nominal LogMX behavior here. Using %C{1} in your Log4j configuration file makes it write only the last part of the 'Category' (or 'Emitter'), like 'z' for 'x.y.z'. So your log file should contain 'z' instead of 'x.y.z'. So, if you really use Log4j to write your logs, you should not have "l10n.map.LocalizationMap" in your logs, but only "LocalizationMap" instead.

Using %C{1} tells LogMX that the emitter (or 'Category') should not contain the character '.' anymore in your logs. LogMX will not use it to take the right part of your emitter.

For more information on this Log4j 'C' conversion character: http://logging.apache.org/log4j/1.2/api ... ayout.html

In order to take only the last part of the Emitter, here is a small sample parser for your log format:

Code: Select all

package sample.parser;

import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import com.lightysoft.logmx.business.ParsedEntry;
import com.lightysoft.logmx.mgr.LogFileParser;

/**
 * Sample LogMX Parser able to parse a log file with multi-line support and Relative Date support.<BR/>
 * Parser can parse logs produced with following Log4j Pattern:
 *   %d{HH:mm:ss.SSS} | [%-5p] | %C{1} | %m 
 * Here is an example of log file suitable for this parser:<BR/>
 * 
 *   18:05:07.338 | [DEBUG] | l10n.map.LocalizationMap | l10n: controllerAbstract.l10nAlertErrorTitle [Error message] 
 *   18:05:08.338 | [INFO ] | l10n.map.LocalizationMap | very long 
 *   message on two lines 
 */
public class DoahhParser extends LogFileParser {
    /** Current parsed log entry */
    private ParsedEntry entry = null;

    /** Entry date format */
    private static SimpleDateFormat dateFormat = new SimpleDateFormat("HH:mm:ss.SS");

    /** Pattern for entry begin */
    private final static Pattern ENTRY_BEGIN_PATTERN = Pattern
        .compile("^(\\d{2}:\\d{2}:\\d{2}\\.\\d+) \\| \\[(.*?)\\] \\| ([^\\|]+) \\| (.*)$");

    /** Buffer for Entry message (improves performance for multi-lines entries)  */
    private StringBuilder entryMsgBuffer = null;

    /** User-defined fields names (here, none) */
    private static final List<String> EXTRA_FIELDS_KEYS = new ArrayList<String>();


    /** 
     * Returns the name of this parser
     * @see com.lightysoft.logmx.mgr.LogFileParser#getParserName()
     */
    public String getParserName() {
        return "Doahh Parser";
    }

    /**
     * Returns the supported file type for this parser
     * @see com.lightysoft.logmx.mgr.LogFileParser#getSupportedFileType()
     */
    public String getSupportedFileType() {
        return "Doahh log files";
    }

    /**
     * Process the new line of text read from file 
     * @see com.lightysoft.logmx.mgr.LogFileParser#parseLine(java.lang.String)
     */
    protected void parseLine(String line) throws Exception {
        // If end of file, records last entry if necessary, and exits
        if (line == null) {
            recordPreviousEntryIfExists();
            return;
        }

        Matcher matcher = ENTRY_BEGIN_PATTERN.matcher(line);
        if (matcher.matches()) {
            // Record previous found entry if exists, then create a new one
            prepareNewEntry();

            // Split emitter around characters '.'
            String[] emitter = matcher.group(3).split("\\.");

            entry.setDate(matcher.group(1));
            entry.setLevel(matcher.group(2).trim());
            entry.setEmitter(emitter[emitter.length - 1]); // take only the last part of '.'
            entryMsgBuffer.append(matcher.group(4));
        } else if (entry != null) {
            entryMsgBuffer.append('\n').append(line); // appends this line to previous entry's text
        }
    }

    /** 
     * Returns the ordered list of user-defined fields to display (given by their key), for each entry.
     * @see com.lightysoft.logmx.mgr.LogFileParser#getUserDefinedFields()
     */
    @Override
    public List<String> getUserDefinedFields() {
        return EXTRA_FIELDS_KEYS;
    }

    /**
     * Returns a relative Date for the given entry
     * @see com.lightysoft.logmx.mgr.LogFileParser#getRelativeEntryDate(com.lightysoft.logmx.business.ParsedEntry)
     */
    public Date getRelativeEntryDate(ParsedEntry pEntry) throws Exception {
        return dateFormat.parse(pEntry.getDate());
    }

    /**
     * Returns the absolute Date for the given entry 
     * @see com.lightysoft.logmx.mgr.LogFileParser#getAbsoluteEntryDate(com.lightysoft.logmx.business.ParsedEntry)
     */
    public Date getAbsoluteEntryDate(ParsedEntry pEntry) throws Exception {
        return null; // absolute date not handled (no day/month/year)
    }

    /**
     * Send to LogMX the current parsed log entry
     * @throws Exception
     */
    private void recordPreviousEntryIfExists() throws Exception {
        if (entry != null) {
            entry.setMessage(entryMsgBuffer.toString());
            addEntry(entry);
        }
    }

    /**
     * Send to LogMX the current parsed log entry, then create a new one
     * @throws Exception
     */
    private void prepareNewEntry() throws Exception {
        recordPreviousEntryIfExists();
        entry = createNewEntry();
        entryMsgBuffer = new StringBuilder(80);
        entry.setUserDefinedFields(new HashMap<String, Object>(1)); // Create an empty Map with only one element allocated
    }
}
To install this parser in LogMX, simple paste this code in a file "DoahhParser.java" in LogMX directory "parsers\src\sample\parser", and compile it with Ant or Eclipse (see http://www.logmx.com/p_parser_dev.php). Feel free to post here if you have any question or problem.

Xavier.
Post Reply