Troubleshooting Exchange 2007 Transport Logs with LogParser

 

There is a series of posts on the Microsoft Exchange team blog on analyzing and troubleshooting your Exchange 2007 logs.

 
Part 1, gives an introduction to LogParser.
Part 2, drills down to analyzing various exchange logs and explains how to draw charts from your mail flow information.

Here are some of the queries you can find in part two:

- Identifying top senders to your system
- Identifying times of highest traffic
- Identifying top senders
- Identifying top email rejecting IP sources
- Identifying outbound email issues
- Content agent filtering log
- Message tracking log

IIS Data Mining with Log Parser 2.X

 

In this video session, you will learn the Log Parser Basics; Input Formats, Output Formats; Functions; Output Templates. You'll also see you how to build on Log Parser; scripting with LogParser.dll, C# Interop, so as some advanced features: new 'CHART' output format, CheckPoint.

The session is presented by Alexis Eller, Microsoft's IIS Program Manager.

SMTP Log parsing

From time to time I need to check my IIs SMTP log files to find why a particular user didn't get email notifications. I have several web servers running community web applications that sends tips, notifications and newsletter to subscribed users.

Digging into SMTP logs is a time consuming process. In addition, The SMTP service sends hundreds of emails and tracking a specific session can be quiet a challenge.

I found that a session can be scattered around the log in a non-consecutive order. I needed a way to collect, group and display SMTP sessions lines in a way that I can see the session flow along with its SMTP verbs.

The natural candidate for log parsing was Microsoft's LogParser 2.2. Boy, this tool is something. After experimenting with log parser I automated the process via PowerShell.

The script query's SMTP log files, in W3C format, and displays the output for all sessions per server IP it finds or for the email address specified in $smtpAddress parameter. I added the option to show all sessions in order to find additional error messages occurred for the queried server.

 

 

############# Parse-W3C.ps1 ###############

param(
    [string]$smtpAddress = $(throw "Please specify email address"),
    [string]$log = $(throw "Please specify log file path, accepts wildcard")
)

#cls
$smtpAddress = "TO:<$smtpAddress>";

 

# the user may have more than one server associated, get list of all mail servers ips
$query="SELECT DISTINCT c-ip FROM $log WHERE cs-uri-query LIKE '%$smtpAddress%'";

$ips = logparser -i:W3C -q:ON $query;

if($ips.length -eq 0){
    write-host "Address not found.`n" -b black -f red;
    return;
}

 

# foreach ip returned emit the session

$ips | foreach {

    $query = "SELECT date,time,c-ip,cs-method,cs-uri-query FROM $log WHERE c-ip = '$_'";
    write-host "[QUERY] $query" -b black -f green;

    $lp=logparser -i:W3C -q:ON $query;

    # set start anchors where line matches "220+" (session starts)
    $session=@();   
    for($i=0;$i -le $lp.length;$i++){if($lp[$i] -match "-\s+220\+"){$session+=$i}}
    $session+=$lp.length;

    if($session.length -eq 0){
        write-host "`nNo sessions found for <$_>`n" -b black -f red;
        return;
    }

    $key = read-host "Found $($session.length) sessions for <$_>`n.Display All Sessions [A], User Sessions [U]?";
    write-host;

    for($i=0;$i -lt $session.length-1;$i++){
        # slice array
        $tmpArr = $lp[$session[$i]..$($session[$i+1]-1)];

        # regex to match any of 421,450,451,452,500,501,
        # 502,503,504,550,551,552,553,554 smtp error codes
        $regex =  "(-\s+)?(421|45[0-2]|5(0|5)[0-4])\+";

        # generic code to colorize SMTP errors
        [scriptblock]$paintRow = {
            $tmpArr | % {
                $email = [regex]::match($_,$smtpAddress).success;
                $err = [regex]::match($_,$regex).success;
                if($email){
                    if($err){write-host $_ -f red -b black}
                    else{write-host $_ -f yellow -b black}
                }
                elseif($err){write-host $_ -f red -b black}               
                else{$_}
            }
        }

        if($tmpArr -match $smtpAddress -and $key -eq "u"){
            & $paintRow;
            if($i -eq 0){read-host "`nPress <ENTER> key to continue"} else{write-host}
        }           
        elseif($key -eq "a"){
            & $paintRow;
            if($i -lt $session.length-2){read-host "`nPress <ENTER> key to continue"} else{write-host}
        }
    }
}

 

###########################################

 

Hope you'll find it useful.

Shay