Filtering and color coding maillogs

January 18th, 2010

We recently made a large change to mail routing at work, I was trying to look at the mail logs but being able to discern useful information from streaming logs at any company of reasonable size can be difficult. Borrowed an Idea from here: http://blog.crythias.com/2008/04/awk-colorizer-for-tail.html

and came up with the following:
* note ^[ is ctrl-v esc not ^ and a [

function colorize(word, color)
{
c["red"] = “^[[1;31;40m"
c["green"] = “^[[1;32;40m"
c["yellow"] = “^[[1;33;40m"
c["blue"] = “^[[1;34;40m"
c["magenta"] = “^[[1;35;40m"
if (line ~ word)
{
line=c["magenta"] $6 c[color] ($7)
print line
}
}
{line = $0
colorize(“to=”,”green”)
colorize(“from=”,”red”)
}

I save this to printcolor.awk and then just pass my maillog through it:

tail -f /var/log/maillog | awk -f ~/printcolor.awk
A5FBD188DB:from=<xxxxxxxxxx@yyyyyyyyyy.com>,
A5FBD188DB:to=<xxxxxxxxxx@yyyyyyyyyy.com>,
BCFE2188EE:from=<xxxxxxxxxx@yyyyyyyyyy.com>,
BCFE2188EE:to=<xxxxxxxxxx@yyyyyyyyyy.com>,

Living inside screen

February 19th, 2009

First of all, what is screen?

According to the website, “Screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells.”. Basically screen lets you run multiple shells that aren’t tied to your active connection. You can disconnect and they stay alive, you can then come back later and reconnect and continue to use them. Screen is used for long running processes you care about seeing the output from, such as viewing log output of a daemon or most commonly a command line IRC client.

I have one server that I am constantly in a screen session in, and I found a tip on parsed.org that makes it a no brainer. Basically whenever you start a shell it calls screen. If screen isn’t running, it starts a new one. This is great for remote systems!


You can use this in your shell dotfile (e.g. ~/.bash_profile, ~/.zshrc) to create a screen session when you log in or reattach to an existing screen:


if [[ $TERM != 'screen' ]] ; then
    if [[ `screen -list | grep -v "No" | awk '$2 { print }' | wc -l` == 0 ]] ; then
        screen
    else
        screen -dr
    fi
fi

Migrating a Maildir archive to IMAP

February 19th, 2009

A while ago I migrated my email from local hosting to a remote hosted solution. All of my new mail went fine but the old mail was left sitting in a Maildir on my home server. To solved this dilemma I wrote a python script to move it for me. It was much easier then I expected it to be.


#!/usr/bin/env python

import mailbox, imaplib, email, getpass

SERVER = "mail.server.com"
USER  = "dgibbons"
PASSWORD = "xxxxxx"
MAILDIR_PATH = '/path/to/maildir'
mdir = 'INBOX.oldmail' # change to the imap location you want your mail to end up at 

# connect to server
server = imaplib.IMAP4_SSL(SERVER)

# login
server.login(USER, PASSWORD)
server.select()
server.create(mdir)
server.select(mdir)

src_mdir = mailbox.Maildir('MAILDIR_PATH')
for m in src_mdir:
    server.append(mdir,None, email.Utils.parsedate(m.getheader('Date')), ''.join(m.headers)+'\n'+m.fp.read())

Checking for Boxee updates with ubuntu 64bit

February 18th, 2009

If you’re one of the people using Boxee on Ubuntu 64, you probably went through some archaic steps involving the 32bit deb package and getlibs, as described here.  What this doesn’t give you however, is a easy method to make sure your boxee version is update.

So I wrote a small script to check for you.

Basically if you run this and you get no output, you’re running the latest. If you get a message telling you to upgrade, well you should know what to do.

Paste the following into a file and chmod +x it.

#!/bin/sh
if [ ! $( which curl ) ]; then
sudo apt-get install -yq curl
fi

LATEST=$(curl -s http://apt.boxee.tv/dists/intrepid/m...86/Packages.gz | zgrep Version Packages.gz | awk '{v=0;for (i = 1; i <= NF; i++) if ($2 > v) v = $2};END{print v}')

CURRENT=$(dpkg -s boxee | awk '/Version/ {print $2}')
if [ $LATEST != $CURRENT ] ; then
echo TIME TO UPGRADE
fi