Eric’s Blog

March 15th, 2007 by eric

Description: File::ReadBackwards works similar to the linux shell command tac. It reads the file line by line strarting from the end of the file.

CPAN: File::ReadBackwards

Example 1:
Being a System’s Administrator, I am usually doing some analysis on a large logfile. Therefore, I may not need all the information contained in the log. This may be especially true if the logs only get rotated once a day or once a week and I don’t need all the information in the log file. Using File::ReadBackwards in combination with a date and time calculation module, I can take only the amount of time I want to use from the logs and then stop processing there. Since we aren’t covering the date calculations here, I will push those out to another subroutine that we will assume works.

# Always use these
use strict;
use warnings;

# Use the module itself
use File::ReadBackwards;

# Define the log file to be read
my $log = "/var/log/log_file";

# Open the logfile by tie'ing it to the module
tie *LOG, "File::ReadBackwards", "$log"
   or die ("$log tie error: $!");

# Iterate over the logfile
while (my $line = ) {

  # Split the log line
  my @entry = split(/\s+/, $line);

  # Take the timestamp and check if we
  #   have hit our threshold yet
  # Break loop if we have
  last if (time_reached($entry[0]) == 1);
}

# Cleanup
untie (*LOG);

March 9th, 2007 by eric

Description: The author of this module notes that it is best used, especially by him, when reading or manipulating log files. I have a tendency to use it for the exact same thing, especially when looking for context around captured lines.

CPAN: File::Bidirectional

Note:
Although I would like to note that using the tie’d interface as I have done takes approximately 2 1/2 times as long as a regular file read according to benchmarks, it is still a very handy tool and allows one not to reinvent the wheel.

Example 1:
Here we are going to go through a log file and when we hit the time stamp we want, we are going to change directions and go back through. There is no real reason to change direction here, I am merely demonstrating how it would be accomplished.

# Always use these
use strict;
use warnings;

# Use the module itself
use File::Bidirectional;

# Define the log file to be read
my $log = "/var/log/log_file";

# Open the logfile by tie'ing it to the module
#  This is exactly the same as File::ReadBackwards
tie *LOG, "File::Bidirectional", "$log", {mode => 'backward'}
   or die ("$log tie error: $!");

# Iterate over the logfile
while (my $line = ) {

  # Split the log line
  my @entry = split(/\s+/, $line);

  # Take the timestamp and check if we
  #   have hit our threshold yet
  # Get the line # then change direction
  if (time_reached($entry[0]) == 1) {
    $line_num = (tied *LOG)->line_num();
    (tied *LOG)->switch();
  }
}

# Cleanup
untie (*LOG);

January 23rd, 2007 by eric

Description: Recently, I have had the pleasure of getting knee deep into various aspects of Email. One of the things that I consistantly found myself wanting to do was to parse through it. I know the best way to do this is to connect to the IMAP server and download the messages. The best way I have come accross on how to accomplish this task is using Mail::IMAPClient.

CPAN: Mail::IMAPClient

Example 1:
Creating a connection to an IMAP server isn’t complicated. And once you are connected there are many things that you can do to manipulate messages. In this first example, I am merely going to show you how to connect to the mail server and download ALL the messages in specific folders:

# Always be safe
use strict;
use warnings;

# Use the module
use Mail::IMAPClient;

 $imap = Mail::IMAPClient->new( Server  => 'mail.server.com:143',
                                User    => 'me',
                              Password  => 'mypass')
        # module uses eval, so we use $@ instead of $!
        or die "IMAP Failure: $@";

 foreach my $box qw( HAM SPAM ) {
   # Which file are the messages going into
   my $file = "mail/$box";

   # Select the mailbox to get messages from
   $imap->select($box)
        or die "IMAP Select Error: $!";

   # Store each message as an array element
   my @msgs = $imap->search('ALL')
        or die "Couldn't get all messages\n";

   # Loop over the messages and store in file
   foreach my $msg (@msgs) {
     # Pipe msgs through 'formail' so they are stored properly
     open my $pipe, "| formail >> $file"
       or die("Formail Open Pipe Error: $!");

     # Send msg through file pipe
     $imap->message_to_file($pipe, $msg);

     # Close the messgae pipe
     close $pipe
       or die("Formail Close Pipe Error: $!");
   }

   # Close the folder
   $imap->close($box);
 }

 # We're all done with IMAP here
 $imap->logout();

Example 2:
In this next example, we are going to take advantage of some other methods that are provided by this useful little module. We will begin by using the same base as is in Example 1, but we will add some nuances in the middle for functionality.

Note: Messages don’t get immediately deleted with IMAP, only marked for deletion. They aren’t actually deleted until the box is expunged. In this case, it gets done after the looping over each mailbox is complete. This is to say that if the program gets interrupted in the middle that the messages won’t be deleted until the mailbox is officially issued an expunge command.

# Always be safe
use strict;
use warnings;

# Use the module
use Mail::IMAPClient;

 $imap = Mail::IMAPClient->new( Server  => 'mail.server.com:143',
                                User    => 'me',
                              Password  => 'mypass')
        # module uses eval, so we use $@ instead of $!
        or die "IMAP Failure: $@";

 foreach my $box qw( HAM SPAM ) {
   # How many msgs are we going to process
   print "There are ". $imap->message_count($box).
          " messages in the $box folder.\n";

   # Which file are the messages going into
   my $file = "mail/$box";

   # Select the mailbox to get messages from
   $imap->select($box)
        or die "IMAP Select Error: $!";

   # Store each message as an array element
   my @msgs = $imap->search('ALL')
        or die "Couldn't get all messages\n";

   # Loop over the messages and store in file
   foreach my $msg (@msgs) {
     # Pipe msgs through 'formail' so they are stored properly
     open my $pipe, "| formail >> $file"
       or die("Formail Open Pipe Error: $!");

     # Skip the msg if its over 100k
     if ($imap->size($msg) > 100000) {
       $imap->delete_message($msg);
       next;
     }

     # Send msg through file pipe
     $imap->message_to_file($pipe, $msg);

     # Close the messgae pipe
     close $pipe
       or die("Formail Close Pipe Error: $!");

     # Delete each message after downloading
     $imap->delete_message($msg);

     # If we are just testing and want to leave
     #  leave the messages untouched, then we
     #  can use the following line of code
     # $imap->deny_seeing($msg);
   }

   # Expunge and close the folder
   $imap->expunge($box);
   $imap->close($box);
 }

 # We're all done with IMAP here
 $imap->logout();