Erics Tech Blog Thoughts, musings, and other idealistic (sometimes useful) systems and development hoopla.

Description: The author of this module notes that it is best used, especially by him, when reading or manipulating log files. I have a tendency to use it for the exact same thing, especially when looking for context around captured lines.

CPAN: File::Bidirectional

Note:
Although I would like to note that using the tie’d interface as I have done takes approximately 2 1/2 times as long as a regular file read according to benchmarks, it is still a very handy tool and allows one not to reinvent the wheel.

Example 1:
Here we are going to go through a log file and when we hit the time stamp we want, we are going to change directions and go back through. There is no real reason to change direction here, I am merely demonstrating how it would be accomplished.

# Always use these
use strict;
use warnings;

# Use the module itself
use File::Bidirectional;

# Define the log file to be read
my $log = "/var/log/log_file";

# Open the logfile by tie'ing it to the module
#  This is exactly the same as File::ReadBackwards
tie *LOG, "File::Bidirectional", "$log", {mode => 'backward'}
   or die ("$log tie error: $!");

# Iterate over the logfile
while (my $line = ) {

  # Split the log line
  my @entry = split(/\s+/, $line);

  # Take the timestamp and check if we
  #   have hit our threshold yet
  # Get the line # then change direction
  if (time_reached($entry[0]) == 1) {
    $line_num = (tied *LOG)->line_num();
    (tied *LOG)->switch();
  }
}

# Cleanup
untie (*LOG);

Description: Recently, I have had the pleasure of getting knee deep into various aspects of Email. One of the things that I consistantly found myself wanting to do was to parse through it. I know the best way to do this is to connect to the IMAP server and download the messages. The best way I have come accross on how to accomplish this task is using Mail::IMAPClient.

CPAN: Mail::IMAPClient

Example 1:
Creating a connection to an IMAP server isn’t complicated. And once you are connected there are many things that you can do to manipulate messages. In this first example, I am merely going to show you how to connect to the mail server and download ALL the messages in specific folders:

# Always be safe
use strict;
use warnings;

# Use the module
use Mail::IMAPClient;

 $imap = Mail::IMAPClient->new( Server  => 'mail.server.com:143',
                                User    => 'me',
                              Password  => 'mypass')
        # module uses eval, so we use $@ instead of $!
        or die "IMAP Failure: $@";

 foreach my $box qw( HAM SPAM ) {
   # Which file are the messages going into
   my $file = "mail/$box";

   # Select the mailbox to get messages from
   $imap->select($box)
        or die "IMAP Select Error: $!";

   # Store each message as an array element
   my @msgs = $imap->search('ALL')
        or die "Couldn't get all messages\n";

   # Loop over the messages and store in file
   foreach my $msg (@msgs) {
     # Pipe msgs through 'formail' so they are stored properly
     open my $pipe, "| formail >> $file"
       or die("Formail Open Pipe Error: $!");

     # Send msg through file pipe
     $imap->message_to_file($pipe, $msg);

     # Close the messgae pipe
     close $pipe
       or die("Formail Close Pipe Error: $!");
   }

   # Close the folder
   $imap->close($box);
 }

 # We're all done with IMAP here
 $imap->logout();

Example 2:
In this next example, we are going to take advantage of some other methods that are provided by this useful little module. We will begin by using the same base as is in Example 1, but we will add some nuances in the middle for functionality.

Note: Messages don’t get immediately deleted with IMAP, only marked for deletion. They aren’t actually deleted until the box is expunged. In this case, it gets done after the looping over each mailbox is complete. This is to say that if the program gets interrupted in the middle that the messages won’t be deleted until the mailbox is officially issued an expunge command.

# Always be safe
use strict;
use warnings;

# Use the module
use Mail::IMAPClient;

 $imap = Mail::IMAPClient->new( Server  => 'mail.server.com:143',
                                User    => 'me',
                              Password  => 'mypass')
        # module uses eval, so we use $@ instead of $!
        or die "IMAP Failure: $@";

 foreach my $box qw( HAM SPAM ) {
   # How many msgs are we going to process
   print "There are ". $imap->message_count($box).
          " messages in the $box folder.\n";

   # Which file are the messages going into
   my $file = "mail/$box";

   # Select the mailbox to get messages from
   $imap->select($box)
        or die "IMAP Select Error: $!";

   # Store each message as an array element
   my @msgs = $imap->search('ALL')
        or die "Couldn't get all messages\n";

   # Loop over the messages and store in file
   foreach my $msg (@msgs) {
     # Pipe msgs through 'formail' so they are stored properly
     open my $pipe, "| formail >> $file"
       or die("Formail Open Pipe Error: $!");

     # Skip the msg if its over 100k
     if ($imap->size($msg) > 100000) {
       $imap->delete_message($msg);
       next;
     }

     # Send msg through file pipe
     $imap->message_to_file($pipe, $msg);

     # Close the messgae pipe
     close $pipe
       or die("Formail Close Pipe Error: $!");

     # Delete each message after downloading
     $imap->delete_message($msg);

     # If we are just testing and want to leave
     #  leave the messages untouched, then we
     #  can use the following line of code
     # $imap->deny_seeing($msg);
   }

   # Expunge and close the folder
   $imap->expunge($box);
   $imap->close($box);
 }

 # We're all done with IMAP here
 $imap->logout();

Description: Sys::Hostname is a relatively small, but very useful module. Just as the module name describes, it gets your system’s hostname. To paraphrase the module’s POD documentation, it will try every conceivable way to get the hostname of the current machine.

CPAN: Sys::Hostname

Example:
The one and only use for this module.

# Always be safe
use strict;
use warnings;

# Use the module
use Sys::Hostname;

# Get the hostname
my $host = Sys::Hostname::hostname();

# The above line can also be written as
#my $host = hostname;

print "You are on: $host\\n";

Description: When taking user input through any number of forms, there could be characters that you aren’t expecting. This is exactly what HTML::Entities was designed to handle. When getting the user input, it converts it into a form that can help in mitigating certain types of web based scripting attacks.

CPAN: HTML::Entities

Example 1:
The general example of using

is probably also the most common. It basically says to encode everything in the string that its possible to encode.

# Always be safe and smart
use strict;
use warnings;

# Use the module
use HTML::Entities;

 my $html = "bad stuff here&#$%";
 $html = encode_entities($html);
 print "HTML: $html\\n";

__OUTPUT__
HTML: bad stuff here&#0

Example 2:
This is the slightly more specific example as it uses only specific sets of characters as the “unsafe” characters.

# Always be safe and smart
use strict;
use warnings;

# Use the module
use HTML::Entities;

 my $html = "bad stuff here&#$%";
 $html = encode_entities($html, "\\x80-\\xff");
 print "HTML: $html\\n";

__OUTPUT__
HTML: bad stuff here&#0

Example 3:
This is an example of

which does the reverse. It checks the string to see if there are any HTML encoded characters and decodes them into their Unicode equivalent. This is the general version of

which is similar to the version of

demonstrated in Example 1.

# Always be safe and smart
use strict;
use warnings;

# Use the module
use HTML::Entities;

 my $html = "encoded: bad stuff here&#0";
 $html = decode_entities($html);
 print "Unicode: $html\\n";

__OUTPUT__
Unicode: encoded: bad stuff here&#0