Erics Tech Blog » System Administration http://eric.lubow.org Thoughts, musings, and other idealistic (sometimes useful) systems and development hoopla. Fri, 18 Nov 2011 14:56:45 +0000 en hourly 1 http://wordpress.org/?v=3.1.4 Distributed Flume Setup With an S3 Sink http://eric.lubow.org/2011/system-administration/distributed-flume-setup-with-an-s3-sink/ http://eric.lubow.org/2011/system-administration/distributed-flume-setup-with-an-s3-sink/#comments Fri, 04 Feb 2011 07:15:03 +0000 eric http://eric.lubow.org/?p=820 I have recently spent a few days getting up to speed with Flume, Cloudera‘s distributed log offering. If you haven’t seen this and deal with lots of logs, you are definitely missing out on a fantastic project. I’m not going to spend time talking about it because you can read more about it in the users guide or in the Quora Flume Topic in ways that are better than I can describe it. But I will tell you about is my experience setting up Flume in a distributed environment to sync logs to an Amazon S3 sink.

As CTO of SimpleReach, a company that does most of it’s work in the cloud, I’m constantly strategizing on how we can take advantage of the cloud for auto-scaling. Depending on the time of day or how much content distribution we are dealing with, we will spawn new instances to accommodate the load. We will still need the logs from those machines for later analysis (batch jobs like making use of Elastic Map Reduce).

I am going to attempt to do this as step by step as possible but much of the terminology I use is described in the users guide and there is an expectation that you have at least skimmed it prior to starting this HOWTO. I am using EMR (Elastic Map Reduce) on EC2 and not the provided Hadoop by Cloudera. Additionally, the Cloudera version that I am working with is cdh3b3.

Context
I have 3 kinds of servers all running CentOS in the Amazon cloud:

  1. a1: This is the agent which is producing all the logs
  2. c1: This is the collector which is aggregating all the logs (from a1, a2, a3, etc)
  3. u1: This is the flume master node which is sending out all the commands

There are actually n agents, but for this example, we’ll keep it simple. Also, for a complete copy of the config files, please check out the full gist available here.

Initial Setup
On both a1 and c1, you’ll have to install flume-node (flume-node contains the files necessary to run the agent or the collector).

# curl http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo > /etc/yum.repos.d/cloudera-cdh3.repo
# yum update yum
# yum install flume flume-node

On u1, you’ll need to install the flume-master RPM:

# curl http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo > /etc/yum.repos.d/cloudera-cdh3.repo
# yum update yum
# yum install flume flume-master

On each host, you need to copy the conf template file to the site specific config file. That is to say:

cp flume-site.xml.template flume-site.xml

First let’s jump onto the agent and set that up. Tune the $master_IP and $collector_IP variables appropriately, but change your /etc/flume/conf/flume-site.xml to look like:

<configuration>
  <property>
    <name>flume.master.servers</name>
    <value>$master_IP</value>
    <description>This is the address for the config servers status server (http)</description>
  </property>

  <property>
    <name>flume.collector.event.host</name>
    <value>$collector_IP</value>
    <description>This is the host name of the default "remote" collector.</description>
  </property>

  <property>
    <name>flume.collector.port</name>
    <value>35853</value>
    <description>This default tcp port that the collector listens to in order to receive events it is collecting.</description>
  </property>

  <property>
    <name>flume.agent.logdir</name>
    <value>/mnt/flume-${user.name}/agent</value>
    <description> This is the directory that write-ahead logging data
      or disk-failover data is collected from applications gets
      written to. The agent watches this directory.
    </description>
  </property>
</configuration>

Now on to the collector. Same file, different config. Replace all the variables with you $master IP address (you should be using Amazon’s internal IPs otherwise you will be paying the regional charge). The $account and $secret variables are both your Amazon EC2/S3 account key and secret Access key respectively. The $bucket is the S3 bucket that will contain the log files. Also worthy of pointing out is the flume.collector.roll.millis and flume.collector.dfs.compress.gzip. The millis is how frequently the log file gets truncated and the next file begins to be written to. It would be nice if this could be done by file size and not only by time, but it works for now. The other config option is flume.collector.dfs.compress.gzip. This ensures that the logfiles are compressed prior to being dumped onto S3 (saves LOTS of space).

<configuration>
  <property>
    <name>flume.master.servers</name>
    <value>$master</value>
    <description>This is the address for the config servers status server (http)</description>
  </property>


  <property>
    <name>flume.collector.event.host</name>
    <value>localhost</value>
    <description>This is the host name of the default "remote" collector.</description>
  </property>

  <property>
    <name>flume.collector.port</name>
    <value>35853</value>
    <description>This default tcp port that the collector listens to in order to receive events it is collecting.</description>
  </property>

  <property>
    <name>fs.default.name</name>
    <value>s3n://$account:$secret@$bucket</value>
  </property>

  <property>
    <name>fs.s3n.impl</name>
    <value>org.apache.hadoop.fs.s3native.NativeS3FileSystem</value>
  </property>
 
  <property>
    <name>fs.s3.awsAccessKeyId</name>
    <value>$account</value>
  </property>
 
  <property>
    <name>fs.s3.awsSecretAccessKey</name>
    <value>$secret</value>
  </property>

  <property>
    <name>fs.s3n.awsAccessKeyId</name>
    <value>$account</value>
  </property>
 
  <property>
    <name>fs.s3n.awsSecretAccessKey</name>
    <value>$secret</value>
  </property>

  <property>
    <name>flume.agent.logdir</name>
    <value>/mnt/flume-${user.name}/agent</value>
    <description> This is the directory that write-ahead logging data
      or disk-failover data is collected from applications gets
      written to. The agent watches this directory.
    </description>
  </property>
 
  <property>
    <name>flume.collector.dfs.dir</name>
    <value>file:///mnt/flume-${user.name}/collected</value>
    <description>This is a dfs directory that is the the final resting
    place for logs to be stored in.  This defaults to a local dir in
    /tmp but can be hadoop URI path that such as hdfs://namenode/path/
    </description>
  </property>  

  <property>
    <name>flume.collector.dfs.compress.gzip</name>
    <value>true</value>
    <description>Writes compressed output in gzip format to dfs. value is
     boolean type, i.e. true/false</description>
  </property>

  <property>
    <name>flume.collector.roll.millis</name>
    <value>60000</value>
    <description>The time (in milliseconds)
    between when hdfs files are closed and a new file is opened
    (rolled).
    </description>
  </property>
</configuration>

While we are still on the collector, in order to properly write to S3, you’ll need to make 4 file adjustments and all of them will go into the /usr/lib/flume/lib/ directory.

  1. commons-codec-1.4.jar
  2. jets3t-0.6.1.jar
  3. commons-httpclient-3.0.1.jar
  4. emr-hadoop-core-0.20.jar

The one thing that should be noted here is that the emr-hadoop-core-0.20.jar file replaces the hadoop-core.jar symlink. The emr-hadoop-core-0.20.jar file is the hadoop-core.jar file from an EC2 Hadoop cluster instance. Note: This will break the ability to seamlessly upgrade via the RPM (which is how you installed it if you’ve been following my HOWTO). Keep these files around just in case. I have added a tarball of the files here, but they are all still available with a quick Google search.

And now on to the master. There was actually no configuration that I did on the master file system to get things up and running. But if flume is writing to a /tmp directory on an ephemeral file system, then it should be fixed.

Web Based Setup

I chose to do the individual machine setup via the master web interface. You can get to this pointing your web browser at http://u1:35871/ (replace u1 with public DNS IP of your flume master). Ensure that the port is accessible from the outside through your security settings. At this point, it was easiest for me to ensure all hosts running flume could talk to all ports on all other hosts running flume. You can certainly lock this down to the individual ports for security once everything is up and running.

At this point, you should go to a1 and c1 run /etc/init.d/flume-node start. If everything goes well, then the master (whose IP is specified in their configs) should be notified of their existence. Now you can configure them from the web. Click on the config link and then fill in the text lines as follows (use what is in bold):

  • Agent Node: $agent_ec2_internal_ip
  • Source: tailDir(“/mnt/logs/”,”.*.log”)
  • Sink: agentBESink(“$collector_ec2_internal_ip”,35853)

Note: I chose to use tailDir since I will control rotating the logs on my own. I am also using agentBESink because I am ok with losing log lines if the case arises.

Now click Submit Query and go back to the config page to setup the collector:

  • Agent Node: $collector_ec2_internal_ip
  • Source: collectorSource(35853)
  • Sink: collectorSink(“s3n://$account:$secret@$bucket/logs/%Y/%m/%d/%H00″,”server”)

This is going to tell the collector that we are sinking to s3native with the $account key and the $secret key into the $bucket with an initial folder of ‘logs’. It will then log to sub-folders with YYYY/MM/DD/HH00 (or 2011/02/03/1300/server-.log). There will be 60 gziped files in each folder since the timing is setup to be 1 file per minute. Now click Submit Query and go to the ‘master’ page and you should see 2 commands listed as “SUCCEEDED” in the command history. If they have not succeeded, ensure a few things have been done (there are probably more, but this is a handy start:

  1. Always use double quotes (“) since single quotes (‘) aren’t interpreted correctly. UPDATE: Single quotes are interpreted correctly, they are just not accepted intentionally (Thanks jmhsieh)
  2. In your regex, use something like “.*\\.log” since the ‘.’ is part of the regex.
  3. In your regex, ensure that your blackslashes are properly escaped: “foo\\bar” is the correct version of trying to match “foo\bar”.
  4. Ensure any ‘/’ are inserted as ‘%2F’ in the Amazon account and secret codes.

Additionally, there are also tables of Node Status and Node Configuration. These should match up with what you think you configured.

At this point everything should work. Admittedly I had a lot of trouble getting to this point. But with the help of the Cloudera folks and the users on irc.freenode.net in #flume, I was able to get things going. The logs sadly aren’t too helpful here in most cases (but look anyway cause they might provide you with more info than they provided for me). If I missed anything in this post or there is something else I am unaware of, then let me know.

References

Related posts:

  1. Pig Queries Parsing JSON on Amazons Elastic Map Reduce Using S3 Data
  2. File::Find

]]> http://eric.lubow.org/2011/system-administration/distributed-flume-setup-with-an-s3-sink/feed/ 6 Sharing a Screen Session http://eric.lubow.org/2010/system-administration/sharing-a-screen-session/ http://eric.lubow.org/2010/system-administration/sharing-a-screen-session/#comments Fri, 23 Jul 2010 12:47:23 +0000 eric http://eric.lubow.org/?p=762 Anyone who has spent any time in a shell and has been cut off while working should know about screen. If not, then I recommend reading up on it (here or here). But I’m not here to tell you about screen as a general tool, I want to show you how to use it for screen sharing. I found a couple of forum posts and other scattered information, so here’s a little centralizing of information.

First a definition, sharing a screen session means both users can at the very least view the same screen (console) at the same time. And depending on the security settings, they can both type as well (which can be very annoying depending on who your sharing the screen session with). I’m going to assume you know the basics so we’ll go on from there.

Let’s assume that I elubow want to share my console with jdoe. After screen is installed, these next 2 steps are critical otherwise the sessions will not be accessible to other users regardless of which commands you issue inside of the screen session:

$ sudo chmod +s /usr/bin/screen  # Make screen suid root
$ sudo chmod 755 /var/run/screen # Make the screen dir more open

The easiest way to do this next part is to actually name your screen session. So let’s call our session test.

$ screen -S test

This will create the session and put you in it. Now we need to add the access control. Note: You can automate the multiuser piece by adding it to you .screenrc.

CTRL-A
:multiuser on

CTRL-A
:acladd jdoe

All jdoe has to do now is connect to the same machine and type the following command:

$ screen -x elubow/test   # $user/$session_name

There are other security combinations as the above gives jdoe full access. Here are a few common ones:

:aclchg user +rx  # Give 'user' read-only access
:aclchg user -x "#,at,aclchg,acladd,acldel,quit"  # Don't allow 'user' any of the previous commands
:aclchg user +rwx  # Give 'user' read-write access (open permission on the session)

Related posts:

  1. Cluster SSH with cSSHx
  2. 10 Tips To Start Securing Your Linux System
  3. SSH Organization Tips

]]> http://eric.lubow.org/2010/system-administration/sharing-a-screen-session/feed/ 1 Creating Configuration Files With Ruby Templates http://eric.lubow.org/2010/system-administration/creating-configuration-files-with-ruby-templates/ http://eric.lubow.org/2010/system-administration/creating-configuration-files-with-ruby-templates/#comments Wed, 30 Jun 2010 11:45:58 +0000 eric http://eric.lubow.org/?p=749 I recently had a very repetitive configuration file that needed creating. There were approximately 50 config blocks of 10 lines each with only the host name changing with each block. So I decided to take a shortcut and do it in Ruby using ERB templates. This is so easy and literally save me hours worth of work.

I started out by creating a template for the rsyslog block I wanted to replicate. In the full version of the script, there is an additional section for the access.log file which you’ll see below. If you know Rails, then this template style should look very familiar.

rsyslog_block = %{#
# < %= virtual_host %>
#
$InputFileName /var/log/httpd/< %= virtual_host %>-error.log
$InputFileTag < %= virtual_host %>_error
$InputFileStateFile stat-< %= virtual_host %>-error.log
$InputFileSeverity info
$InputFileFacility local5
$InputRunFileMonitor

}

The below block of code creates a configuration block (above) for each of virtual hosts in the array and then prints it to the screen.

%w{
    vhost.com
    bar.myhost.com
}.each { |virtual_host|  puts ERB.new( rsyslog_block ).result(binding) }

This is going to produce the following block which you can then pipe to a file:

...Removed for brevity...
#
# bar.myhost.com
#
$InputFileName /var/log/httpd/bar.myhost.com-error.log
$InputFileTag bar.myhost.com_error
$InputFileStateFile stat-bar.myhost.com-error.log
$InputFileSeverity info
$InputFileFacility local5
$InputRunFileMonitor

Here is the script in its entirety:

Related posts:

  1. Parsing Ini Files With Ruby
  2. Multiple Input Locations From Bash Into Ruby
  3. Interesting Object Methods in Ruby

]]> http://eric.lubow.org/2010/system-administration/creating-configuration-files-with-ruby-templates/feed/ 2 Nagios notify-by-campfire Plugin http://eric.lubow.org/2010/system-administration/nagios-notify-by-campfire-plugin/ http://eric.lubow.org/2010/system-administration/nagios-notify-by-campfire-plugin/#comments Thu, 06 May 2010 09:45:01 +0000 eric http://eric.lubow.org/?p=712 Since one of the core communication methods for my company amongst engineers is 37Signals Campfire and Nagios is one of our main monitoring tools for all of our applications and services, I thought it would be a good idea to combine the two. So with a few simple additions to the Nagios configuration and a Ruby Campfire script, you can get this up and running.

On the Campfire side, I will leave the exercise of adding a Campfire user up to you. You may need to go through your Campfire administrator for that.

On the Nagios side, the first thing that needs to happen is that you need to add the following commands to your command.cfg file in your Nagios configuration. Make sure that you specify the proper location of your notify-by-campfire.rb file.

# 'notify-host-by-campfire' command definition
define command{
    command_name    notify-host-by-campfire
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\nHost: $HOSTNAME$\nState: $HOSTSTATE$\nAddress: $HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/local/bin/ruby /usr/local/bin/notify-by-campfire.rb -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **"
    }  

# 'notify-service-by-campfire' command definition
define command{
    command_name    notify-service-by-campfire
    command_line    /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type: $NOTIFICATIONTYPE$\n\nService: $SERVICEDESC$\nHost: $HOSTALIAS$\nAddress: $HOSTADDRESS$\nState: $SERVICESTATE$\n\nDate/Time: $LONGDATETIME$\n\nAdditional Info:\n\n$SERVICEOUTPUT$" | /usr/local/bin/ruby /usr/local/bin/notify-by-campfire.rb -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **"
    }

Add this contact template definition for the Campfire contact to your contacts.cfg. Don’t forget to add it to a contactgroup.

define contact{
        name                            campfire-contact
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    w,u,c,r,f,s
        host_notification_options       d,u,r,f,s
        service_notification_commands   notify-service-by-campfire
        host_notification_commands      notify-host-by-campfire
        register                        0
        }

The Ruby code for notify-by-campfire.rb is here. Just install it wherever you’d like and as I said above, make sure its specified properly in your commands.cfg.

Note: I am aware that I could have made this script much shorter, but my goal is to make this extensible should I want to provide more information or change the format. Feel free to modify for your own usage.

Update (2010-05-17): To make it a little easier to read in the context of a smaller Campfire window, I have amended the notify-host-by-campfire and notify-service-by-campfire entries to be shorter:

# 'notify-host-by-campfire' command definition
define command{
    command_name    notify-host-by-campfire
    command_line    /usr/bin/printf "%b" "Date/Time: $LONGDATETIME$\n\nHost: $HOSTNAME$\nType/State: $NOTIFICATIONTYPE$/$HOSTSTATE$\n\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | /usr/local/bin/ruby /usr/local/bin/notify-by-campfire.rb -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ ($HOSTADDRESS$) is $HOSTSTATE$ **"
    }

# 'notify-service-by-campfire' command definition
define command{
    command_name    notify-service-by-campfire
    command_line    /usr/bin/printf "%b" "Date/Time: $LONGDATETIME$\nType/State: $NOTIFICATIONTYPE$/$SERVICESTATE$\n\n$SERVICEOUTPUT$" | /usr/local/bin/ruby /usr/local/bin/notify-by-campfire.rb -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ ($HOSTADDRESS$) is $SERVICESTATE$ **"
    }

Related posts:

  1. Monitoring Services with Nagios::Plugin
  2. Sys::Hostname

]]> http://eric.lubow.org/2010/system-administration/nagios-notify-by-campfire-plugin/feed/ 2 Creating Dummy Packages On Debian http://eric.lubow.org/2010/system-administration/creating-dummy-packages-on-debian/ http://eric.lubow.org/2010/system-administration/creating-dummy-packages-on-debian/#comments Tue, 04 May 2010 11:00:37 +0000 eric http://eric.lubow.org/?p=708 One of my favorite things about Debian is its awesome package management system. Apt is one of the reasons I have used Debian for servers for so many years and eased my initial transition to Ubuntu (which as most people know was initially a Debian fork). Apt is a great tool as long as you aren’t building packages from source (and not making debs out of them). I have packaged a whole bunch of debs, but sometimes it just isn’t necessary. So if you haven’t used equivs, then you need to check it out.

Essentially it allows you to create dummy debs to provide packages that you installed from source (circumvent dependency checking). It can get dirty when time comes to do dist-upgrades, but it has its usefulness. For example, I wanted to install altermime and I had installed the latest Postfix from source and altermime requires an MTA to be installed otherwise it would install exim (amongst other packages). So I built a dummy packages doing the following:

# apt-get install equivs
# equivs-control postfix

That will create a file named postfix in the current directory. Make the file look similar to this:

Section: misc
Priority: optional
Standards-Version: 2.3.3

Package: postfix-dummy
Version: 2.7.0
Section: mail
Maintainer: Eric Lubow
Provides: mail-transport-agent
Architecture: all
Description: Dummy Postfix package

Then go ahead and build your dummy package:

# equivs-build postfix
dh_testdir
dh_testroot
dh_clean -k
dh_testdir
dh_testroot
dh_install
dh_installdocs
dh_installchangelogs
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb
dpkg-deb: building package `postfix-dummy' in `../postfix-dummy_2.7.0_all.deb'.

The package has been created.
Attention, the package has been created in the current directory,
not in ".." as indicated by the message above!

Then I just installed the postfix-dummy package with:

# dpkg -i postfix-dummy_2.7.0_all.deb

And everything worked beautifully. I installed altermime with no problem and we were off to the races.

Related posts:

  1. Deploying Amavisd-logwatch
  2. Creating Configuration Files With Ruby Templates
  3. A Few Words About Setting Up Postfix Multi Instance

]]> http://eric.lubow.org/2010/system-administration/creating-dummy-packages-on-debian/feed/ 2 Monitoring Services with Nagios::Plugin http://eric.lubow.org/2010/perl/perl-modules/monitoring-services-with-nagios-plugin/ http://eric.lubow.org/2010/perl/perl-modules/monitoring-services-with-nagios-plugin/#comments Wed, 07 Apr 2010 11:15:03 +0000 eric http://eric.lubow.org/?p=664 There are a lot of people who say, “if it isn’t monitored, then it isn’t a service.” The problem is that I don’t think enough people outside of the systems world believe that or even understand why its said. I think the primary offenders here are developers. It isn’t because they don’t know better, but typically developers just want to get the application up and running and then move on to developing the next thing. I also think there is some fault on the side of the administrators and the managers not insisting that part of the completed version of a project includes monitoring. But I don’t want to harp on this as much as I would like to show just how easy it is to compensate here by taking advantage of Nagios::Plugin.

Nagios::Plugin is a Perl module that can be used to shortcut how easy it is to add a custom check for your application to your Nagios monitoring infrastructure. It is so easy to write a Nagios plugin that even with comments and performance data, I have written one to check the alive-ness in under 50 lines.

use Nagios::Plugin;
use Time::HiRes qw( gettimeofday tv_interval );

my $np = Nagios::Plugin->new(
    shortname => 'template',
);

 # We need a URL to test
 my $url = "http://www.myapp.com/";

 # Create the UserAgent
 my $ua = LWP::UserAgent->new (
    agent       =>  'Nagios Application Check',
    from        =>  'webmaster@myapp.com',
 );

 # Change the timeout to 10 seconds instead of 3 min (180 seconds)
 $ua->timeout( 10 );

 my $t0 = [gettimeofday];
 my $resp = $ua->get($url);
 my $total = tv_interval($t0, [gettimeofday]);

 my $content;
 if (!$resp->is_success) {
    $np->nagios_exit( CRITICAL, $resp->status_line);
 }

 # Time it
 $np->add_perfdata(
    label => "time",
    value => "${total}",
    uom => "s",
 );

 $np->nagios_exit(OK, "Test Suggestions Found");

exit 0;

By reading the Pod documentation, you’ll also find that you can get a lot more specific. You can add command line switches, lots of performance data, and my favorite, some pretty extensive success, warning or critical messages that can be displayed on the dashboard.

Related posts:

  1. Nagios notify-by-campfire Plugin
  2. Textmate Minimap Plugin

]]> http://eric.lubow.org/2010/perl/perl-modules/monitoring-services-with-nagios-plugin/feed/ 1 Cluster SSH with cSSHx http://eric.lubow.org/2010/system-administration/cluster-ssh-with-csshx/ http://eric.lubow.org/2010/system-administration/cluster-ssh-with-csshx/#comments Mon, 29 Mar 2010 09:00:21 +0000 eric http://eric.lubow.org/?p=635 I am in the middle of building out a group of about 25 machines in a data center for my company. I hadn’t really dove into it on a micro level until a few days ago. I was moving around on individual machines that others were working on. When I had gotten to one of the “untouched” machines, I found that vim wasn’t installed. There was about 15 machines that were “untouched” and therefore were missing vim (along with other stuff). And seriously who wants to install a bunch of the same software on every machine after they’ve already been kickstarted?

So I could try to script it, but you really only get one shot when you script it. Then you have to deal with expect (or log in as root) and that’s a little bit of a pain too. So I decided to see what cluster SSH solutions are out there (since I haven’t needed them in a while). The kicker was that it had to work for Mac OS X. Typically anything that works on Linux has a variation on the Mac so I didn’t think that would be a problem.

First thing I found was a Perl application bundle that uses Tk to be a cluster SSH solution. Its available here. I had a little trouble getting some of the required CPAN modules installed and because I like instant gratification (plus I needed to get some stuff done), I had to pass.

The next thing I found was cSSHx and it’s just awesome. All you do is untar it, put it in your path and launch it as follows:

# csshx --login elubow 192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 192.168.1.5

One of the nice things is that it equally sizes all the Terminal windows that it opens so that they share equal screen real estate. It provides a master typing bar to allow you to see what you’re typing in all sessions. And if you need to something to one particular machine, just click on that machine’s Terminal window and only that session will receive input (since it is just a Terminal session after all). Here is a screen shot of all of the 25 hosts open at once on my screen.

Screenshot of cSSHx

Screenshot of cSSHx with 20 sessions

This is an awesome piece of software. There are only 2 additional features that I think could make it better. I would like to be able to open up a range if the IPs are sequential. For instance 192.168.1.1/30 opens 192.168.1.1 and 192.168.1.2 (but on a much larger scale). This is easier than creating a list each time I want to go to just a subset of machines. I think it would also be great to be able to specify the password on the command line. I know security folks would go nuts over this, but in the long run, sometimes its just easier to put the password in the command line.

I want to leave you with 1 more thing. Although this is a nice tool, it can be even better with aliases. Add a bunch of groups to your .bashrc or you .bash_profile. Something along the lines of:

alias csshmail='csshx --login elubow 192.168.1.3 192.168.1.4 192.168.1.5'
alias csshdb='csshx --login elubow 192.168.1.35 192.168.1.34 192.168.1.37'

Related posts:

  1. Sharing a Screen Session
  2. SSH Organization Tips
  3. SSH Over The Web With Web Shell

]]> http://eric.lubow.org/2010/system-administration/cluster-ssh-with-csshx/feed/ 9 Git Branch Name in Your Bash Prompt http://eric.lubow.org/2009/system-administration/git-branch-name-in-your-bash-prompt/ http://eric.lubow.org/2009/system-administration/git-branch-name-in-your-bash-prompt/#comments Fri, 11 Dec 2009 11:50:32 +0000 eric http://eric.lubow.org/?p=461 I work with a few repositories at any given time. And during that time, I typically have multiple branches created for each repository. I figured that it would make my life easier if I knew which branch and/or repository I was working in. Luckily, very little hackery is required here since the git distribution already comes with such a tool. (Note: If you didn’t build Git from source, then you may not have this file.)

First thing I did was copy the git auto-completion file into my home directory:

elubow@beacon repo$ cp /usr/local/src/git/contrib/git-completion.bash ~/.git-completion.sh

Then I edited my .bashrc file to load up my new PS1 variable (PS1 is variable name for the Bash prompt) with git information only if we are currently in a repository. Don’t forget to set a default PS1 in case your .bashrc can’t find your .git-completion.sh file. Another item of note is that you should have the if state below load after your load your PATH variable. This removes the potential for git having to be located after every commend.

# Set the base PS1
PS1="\u:\h \W$ "

# Source the git bash completion file
if [ -f ~/.git-completion.sh ]; then
    source ~/.git-completion.sh
    GIT_PS1_SHOWDIRTYSTATE=true
    PS1='\u@\h$(__git_ps1 " (%s)") \W$ '
fi

export PS1

You’ll notice I also set a variable called GIT_PS1_SHOWDIRTYSTATE. What this variable does is it adds a * to the branch name if the branch has been changed.

There are other options here if you are looking for something more in depth. Git Prompt provides you with a much more in depth version on the command line fo your dirty files and is much more visually oriented.

Additionally, you now get to make use of the other auto-complete features that this script was actually intended for.

elubow@beacon (master*) repo$ git sta
stage    stash    status

Note the difference between my prompt at the beginning of the post and the end. When I am not in a repository, nothing shows in my PS1 variable.

It’s just that simple to do this and definitely saves some time. Enjoy.

Related posts:

  1. Converting From Subversion To Git
  2. Printing New Lines in Bash
  3. Git Command Aliases

]]> http://eric.lubow.org/2009/system-administration/git-branch-name-in-your-bash-prompt/feed/ 2 Creating a Slave DNS Server on Bind9 http://eric.lubow.org/2009/system-administration/creating-a-slave-dns-server-on-bind9/ http://eric.lubow.org/2009/system-administration/creating-a-slave-dns-server-on-bind9/#comments Sun, 29 Nov 2009 16:10:13 +0000 eric http://eric.lubow.org/?p=174 I couldn’t find a quick and dirty list of commands for setting up a slave DNS server so I figured I would just throw it together.

Starting with a fully working primary name server, we are going to set up a slave name server. We are going to make the following assumptions:
primary – 1.2.3.4
slave – 4.5.6.7
* We want to have the domain example.com have a slave name server

On the primary (or master) name server, add the following lines to the options section.

options {
    allow-transfer { 4.5.6.7; };
    notify yes;
};

Ensure that you update the serial number in the SOA on the master. Then run:

# rndc reload

On the slave name server, add the following entry to the named.conf file (or whichever file houses your zone entries). Ensure that the path leading up to the zone file exists and that bind has write access to that directory.

 zone "example.com"  { type slave; file "/etc/bind9/zones/example.com.slave"; masters { 1.2.3.4; }; };

Then once you made the changes to the slave, you will need to reload the configuration. Do this the same way you did on the master:

# rndc reload

If you watch your DNS log, you should see the transfer happen as soon as you restart both named servers.

Related posts:

  1. Cloning a Virtual Machine in VMWare VI3 without Virtual Server
  2. Creating Dummy Packages On Debian
  3. Database Read/Write Splitting in Frameworks/ORMs

]]> http://eric.lubow.org/2009/system-administration/creating-a-slave-dns-server-on-bind9/feed/ 0 SSH Over The Web With Web Shell http://eric.lubow.org/2009/system-administration/ssh-over-the-web-with-web-shell/ http://eric.lubow.org/2009/system-administration/ssh-over-the-web-with-web-shell/#comments Fri, 27 Nov 2009 11:45:46 +0000 eric http://eric.lubow.org/?p=431 After reading a Tweet from Matt Cutts about being able to SSH from the iPhone (and the web in general), I had to give it a try. I am always looking for better ways to be able to check on systems when necessary. I have iPhone apps for SSHing around if I need as well, but like with any “new” tool, I have to try it out to see if it serves a purpose or makes my admin life easier in any way.

First go check out the Google Code repository for Web Shell. Webshell is written in Python and is based on Ajaxterm. All that’s required is SSL And Python 2.3 or greater. It works on any browser that has Javascript and can make use of AJAX.

The way Web Shell works is you start it up on a server and then can use a web browser to access only that machine over SSH. The works best if you have a gateway server to a network and use a single point of entry to access the rest of the servers. Web Shell runs on HTTPS on port 8022. Reading the README will lead you through the same set of instructions I used below. Once installed, we connect by using a web browser: https://server.com:8022/

elubow@server:/home/web/webshell.lubow.org-8022$ ./make_certificate.sh
Generating a 1024 bit RSA private key
...............................................................++++++
...........................................++++++
writing new private key to 'webshell.pem'
-----
elubow@server:/home/web/webshell.lubow.org-8022$ ./webshell.py
The python SSL extensions seem to be not installed.
You can run WebShell without SSL encryption with the --ssl-disable command line switch.

You’ll notice that while it is possible to run the terminal without SSL, it is just not in our best interest. So let’s go ahead and install it. I use Debian on my servers. So in my case, it’s as simple as:

server:~# apt-get install python-pyopenssl python-openssl
Get:1 http://ftp.de.debian.org lenny/main python-openssl 0.7-2 [88.9kB]
Get:2 http://ftp.de.debian.org lenny/main python-pyopenssl 0.7-2 [9.0kB]
Unpacking python-openssl (from .../python-openssl_0.7-2_i386.deb) ...
Setting up python-openssl (0.7-2) ...
Setting up python-pyopenssl (0.7-2) ...
Processing triggers for python-support ...

Then we run the server:

elubow@server:/home/web/ssh.lubow.org-8022$ ./webshell.py
WebShell (https) at 0.0.0.0, port 8022

I’ve included a screenshot from my iPhone below. The background is interchangeable with any other transparent background image. If you’re in need of a quick SSH or someone else’s smart phone has service and yours doesn’t with your fancy SSH programs, then this is definitely a cool and viable alternative.

Web Shell SSH on iPhone

Web Shell SSH on iPhone
Web Shell SSH with keyboard out

Web Shell SSH with keyboard out

Related posts:

  1. MySQL for Python

]]> http://eric.lubow.org/2009/system-administration/ssh-over-the-web-with-web-shell/feed/ 3