The first and foremost reason that I write blog posts is that engineers who spend a lot of time figuring things out on the fly could use a helping hand. A lot of that figuring is done piecing together parts of other people’s solutions to problems from various blogs and papers. Every time I run into an issue or fix a problem, I try to write a blog post about it. I don’t do this because I feel that I have more to offer than anyone else, I just feel like my work should be able to benefit others (there is no use in reinventing the wheel). And to top it off, if I do something and someone has a better way, I like hearing about it in the comments or from an email.

My personal preference when writing blog posts is when possible, short and sweet. Just do what you’re going to do, explain when necessary, but convey the idea and intention. If something needs substantiation, context, or proof then give it. Don’t just say something like, “writing your query like XXX will speed it up by 100ms” without context. I wrote a lot of queries, that run in hours; 100ms won’t mean anything to me. I also am not a huge fan of academic papers or pure blue sky optimistic benchmarks. Now that’s not to say that those papers don’t have their place, but in reality, most situations aren’t blue sky with unlimited budget and all hardware being equal.

Nirvdrum prefers academic papers with lots of peer review and benchmarks. Those types of things are great for theory, but in practice, implementing in such circumstances is not the norm. I typically lump white papers into that group as they tend to be slightly biased towards the companies that prepare them. White papers are great for ideas for solutions, but I think the real nitty gritty lies in finding the right blog post or posts to help you find your solution.

Related posts:

1. Busiest Person You Know

To put the situation in context, I basically allowed thousands of mail.* files to be created. These files littered the /var/log/ directory and basically slowed down the entire file system access. I figured out this a number of ways.

The first way was when I tried to do an ls anywhere, it would just hang. My first reaction was to check to see what was eating up the CPU. To do this, I did a top. I noticed that logrotate was hogging all the CPU cycles. Since I know that logrotate basically only operates on one parent directory (by default) /var/log, I headed on over there and did an ls. Once again, it just hung. Then I figured the file system was slow and decided to check out some file system information. The next two commands I ran were df -h and df -i. I ran the df -h to see if we were out of disk space (and yes I lazily use human readable format). I ran the second to check to see how many inodes were in use. (For more information on inodes, check out the wikipedia entry here).

Now that I know the system is short on inodes, I checked out the output of lsof. Now I know that we have some serious problems in the /var/log dir. After some quick investigation, I realized that there were too many mail.* files. How do I get rid of them? Glad you asked… Let’s assume that we want to delete ALL the mail.* files in the /var/log directory.

1) The easiest way is to do it with find:
1a) Using find‘s delete command:

or
1b) using find‘s exec command with rm:

These will work, but either will be slow since they doesn’t do batch execution.

2) A slightly more preferred way is to use bash:

This is a little faster, but will still be relatively slow since there is no batch execution. (Note: The -v in the rm will cause quite a bit of output since it is showing you EVERY file it deletes. Feel free to leave this out if you really screwed up.)

3) The actual preferred method is to use find:

I believe this is the preferred method because although it removes the files one at a time, it is more efficient for the file system since it batches it up.

There are certainly other ways to accomplish this task. It can always be done with a Perl one-liner or even using some Perl modules to save some time. These are just a few ideas to point someone in the right direction.

Related posts:

1. Parsing Ini Files With Ruby
2. SSH Organization Tips
3. File::Find