Working on groups of web servers, especially ones that are highly susceptible to attack, it is a good idea to have a string of commands that will allow you to check what is going on.
Check for DDos:
1 | netstat -n | grep EST | awk '{ print $5 }' | cut -d: -f1 | sort | uniq -c | sort -nr | perl -an -e 'use Socket; ($hostname, @trash) = gethostbyaddr(inet_aton($F[1]), AF_INET); print "$F[0]\t$F[1]\t$hostname\n";' |
Using this command will produce a list of hostnames that have a connect to the machine in an ESTABLISHED state. This is handy for creating a firewall rule either on the host (iptables, ipfw) or a little further away from the machine (at the edge router).
Check for web attacks:
1 | cat eric.lubow.org-access_log.20081015 | awk '{print $1 }' | sort | uniq -c | sort -nr | head | perl -an -e 'use Socket; ($hostname, @trash) = gethostbyaddr(inet_aton($F[1]), AF_INET); print "$F[0]\t$F[1]\t$hostname\n";' |
By using this command, you will get a hostname lookup on the IP sorted by total hit count descending. As when checking for DDos attacks, you can use this information to write firewall rules.
More web attack checks:
1 | for i in `ls *.20081015 | grep -v error`; do echo "##### $i ######"; tail -n 10000 $i| awk '{print $1};' | sort -n | uniq -c | sort -nr | head -2; done |
The difference between this check and the previous check is that this time, you may have a lot more logfiles to go through. I am also assuming that they are stored by
Referrer Check:
1 | for file in `ls -lrS *access*20080525* | tail -n20`; do echo "==========" $file; gawk --re-interval -F'"' '{ split($4, myrt, "/"); split($0, myct); split(myct[3], myc, " "); if (length(myrt[3])==0) { myrt[3]="none"}; if (myrt[3] ~ /([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}/) { referrers[myrt[3]"/"myc[1]]++; } else { t=split(myrt[3], myrt2, "."); myref="*."myrt2[t-1]"."myrt2[t]; referrers[myref"/"myc[1]]++; } } END { for (referrer in referrers) { print referrers[referrer], referrer } }' $file | grep -v none | sort -n; done |
This last check will get the referrer for a page from the logs and count up the number of times that exact referrer drives traffic to your page. Although this may initially appear to be only tangentially useful, if you are getting DDos, it may be hard to track down. Let’s say that you have some static content like a funny image and want to know why everyone is going to that image. Maybe your getting Dugg or ./ and this will help you tell (and find out what your page is so you can Digg yourself if you’re into that sort of thing).
