Adding Features to NSQ

After being a fairly heavy user of NSQ over the past year or so and finding that it was missing a few features, I decided to jump in and try to add them myself. The only issue was that I didn’t know Go. Since something as simple as not knowing the language the application was written in has never stopped me before, I wasn’t going to let it stop me now.
Read the rest of this entry »

Posted in NSQ. Tags: . No Comments »

Learning to Hardware Hack at RobotsConf

I’ve been a programmer (if you can call me that) for quite a few years now. But for the most part, it’s really always been about designing software based systems. Even though these systems are larger than the average startup or SaaS company would get to work with, it’s (as I said) still about designing software based systems. Enter Robots Conf. Read the rest of this entry »

Adding Cross Zone Load Balancing in AWS

One of the new hotness features that Amazon added to their Elastic Load Balancers is cross zone load balancing. This offers the ability to have an unbalanced number of nodes per availability zone within an Amazon region. For instance, if you were load balances across us-east-1a, us-east-1b, and us-east-1c, then you needed to have the same number of instances in each zone otherwise the traffic would skew and overload the zone with fewer instances. If you are auto-scaling, using spots, or just happen to lose instances from time to time, you can easily see where this becomes a problem. Read the rest of this entry »

Redis Setup Notes and One-Liners

Being a heavy user of Redis has forced some weird Bash-fu and other commands when I want to find out how things are going. Because Redis is single threaded (see here for more information), I commonly run multiple Redis instances per machine. As a result, when running on AWS, I use a specific machine layout to get the best CPU utilization for Redis. On an m2.4xlarge machine, it comes with 8 cores and 68G of RAM. To take full advantage of that I run 7 instances of Redis and pin one instance to a CPU core (this can be done using taskset in schedutils package). For extra performance, I leave an entire core to the OS (even though the machines do little other than process Redis commands. Read the rest of this entry »