Python Multiprocessing Pools and MySQL

21 Dec 2009 — eric

There really isn’t a solid Python module for multiprocessing and MySQL. Now this may be because MySQL on a single server is disk bound and therefore limited in speed or just because no one has written it. So here is a quick and dirty example using the Pool module in multiprocessing in Python 2.6 and MySQLdb.

I also tried using PySQLPool. This was designed for threading and not forking as I am doing with Pool method. Although I am sure it is possible to use PySQLPool with forking by passing the connection (pool) object down to the child process or possibly doing something with IPC, I decided to keep it simple (although slightly more expensive) and instantiate MySQLdb connections upon fork.
Read the rest of this entry »

Posted in python. Tags: , , , . 2 Comments »

Python’s MySQLdb 2014 Error – Commands out of sync

18 Dec 2009 — eric

While writing a simple Python script to access and process data in a database, I came across an error that said:

Error 2014: Commands out of sync; you can't run this command now

After quite a bit of Googling and with very little findings, I had to dive in a little and try to figure out what was going on. The whole error looked like this:
Read the rest of this entry »

Posted in python. Tags: , . 3 Comments »

Percona Conference Slides

9 May 2009 — eric

For everyone who was not able to attend this year, here is the link to the Percona Perfmance Conference slides: http://conferences.percona.com/percona-performance-conference-2009/schedule.html.

I suggest you check them out if you work with MySQL. Enjoy.

Posted in Misc. Tags: . No Comments »

Counting Email Addresses By Domain in MySQL

1 Apr 2009 — eric

Every so often I find some statistical need that although Perl program would be easy to write for it, its probably something the database should just handle. So I have a column in an email management table that has just the email addresses in the format user@domain.tld.

I want to know which domains make up the majority of the users. (The numbers have been changed to protect the innocent):

mysql> SELECT SUBSTRING_INDEX(email, '@', -1) as Domain, count(*) as Total
      FROM email_list
GROUP BY Domain
ORDER BY Total DESC
       LIMIT 15;
+----------------+---------+
| Domain         | Total   |
+----------------+---------+
| yahoo.com      | 1304000 |
| hotmail.com    |  908400 |
| aol.com        |  800000 |
| msn.com        |  168000 |
| gmail.com      |  161000 |
| comcast.net    |  143000 |
| sbcglobal.net  |  110000 |
| bellsouth.net  |   62000 |
| cox.net        |   58000 |
| verizon.net    |   56000 |
| earthlink.net  |   52000 |
| charter.net    |   46000 |
| juno.com       |   30000 |
| optonline.net  |   22000 |
| netzero.com    |   17000 |
+----------------+---------+
Posted in MySQL. Tags: , . 10 Comments »
« Older posts «
Newer posts » »