There are many reasons for which databases must be scaled. The majority of the time it must be scaled to accommodate for performance issues as the product grows. Though NoSQL is making a lot of noise these days, it is to no one’s surprise that SQL is still extremely popular. In general the same principles are followed while scaling out any SQL product, be it MySQL, MsSQL, Oracle or even DB2. Scaling is often done to overcome performance issues as the product grows. However, dealing with big data scaling is often done to balance the data across multiple hardware nodes or clusters.
Sometimes I like to think of the past as a good example of how much I have learned. Scaling is a particular domain that I have learned a lot over the past year. 2 years ago, I had always assumed that scaling merely consisted of upgrading servers. The concept of putting servers together to work side by side was baffling to me.
I do like to poke some fun at my past, only because today I am proud of my accomplishments, proud to say I have scaled a product that can easily handle hundreds of thousands / millions of users with a very small pool of servers.
However, the point of this post is not so much to poke fun at myself, but to teach some of the places I have made mistakes and how to get around them. Of course, without giving some of my secrets or the key to making extremely responsive APIs, like igapi.