The Drawback to Web Frameworks

(imagine my best old man voice)

Pull up a chair sonny and let me get my old man cane out of the way so I can tell you what web development looked like back in my day.

Back when I was coming up you had to write the frameworks yourself! It was a rite of passage for every PHP developer out there and everybody thought their framework was better than everybody else's.  That hasn't really stopped either as PHP is saturated with frameworks these days. Most of them are modeled in one form or another, after Ruby on Rail's MVC approach too.

Before all of these different frameworks existed web developers tended to be SQL junkies writing crazy, page long queries that would put the pieces of their heavily normalized databases back together.  Data management logic would actually live in the database itself in the form of SQL functions, triggers and stored procedures.

Then frameworks came along.  You didn't have to "mess with" all of that ugly SQL anymore.

There in lies the problem and the project I just finished up is a perfect example of it.  I've never seen a project handled badly by somebody who understood databases.  The database(s) is the backbone of every major web application in existence.  

The problem with the advent of the framework era is that rather than taking advantage of all of those extremely powerful tools under the hood of their database, many developers avoid them like the plague. Simply because they haven't ever been forced to work directly with the database thanks to the framework hiding so much of it from them.

That's the issue in totality. Databases exist to manage data, not simply to be a dumping ground for you to save and retrieve things from. That's what a file system does.

Databases have tools built in to handle statistical analysis, complex mathematics, caching, triggered actions based on data changes, transactions, flexible full text searches, specialized data types that allow for processing and traversing XML, data compression and that's barely scratching the surface.

But if you were to ask most developers who have learned with frameworks, you'd be lucky if they could tell you the difference between a LEFT JOIN and an INNER JOIN.  It's maddening.  

I just finished spending a year cleaning up a mess left by some very good Ruby on Rails developers. The problem is that Ruby on Rails was where their talent began an ended. The mess left by these guys because they didn't have any idea what their database could do was so bad that it almost sank a company. There were pages that you could visit which would crash the site. In order to look up what the status was on a particular object related to a user, they used some beautiful looking Rails code to find, filter, and combine results (and THEN paginate). The problem is that after Rails goes one level deep from a single record it starts performing single queries for each record in each relationship. That meant, for example, that in order to retrieve the most recent objects for a user who had over 18,000 in his account history that upwards of 50,000 queries were executed.  The results of those 50,000 queries were then loaded into the web server's RAM (and SWAP), processed/sorted/filtered, and THEN paginated just to show the first 100 results.  It was appalling and that is only one example.

Data management logic belongs in the database in most cases.  There are exceptions of course but understanding how the data needs to be processed and the constraints involved will allow you to identify what those exceptions are going to be as well as how to handle them.  

Putting data management logic in your application code base is probably the #1 cause of race conditions as well. That unique check on your model's before save hook is not always going to work. The unique index on your database will.

Please, for the love of all that is good about the web; if you are a web developer who's come up on frameworks and don't feel like you could write complex SQL with your eyes closed...LEARN! Please! If you have questions, post them in a comment on this page and I'll be happy to take a look.

That database backing your system is a powerful thing.  Don't just ignore it, take advantage of it.  That's the biggest drawback to web frameworks: they teach developers to ignore it.