Database Administration as a Service

A DBA should provide two things, a service and leadership. For Grant Fritchey, it was whilst serving a role in the Scouts of America that he had his epiphany. Creative chaos and energy, if tactfully harnessed and directed, led to effective ways to perform team-based tasks. Then he wondered why these skills couldn't be applied to the workplace. Are we DBAs doing it wrong in the way we interact with our co-workers?

Database Administrators are, by our nature and the requirements of our job, control-oriented. We are like commissars in that we must maintain and enforce security mechanisms on the data from a business and regulatory standpoint. We prepare for disaster, outages and deletions of the information that defines and supports the businesses that pay us. We get up at 2:30AM on a Sunday to put out figurative, or even literal, fires. All of this reinforces that controlling nature and leads to us to become, for want of a better term, jerks. This is a problem for the DBA profession. We need to stop. I think that we, as DBAs, can be better by embracing the ideas of “leadership through service”.

Why DBAs Become Control Freaks

As DBAs, we want to have as clean a production environment as possible. We want to know, as far as we can, exactly what is going into it, and when, so that we can manage backups, disaster recovery, downtime and all the rest. We want these things, in fact, we’re required to provide these things on production, so when we look over our shoulder at what is coming towards us from development, we instinctively cringe. This is the moment where the control over development starts, in order to tame the beast, and it’s at this moment that we’ve gone wrong.

Our responsibilities as DBAs require us to set some direction on how to manage information within our databases and the organization. The organization has tasked us with those responsibilities, but they didn’t tell us to become little martinet’s, jealously guarding our territory, privileges and access. They didn’t require us to force developers to become supplicants who must kowtow in order to get the database they need to start developing. No. The business expects us to provide a service, and deliver it like any other well-run service.

Some time ago, I called up my cable company to arrange a new installation. They gave me a day, two weeks out, and a time, sometime between the hours of 7AM and 1PM (there goes half a day). I waited on the appointed morning, only for them to show up at 4PM (the rest of the day wasted), only, oops, they didn’t have all the parts they needed for they install, so said they’d need to come back the next day…sometime between the hours of 7AM and 1PM.

Great service, right? No. Well, if you make the development team jump through hoops in order to get a new database into development, implement continuous integration, try out an Object Relational Mapping tool, or experiment with a new development paradigm, then you’re making them feel like they’re working with the cable company. As soon as a viable alternative to cable presented itself, I jumped at it. When your developers find a viable alternative to working through you, even if it’s not actually viable, what do you think will happen?

Instead of barking at the developers the next time they come up with a hair-brained scheme to only ever do inserts and deletes in order to avoid waits caused by updates (and yeah, I had this discussion with a development team), we need to try working with them, and we need to start working towards leadership through service.

Leadership through Service

Leadership skills aren’t part of the average DBA’s training. My first insight of the way to learn, and teach, these skills came from my long-time role in Scouting. The Scouts of America do a great job of turning out leaders. You can do your own searches on the internet to find the number of presidents or astronauts or other business leaders that were either Eagle Scouts, or were just involved with the Scouts for some period. Scouting teaches, among many other things, the concept of leadership through service.

A campout with the Scouts looks chaotic, with people running in circles shouting, gathering sticks for fires, some tents getting set up, and so on. You soon start to realize, however, that despite the apparent pandemonium, there is progress, and organization. Look harder and you start to see how it’s happening. On the side, experienced scouts are talking new scouts, scout masters are talking to scouts, pointing out what has to happen, suggesting how something should get done, stepping in to do things where absolutely necessary, teaching and then walking away, where possible. It’s leadership through service.

The concept is not to have someone standing tall and barking out commands. Believe me, it would be so much easier for me to walk out into a campsite, point at each scout, assign a task, and ensure each one does it exactly as I would do it, but then they’re not doing it themselves, and more importantly, they’re not learning to lead.

No, instead, the unit leaders, scouts and adults, are quietly getting things done that need to get done in order to support the young men and women (look up Venturing, women are in Scouting too) in their efforts to become better people and the leaders of tomorrow.

That chaotic period in the Scouts is creative chaos, it is group thought. You’d kill it stone dead by clumsy intervention. Instead, the leaders harness the creative energy gradually, and use their influence to give it focus and purpose. The same is true of our professional lives. Development should be chaotic. Production shouldn’t be. I like to describe it by analogy. Production is an old Eastern city, like Boston. It’s sedate, stodgy, boring, but safe and relatively controlled. Development is a cow town in the Old West on a Saturday night after payday at the end of a long cattle drive. It’s nothing short of absolute insanity. Each of these environments is exactly as it should be.

Our job is to take what Dodge City produces and gentrify it to the point that we can let it loose in Boston without causing mass chaos. The very best way we can do this is not to try to make Dodge City into Boston. It won’t work. Instead, we need to reach out and offer ourselves as a service, even as servants, to make Dodge City work as much like Dodge City as we possibly can. Our service should be to enable the developers to be developers, to develop.

Leadership through service teaches us that we can best set direction if we best serve, best help, best provide an environment through which people can get what they need, but do it within a paradigm that also moves in a good direction. In short, DBAs can act as a roadblock or as a pathway. I’m here to tell you that acting as a pathway will absolutely give you more control over what goes into your production system than acting as a roadblock.

Solutions

Here are my suggestions for how to get this done:

  • Define a process
  • Work backwards from production
  • Integrate tightly with development team
  • Automate as much as possible

You’ll need to have a process to get code from development to production, but you don’t want to control development. You just want to control production. So, work backwards from there. Figure out what you need in order to establish a safe production environment that will support the business, then layout a pathway and a process that enables the developers to get there. Provide this as a service for them.

This means talking to them, getting to know their development paradigm and supporting it. Be a service. Be a servant. Never be a roadblock. Help them get things done, but always keeping in sight your end goal of a safe production landing. To make this work, automate everything you can. The more manual your processes, the more pain you’re going to feel.

A word of caution, though, and a bit of a brush back for any developers who are rubbing their hands thinking, “Oh boy, the DBA has to do anything I say.” Nothing I’ve suggested here should changes the fact that the DBA’s job is to keep the production environment online and secure. Yes, we want to provide a service, but we need to provide leadership as well. Combining the two is the important point. So when developers have that crazy idea, support it, but guide it. Find out why they want to do it. Is it a good idea? Do you have viable alternatives? Is there a better way? Working with them and help guide their efforts, where necessary, without setting the absolute direction, or doggedly imposing your own ideas.

I understand that some organizations just won’t be able to adapt this approach. Organizations where, for example, developers have been ill-treated for so long that they have completely bypassed the DBA team in every regard; IT departments where management won’t support tighter coupling between DBAs and developers; places where DBAs just can’t surrender any level of control. Just because this is difficult doesn’t mean it’s wrong. In fact, as with many things, the more difficult approach is the right approach. Doing things the way we’ve always done them may be easy, but it doesn’t mean it’s right.

Tools

To make this happen, you’re going to have to bring some different tools to the job. No, I don’t mean PowerShell, and this isn’t a long softball sales pitch either. I’m talking internal tools, those soft skills of the world of work, namely people skills, leadership skills, communication.

After years of been treated like mushrooms, kept in the dark and fed, well, you know what, DBAs let that aspect of their workplace skills slide. Funnily enough, this isolation was often a positive choice by the DBA. I know that I was frequently happiest when I was invisible. I remember reading an article, a long time ago, suggesting that the best DBA was the one that no one in the company knew because everything “just worked”. Those days are gone. In fact, those days are part of the problem.

As DBAs, we need to engage. We need to talk to the business in order to understand their needs, so we know what services we need to provide. We can’t even set up a simple backup scheme without first understanding the recovery time objectives and recovery point objectives. We can’t even meet those requirements without communicating to the business some level of the technology that might be required to meet those times, if they’re seriously difficult. That’s a heck of a lot of talking to the business.

We need to talk to the developers for the same reasons. We need to understand their development paradigm, negotiate deployment windows and all the rest. Then, of course, we have system administrators and SAN administrators. All these people need to know who you are and what you look like. Communication is as much part of your job as knowing the difference between a clustered index and a non-clustered index.

One of the great rewards of being a scout leader, for me, is to see someone who lacks natural communication skills suddenly “get it”, and start to find a way to influence the decision process. In the same way, DBAs can learn the subtle arts of group work within an organization, as long as we get out of the cubicle and actually communicate with people, in a clear and useful way. Laughing at a database design and calling it stupid does not help make it better (trust me on this; it’s one of my big mistakes). We also have to write up requirements documents, process flow-control documents, emails, again, with clarity.

If you’re one of those DBAs who has grown to like being in a dank cubicle in the basement, this is not going to be easy. Your skills at influencing others in the workplace may be rusty, but the one constant in our universe is change, so you may as well embrace it and manage it or it’s going to manage you. Just remember that these changes are going to be iterative. You’re going to have to adjust as you gain understanding of what your organization needs.

Call to Action

As a DBA, you’re providing two things, a service and leadership. Start working on your soft skills. Provide a good service. Be a good servant. The leadership will flow naturally from this as you demonstrate your interest in being a pathway, not a roadblock.

Now is the time. Get up out of your cube. Walk down to the development team. Immediately say yes to something they need, get it set up for them, and start to build that pathway. Then you can come back and tell me what an idiot I am, in the comments.