6 Problems Database DevOps Helps to Solve
In a recent article on DevOps.com, Helen Beal discusses six common sources of conflict within IT that DevOps can help to solve (see article here). These sources of conflict have arisen as IT tries to form new behaviors and processes to meet the needs of the business in delivering faster. These growing pains are natural, but according to Beal they can be reduced significantly through embracing database DevOps practices.
Problem #1: Unplanned Infrastructure Requests
“I need a server… now”
Operations teams have a full plate with work that has been planned and prepared for. When development requests new environments, it adds unplanned work to the mix and catches operations off guard. Development sees this as willful neglect in trying to meet the needs of the business to go faster, which just isn’t the case. Remember that in addition to this type of unplanned work, operations also deals with “unexpected system failures and defects.”
Beal’s biggest piece of DevOps advice here is to invite operations to collaborate sooner in the dev cycle. This way, operations has a better understanding of the work that’s coming down the pipeline and can plan it into their workflows.
The other area where DevOps can reduce the strain is through automation. This makes it easier to deliver new infrastructure by automating it, creating or using cloud and virtualization infrastructure.
Problem #2 – Excessive Access to Production Elements
“I want access to the production systems”
This happens when a development organization is frustrated by what they see as a lack of responsiveness from operations. While it seems like a quick fix, it increases the risk to production systems as more access points are introduced. Increased risk in production leads to more unplanned work for operations. More work means decreasing operations’ ability to support development.
“DevOps doesn’t mean throwing all process out of the window and putting live environments at risk,” writes Beal. “It’s by no means mutually exclusive with methodologies like ITIL but it is about individuals collaborating better to achieve a shared desired outcome.” To foster that collaboration, Beal recommends that operations involve development in “the definition of your change control processes and set up of any systems.” Work to establish some SLAs together as a team.
<Check out the The Business Case for DevOps to learn more about the benefits for DevOps.>
Problem #3 – Lack of Retrospective Failure Analysis
“Something is wrong… whose fault is it?”DevOps, like Agile, is about removing blame and focusing on developing solutions as a team. “That means no finger-pointing and no swearing at each other,” writes Beal. Two areas where DevOps can help is in:
1. Setting up and running blameless retrospectives
2. Figure out ways through tooling to either “pre-empt failure or fail smartly.”
Problem #4 – Too Many DevOps Monitoring Tools
“So many alerts they are meaningless”
DevOps advocates for establishing feedback loops from production to as far back as development. But, in working with clients, Beal has found that this critical feedback isn’t missing due to a lack of available database DevOps tools.
“One of the challenges we often find,” writes Beal, “is that enterprises already have monitoring systems. Lots of them. Monitoring lots of different things. All producing a multitude of alerts.”
The problem, according to Beal, is that this plethora of monitoring leads to “alert fatigue.” There is data, but there isn’t the kind of feedback necessary to make improvements, or that critical feedback is getting lost in a sea of information.
Beal’s advice in this regard is to combine and rationalize your existing systems. “Take some time to really understand what alerts matter and adjust your policies accordingly,” she writes. “Consider this an ongoing task to tweak your systems as your situation changes.”
Problem #5 – Not Enough DevOps Training Time
“I don’t have time to save time”
This one is the silent killer. The teams which need to use DevOps the most don’t have the time to think about how they can do things better, much less stop to catch their breath. The increase in expectations to deliver innovation, coupled with “the fragility of today’s evolution of IT infrastructure,” result in an increased workload in IT, and a great deal of unplanned work.
These 2 factors are often business drivers behind the adoption of widespread DevOps metrics for the database. According to Beal, “DevOps is designed to streamline the throughput of work at a cultural, interaction and application-based level.” This can help to diffuse “the pressure building up in many IT departments today.”
Her advice on this front is to bring in executive sponsorship early, to help force a change for the better. Two things executives can do to help is:
Launch a DevOps pilot somewhere in the organization to jumpstart change.
Carve out some room in the budget to bring in an external consultancy to look at your “DevOps state.” It’s likely that the consultants won’t tell you anything you don’t already know. It still helps in starting to build a business case to have outside observation and documentation.
Problem #6 – The DBA is a Bottleneck or Roadblock to DevOps
“Our hero is a bottleneck…”
Sometimes, one person becomes the bottleneck in trying to deliver faster, but not for the reasons that you might assume.
“What tends to happen is this: one day an engineer decides that building servers is a tedious, manual process and that they could write a quick script to make that bit a bit quicker. The next day, they write another one. And another one.”
This behavior is a good thing. It’s an individual taking initiative to improve workflow by automating different aspects of the process. But, according to Beal, “The problem is that even though the script monster works pretty well most of the time, when it fails it’s virtually impossible for anyone other than its creator to troubleshoot.” This, as you can probably assume, doesn’t scale well. It presents a skewed picture of the level of automation within the organization. It also introduces an individual as the single point of failure for the whole system.
These heroic individuals began automating tasks to keep their own sanity. Instead of making it a team problem, or a part of the process, these individuals take it on themselves to solve the problem, probably to avoid bureaucracy. If the culture in the organization is more responsive, or open to exploring new techniques and methods, it might offer these individuals other alternatives to explore besides fixing it on their own.
Another option is to remove these talented individuals from the daily process. You’ve recognized that these are some of the most talented individuals in the organization. Removing them from the daily work can focus their talent and innovation on optimizing the whole process or developing new ones. Their general approach can help to elevate the entire system.
Check out the The Business Case for DevOps to learn more about the benefits for DevOps.