A Hierarchy of Process

by David Stiennon

Programmers hate corporate process.

We are constantly fighting to simplify it. But in the course of fighting it, we are tempted to lose sight of how excessive process is not fundamentally a technical problem, but a symptom of lack of trust within the organization.

I divide corporate processes into a few broad types

The Manual Process

This is the first thing you think of when you think "Corporate process". It is the bread and butter of any bureacracy. When you want to do something, it involves submitting some kind of form, and taking it through a list of prescribed steps. Do you want to get reimbursed for your business travel? Submit an expense report. Do you want to get paid for the week? Submit a timesheet to HR. Do you need to pay your taxes? The IRS has many, many forms to help with that.

The Automatic Process

If the "expense report" process above involves filling out a form by hand and faxing it, it could be much improved by emailing a Word document. That Word document could be replaced by a PDF template. The process could be further improved by replacing the pdf template with a form page on the company intranet. Then, whoever is responsible for reviewing expenses could approve it with a single click. There might be other automations that are possible: Some ability to automatically fill in the online form by scanning your credit card statement, or by taking a cell-phone photo or your receipts.

We developers love to do this sort of thing. For many of us, making these fusty bureaucratic processes more efficient is our calling in life. The goal is to eliminate any hand-copying of information from one place to another. Rather, it ought to be entered only once, and from there a computer can route it where it needs to go.

But, sometimes I fear that us developer are over-eager to attack these processes. Moving from a pencil-and-fax form to a pdf form might do away with 2/3 of the time spent on the process, and only take an afternoon to set up. If it was a half-hour process before, it would now only take about ten minutes. But getting a Web Form that automatically enters the expense report into the database might take a week of programmer time, and only shave the time spent on the process from ten minutes down to five. Getting that receipt scanner working could take months of programmer time, and only move the process from five minutes down to three. In other words, the return on investment from further automating the process falls off at an exponential rate.

Writing that receipt scanner could still be useful if there are 20000 people in the company who frequently submit expense reports, or if this "expense entry software" is a product you're developing to be sold around the world. But its simply not going to be worthwhile for smaller operations.

My even larger concern, though, is that focusing on improving manual processes is sometimes barking up the wrong tree. As boring and frustrating as they are, they are only scraping the surface of how inefficient corporate processes can become.

The Permissions Process

I'm a new employee starting at the company, and I ask around for how to submit an expense report. "Oh," I hear, "Fred's in charge of that sort of thing. His cubicle is up on the 4th floor. Here, I'll give you his email" This is the case where there is no mechanical process. Rather, there is a guy who is in charge of approving and dispensing travel reimbursements.

In some organizations, this can be alright. Sometimes Fred is a really helpful guy. He knows the system inside-out, and will even advocate for me, suggesting reimbursement for things which I hadn't even thought of, and waiving requirements for receipts I forgot to save. For something trivial like expense reports, this is still not very efficient, though it might be just fine if its a small company with only a couple people submitting expense reports in any given month. What took a half-hour when conveyed through a pencil-and-fax form is now closer to a round hour when you add up the time me and Fred spend on it. It's drastically worse if helpful-Fred is replaced by two-bit-tyrant-Fred, who rules over his tiny fiefdom of expense reports with an iron fist, who loves issuing opaque denials and haggling over trivial amounts of money in order to shew forth his power.

The Process by Committee

But my grousing over tyrant-Fred aside, sometimes this permissions process really is the best way to go. It's overkill in the case of expense reports, but there are other processes where in the course of them, some significant judgement call needs to be made. It might be something more along the lines of, "Our engineers have identified this list of bugs in our software, and we need to make a decision on which ones need to be fixed immediately, and which ones can be postponed" In these cases, having one person like Fred whose expertise and judgement is widely trusted, and giving him authority to make those calls, might be exactly the way to handle things.

But suppose there isn't one guy who is trusted to make the final call. Maybe there used to be, but there was one time, long ago, when Fred made a mistake. Fred's mistake caused some sort of embarrassment to the folks higher up the chain of command, and they decided that it couldn't be allowed to happen again. Maybe Fred left the company, and no one person could replace him. Whatever the cause, there is no longer a single individual who is entrusted with making the judgement calls.

If it's an expense report, there's no longer one "Fred" who I can take it to for approval. Rather, I'll be told, "Bring it up during the next Expense Request Approval Meeting."

This is the true ninth circle of corporate process.

A few common features of committee processes:

  • There will be a meeting scheduled on a daily or weekly basis, whose purpose is to move a routine process forward
  • In between meetings, committee members volley issues back and forth via long email chains
  • Anybody in the committee can say "no", but it takes everybody to say "yes"
  • If you observe the committee, you have a nagging feeling that the purpose of the whole thing is to dilute responsibility. It's less to prevent things going wrong, and more to make sure that when something does go wrong, no one person catches the blame.
  • Whenever something goes wrong, the committee will add more steps to the process, to "ensure that it never happens again". This leads to greater complexity; to items such as a meta-process to manage the agenda for the daily committee meeting.

Due to that last point, the size of a committee process grows with time. Do you ever wonder how some bureacracies seem to have their expenses increase exponentially over time, but without improving the quality of their services? I credit Scott Alexander with giving it the name, "Cost Disease". It's a major social problem we have in sectors such as healthcare and higher education. I suspect one of the main causes of Cost Disease is that many, many things which should happen automatically are done by a system of individual permissions, and many, many things which ought to be decided by individual permission are decided by committee. I notice such a strong correlation between regulated industries and cost disease that I can't help but suspect that there's something about regulation that encourages organizations to seek safety from individual responsibility by burying it in committee-type processes

The Mu process

At the other end of the scale, there is a transcendent moment of enlightenment, where the process disappears. In the running example of expense reports, there would be no expense reports. If I travel, my boss just hands me a company credit card, with his full trust that I will use it responsibly. There is a pattern I notice here. The transcendent non-process can only occur in an organization which places high levels of trust in ordinary employees. By the same line of reasoning, excessive process is a sign that the managment of an organization does not trust employees to make reasonable judgements.