Business and Empiricism

This might ruffle some feathers, so please don’t take this personally.

I love reading. So I’ve read a lot of business management books, business magazine articles (many specific to the building industry). Furthermore, I receive quite a few emails offering seminars, webinars, and magazine subscriptions all offering to help me improve my business.

It’s taken me years to notice that almost all of this advice is premised upon deeply flawed data. All of the popular business/finance books on my shelf – ‘Good to Great’, ‘Built to Last’, ‘The E-Myth Revisited’, ‘Raising the Bar’, ‘The Millionaire Next Door’, ‘It’s Called Work for a Reason’ – all of these and many more build their cases on pseudo-scientific methods. This is not to say that they are specifically wrong. Just that their methods are so deeply flawed and pseudo-scientific that it’s difficult to tell if acting upon their advice (insofar as they’re specific enough to generate an actionable premise) will be beneficial or harmful to one’s business. In other words, these books tell stories, some of which might be true.

I won’t get into details about all of the logical mistakes of the authors of the books above – the idea isn’t original and a lot of folks have discussed it in depth (Taleb, Shiller, Pfeffer, Sutton). But here are some of my favorite examples:

– None use proper controls (although some use comparison companies, this is largely to illustrate the thesis of the book). A high percentage of millionaires driving domestic or non-luxury cars sounds compelling until we adjust for the fact that high-net worth individuals are generally over the age of 55.

– Many are retrospective and based upon the memories of individuals in companies (hindsight bias). Imagine if the FDA approved drugs on the basis of enough individuals saying they remembered being healed by a certain drug? Sounds like too fine a point? Memory is notoriously unreliable. This has been confirmed in controlled studies of business managers, who consistently overrate the value of their input (see Pfeffer and Sutton’s ‘Hard Facts…’.)

– Many are persistently nonsensical and avoid saying anything definitive (despite having raced 6 ironmans I can’t see how ‘rinsing cottage cheese’ is a useful analogy). Also note that I wouldn’t put ‘Who Moved my Cheese’ on the list above because it’s not even pseudo-scientific – it’s really just a rambling false analogy.

– Even if many of the books correctly distinguished characteristics of successful individuals or businesses (and they haven’t) the traits may be only correlated with success and not causal. Perhaps people who are motivated enough to write business plans have some other trait that makes their businesses successful. Knowing that successful people have business plans isn’t helpful for people who want to know if writing a business plan will help cause their success.

– Few make allowances for the possibility of luck being a strong factor in success.

– Few acknowledge the strength and dynamism of competition. If you want to be successful in business you can’t just do things well. You actually have to do things better than your competition. (See Rosenzweig’s book, ‘The Halo Effect’ for more on this.)

– They all fail to acknowledge the possibility that reading success stories might have no effect on one’s own prospect for success. (Admittedly, you probably wouldn’t read a business book called ‘Why Reading This Book Won’t Help Your Business.’) I know I should return phone calls promptly, yet sometimes I don’t. My problem isn’t that I failed to read somewhere that I should return calls.

And the problem isn’t just with these books. I agree that, on average, construction management consultants/writers seem better than those in other industries (Industry specific books can be quite good – Gerstel & Faller are well worth reading, but definitely leave on your critical thinking cap). But the lack of empiricism is still pretty pervasive. For years I believed I could subjectively pre-qualify and identify good clients based upon subjective factors. It wasn’t until I began quantifying my results (tracking the hits and misses) that it became clear how wrong I had been. But none of the management gurus I’ve read entertain the possibility that subjective pre-qualifying might not work, let alone attempt to study it in a controlled fashion. Ditto subjective job interview questions. Ditto the myriad of approaches to advertising. Ditto Pareto or 80/20 principle.

OK, so I’ve gone on longer than I intended and you’re starting to wonder what my point is. The books aren’t empirical, but they still offer valuable lessons and case studies, right? What’s the harm in following their principles? I agree that to a large extent the themes of the business genre are sound. Hard work, frugality, focus, and perseverance are all good characteristics, and they will probably make one more likely to succeed. But these are also trivialities that most of us figure out in our first few months (or, in my case, years) of business. Reading business advice can be generally harmful in that it takes precious resources that could be better used on other projects or ideas (if, like me, you suffered through ‘The Sisterhood of the Traveling Pants’ you know exactly what I’m talking about – neither of us will ever get that time back.)

But following pseudo-scientific advice can also be specifically harmful. One example, the Pareto or 80/20 principle, is as ubiquitous as it is fallacious. The theory goes that 20% of causes are responsible for 80% effects. So in business 20% of clients are responsible for 80% of profits. Conversely, another 20% of clients are responsible for 80% of headaches. So the solution – ditch the problematic 20% and find more of the profitable 20%. Sounds easy, right? Well not really. One problem is that the ‘principle’ is not actually true. Specialized businesses (such as roofing, siding) will probably have a much broader base of clients who account for most profits (the 70/80 principle?). But even if the principle were true, the biggest problem is that nothing logically flows from the principle. Predicting profitable clients or jobs is exceedingly difficult. Being prolific might be the safest way to acquire profitable work – and this exposes us to the possibility that we’ll take on some bad jobs along the way. Firing bad clients might prevent us from acquiring good clients in the future.

Another (perhaps fuzzier) example is when simplistic advice becomes entrenched in the industry. A recurring theme is that contractors need to “know their business,” “think like a businessman,” and “get out of the field and focus on business.” But how do we know this is required to be successful? Probably because the people saying it have found that successful contractors are mainly businessmen (again survivor bias). But this doesn’t mean that ‘learning business’ will lead to individual success. What about contractors who lack the interest or aptitude, or those who enjoy carpentry (or plumbing – ha!) too much to give it up? This belief has probably hindered the evolution of an entirely different business model. What about a model in which highly paid builders focus on technical issues and outsource the largely generic and boring work of running a business. If this sounds crazy look no further then the practice of medicine. Here the model is embraced by many of the most successful practices. It takes a lot of energy to build (or doctor) well. Is it possible that focusing on our single passion will yield better results than diverting attention to something many of us find considerably less gratifying?

I don’t entirely blame ‘experts’ for these lapses. I see two parts to our (my) propensity to embrace bad business logic. The first is that most of us come from backgrounds that are somewhat empirical. Our craft is shaped by learning proper building techniques, and these techniques are at least somewhat grounded in science. So when we go from being carpenters (or, heaven help us, plumbers – a trade with only three empirical rules) to general contractors we look for the same prescriptive advice we received when learning proper building techniques. We knew leaky windows were bad, so we used details that prevented window leaks. But in management, the model doesn’t exist – the multivariate field of business doesn’t lend itself to empirical study in the way buildings do.

The second is our natural tendency to have biases and enjoy narratives. For instance, on a broad level, we know that our industry is cyclical. Yet we’re almost incapable of applying this knowledge to ourselves. We’re far more likely to embrace a new strategy when business is unusually slow (hire a consultant, advertise, revamp website, new signs, change business direction, etc.). As business picks up, we take the cyclical up tick as evidence of our new strategy’s effectiveness. Yet the situation may have improved had we done nothing (regression towards the mean). Business gurus famously exploit this weakness in our cognition by holding their expertise accountable for these cyclical improvements. Perhaps doing nothing (or something else) would have yielded the same (or better) results. After choosing to implement an idea, the problem is compounded by our tendency to seek out evidence of it’s success and disregard evidence of it’s failure (confirmation bias).

My long-winded point? I now know far less about business than I thought I knew 10 years ago.

Thanks for reading,
Jesse