Over the past few months I’ve been working on a data warehouse project that’s been at times both rewarding and exasperating. Today I thought we were nearing completion of the project when several data quality issues jumped up and presented themselves. As I worked through the issues I was struck by the notion that I am encountering business intelligence in its most nascent form.

Until recently a select few people have analyzed data. The data’s quirks and oddities were handled in the brains of those crunching numbers, and not by some formal business process. So with each data issue came an oh-by-the-way moment and new details would emerge. These missing details resulted from people not articulating knowledge they use intuitively.

Imagine you are lending a co-worker your car. You hand over the keys with instructions on where it’s parked. Your co-worker gets to the parking lot and is flummoxed because your car has a manual transmission. The co-worker doesn’t know how to drive a stick shift. You’ve never owned car that has an automatic transmission and the thought never occurred to ask if they knew how to use a clutch. And so it is with emerging business intelligence and data.

As painful as it is sometimes, we are formalizing and standardizing knowledge. This knowledge can now be passed on as personnel changes take place. Years down the road some poor soul is not going to look at a mound of data from 2008 and how wonder how it all fits together.