I wrote yesterday about democratisation of analytics and data. I believe getting information into the hands of business users is a "good thing"; accurate, timely information to influence the operational decisions they make each day. However, getting information "at any cost" is not a "good thing".
Quite often our users get their information from "data shadow" systems - groups of spreadsheets and local, customised databases. While these systems appear to provide exactly the information that business users are asking for, they are outside the purview of the IT group, and they often spawn data silos with the usual problems of inconsistency and quality.
Business groups build data shadow systems to answer the business questions that the enterprise applications, data warehouse or reports fail to answer for them. They're filling a gap in the services they receive from their IT departments. Users may not want to get the information this way, but they don't see any alternative. Worse still, as data shadow systems evolve over time, they encompass more and more information, and increasing numbers of business users come to depend on them.
Data shadow systems give business groups what they want, but most business users do not want to spend so much time creating these systems. Nor should they. They should be spending their time gaining a better understanding of their business, not wrestling with technology.
Because dealing with technology is not what business users do best, they cobble data shadow systems together without an overarching design. Each addition gets more difficult to implement and more costly to maintain. And when data management principles and disciplines aren't followed, data consistency and integrity suffer. Data shadow systems often fulfil their business's need, but they do so in a very costly manner that uses too many resources and sacrifices data quality.
My experience suggests that every company, large or small, has at least one data shadow system. Keeping the needs of business users in mind, it is possible to replace or rework these shadow systems with solutions that dovetail with a company's overall data warehousing architecture. Replacement doesn't need to be a huge effort, either. The best data warehouse projects deliver in a strictly incremental fashion, piece-by-piece.
Rebuilding data shadow systems is the right thing to do to ensure consistent, quality information for running a business.
Whilst many data shadows are created with Microsoft software, there are many created with SAS software too. Is your own system a data shadow? If so, you've got the best set of tools in your hands for rectifying the situation. Get to it!
