A good piece in Government Technology offers ten tips for how cities can exploit big data. “Use Standards” is listed at #5. No prizes for guessing I'd put it at #1. Also, this article perpetuates the idea standards primarily have an “external” use. I see this idea around a lot. It's not stated explicitly, but it accounts for some of the disordered priorities I often see in capsule advice about data management.
So: “Use Standards” is at #5. Ahead of that at #3, we get “Integrate and Unify Internal Data”. The gist of this tip is that you do data analysis on your existing systems. This is weird because one of the points made about using standards in tip #5 is: “Applying standards to the data you store will simplify data management and provide new opportunities to analyze and benchmark your data across disparate systems.”
I think what's going on here is an understandable attempt to break the data management problem down into digestible chunks. It seems sensible to call for getting your own house in order before looking to “external” issues like data standards – and, in this context, “gaining the ability to benchmark your data with other agencies”.
The error here is to believe there's still a hard and fast boundary between internal and external. But despite agencies and departments having separate names, staffs, buildings, budgets, and so on, the boundaries between them are porous to information. From the point of view of data, there are no natural parking places – only the artificial dam sites we know as silos.
And from the pure pragmatic perspective, the very best way to bring order and consistency to your internal data is to apply the relevant data standard. Whoever you are, whatever you do, there's a pre-existing data standard that covers at least 90 per cent of your data. Believe it. The Government Technology article points to Code for America's useful list of standards in the public sector. If you're in commerce or industry, your trade association will be able to direct you. GT