Two major themes of our times, big data and open data, rely on the successful agreement and usage of data standards. The benefits of working with masses of data, and with massively available data, are absolutely predicated on clear and robust schemes for understanding, collating, and sharing data in meaningful ways.
There's lots going on in the open data arena, partly because opening up data can generate new types of services, and partly because the need to standardize data ahead of opening it up makes for significant cost savings in public sector organizations.
People building and advocating data standards for open data can learn a lot from the experiences of standards groups in commerce and industry. The ACORD community has much to teach – not necessarily about individual modeling decisions, but about the process of developing, promoting, and implementing standards. We've evolved and we know what doesn't work as well as what does.
Some of the advice I'd give sounds a little paradoxical. For example, I'd say you need every stakeholder involved in the creation of a viable standard. But I'd also say the user community can't wait around for the outliers to have their say. At some point, the bus has to leave.
Similarly, I'd say that while you want to get the best possible structure and nomenclature for every item in your standard, you also have to be prepared to go with something that's 90 per cent there. Standards work attracts perfectionists – and thank goodness it does. But we also have to trade perfection against timeliness. There will always be a chance to improve a standard. There may not be a second chance for an industry to enter a new market in strength.
Comments