Gregory Maciag: Down with Dirty Data

Tuesday, 08 March 2016

Down with Dirty Data

Gotta love the 80/20 rule. It's a universal phenomenon that applies in every aspect of data management and exploitation. Here's an instance from a data scientist working in housing: “Anecdotally, a data scientist will spend 80% of his or her time cleaning a data set.” This is tragic. The valuable time and skills of motivated professionals are being squandered on dumb, repetitive tasks that could be removed at a stroke – with industry data standards.

Whenever we see people in this kind of data mess, we don't need to look far for either the cause, or the solution. Every community that suffers needless data complexity is generating its own problem, which it can cure with data standards. Let's stop wasting people lives with thoughtless data creation habits. This is a business hygiene issue. Down with dirty data! HACT

Posted on Tuesday, 08 March 2016 | Permalink

Comments

Verify your Comment

Previewing your Comment

Posted by: |

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:

Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name is required. Email address will not be displayed with the comment.)

Name is required to post a comment

Please enter a valid email address

Invalid URL

New Book

My latest book presents the challenges members face when adopting industry standards as well as the opportunities that come as a result. It features my discussions with many people over many years and follows the foundation I set in my first book "The Business Information Revolution".

Industry standards are never adopted in a vacuum. They become part and parcel of all the trials and tribulations managers face in their day to day work. ACORD Standards are always part of a larger software development project that brings along people, priorities and politics. Adopting industry standards isn't simple, but the benefits far outweigh the problems of building and maintaining proprietary alternatives.

I trust that you will appreciate my frankness, identify with some of the challenges and learn from what others have done to pave the way.