Getting more value from your data quality tools

Why haven't you fixed that data yet?

I'm quite used to a certain type of conversation with data managers.

We're so unsophisticated.

We do everything in Excel.

I wish we could do things properly but no one will give us the budget.

I hear that less often now.

New lines of conversation have arisen instead.

We need to improve our data quality. We've got this tool which flags loads of errors, but we don't have any bandwidth to fix them.

Meanwhile, the data technology vendors tell me

Company X have our tool, but they're barely using it!

The inference is that, when it comes to renewal, no client will extend an agreement to licence a tool which isn't useful.

Let's look at our client’s challenge.

We need to improve our data quality. We've got this tool which flags loads of errors, but we don't have any bandwidth to fix them.

What's going on here?

We've got the Data Quality tool in place (or even a set of SQL rules run over their databases).

The rules came from the vendor's "rules library", or maybe someone in the business created additional rules as and when they found data problems.

Whilst I would always recommend tackling these data quality issues with a root cause analysis, we aren't even that far yet. We simply have no one available to go through the issues raised to validate them and decide if they are worth fixing, or even if the correct data can be obtained.

Tell me if this resonates with you:

• If I have one or two items on my personal "to do" list, I'll do them.

• If I have five or six items on that list, I'll probably get two or three done.

• If I have twenty items on the list, I might get one or two done. Or I might take a look at the long list, take some "me time" and do none of them.

Are you the same?

Back to our example - we have lots of data to fix and we don't even know if we should be.

Meanwhile, there's a DQ report being viewed by the leadership team, who see that the Data Team oversee a load of bad data.

That is not good for the career progression of the data team.

My view is that progress is being held back by the lack of a data governance framework. Simply put:

• We want to be focusing on the most significant pain points or opportunities (let's call them activities) for the organisation; which requires that…

• We identify the data and data quality requirements to address those activities; which requires that…

• We measure our data availability and data quality to see how we compare to those requirements.

So instead of leadership being told that there are 10,000 insurance policies with an invalid ID; we understand that there are 20 marine policies where the premium field is blank.

Instead of the retailer having 27,000 invalid SKUs; they focus on the fifty of them which generate the most revenue.

If that goes well, they can do the next fifty - or even more.

These are solvable problems.

It might not be the data team who solve them. More likely it will be the marine underwriter who understands why the premium field is sometimes left empty; or the category lead who is responsible for SKU allocation.

They'll be tweaking their processes accordingly.

And then…

• The data tooling has served its purpose;

• The data team have demonstrated their value;

• The budget holders can demonstrate RoI on their data investments; and

• The vendor retains a satisfied customer.

I've used three simple bullet points above to summarise what should be done, but I'm not saying that any of them are easy to do.

Especially when everyone is busy with a day job.

At Datazed, we can do this for you; or we can work with your team to show them how do it. Let's talk.

Was this message forwarded to you, or are you reading it online. Click here to subscribe and get this newsletter sent directly to you. Unsubscribe instantly and any time using the link in the footer.

Have a wonderful week,
Charles