By David Loudon, Founder and Managing Director of DTL Creative

What on earth is he talking about now? Fear not: I haven’t lost it (yet).

You may or may not have heard about Data Swamps. Yes, they’re a thing.

We all know that more and more data is being collected by companies. The need to collect increasing amounts of data and, of course, store it, risks creating data swamps. The word swamp came about as a way of describing the out of control, somewhat messy version of what was intended to be a data lake. I generally use the term for any scenario where the overall data repositories are messy, somewhat disorganised.

A data swamp is a horrible place — unlike a data lake which is a beautiful place that allows companies to retrieve and use their data efficiently and effectively. The uglier, smelly, dangerous even, data swamps can make both those goals very difficult and perhaps impossible.

They’re a sign of mishandling data for sometimes years. They’re a sign of poor data management and strategies. However, all is not lost. There are a number of strategies you can employ to ensure you don’t sink into the murky depths of a data swamp, which, let’s face it, a number of those reading this will have one foot in already.

Data management strategy and governance

Ask yourself this: do we have an actual over-arching data strategy? In response to this we have heard the following from landlords. Surely, we don’t need one if our systems are working ok? Surely, that’s for really large companies? Surely, all we need to do is follow GDPR? All wrong, and indeed dangerous.

Excellent data governance is what equips your organisation to maintain a high level of data quality throughout the entire data lifecycle, from creation to destruction.

Data governance defines how to work with data; who should access it, handle it; how long you retain the information for; deciding where the data is stored; and so much more.

It simply isn’t enough to assume that these clever systems you have spent a lot of money on will do it for you. Sure, setting up roles and permissions in a system, in your active directory, and perhaps even making these the same thing isn’t enough. You need a strategy and indeed associated policies. You also need a strong governance model. They all go hand-in-hand so don’t scrimp. Get started, or get improving now, before it’s too late.

Set yourself some homework

You don’t get results from sitting back and doing nothing. A good strategy and policies ensure you have regular homework to do. If homework is a term that fills you with fear from your school days, then it’s easy; think of it as admin or maintenance. Maintenance is very apt as we all know that working parts need maintenance for them to continue to work at their best.

Data is no different. It can very much be seen as a moving part in the operations of your company.

Tasks you would expect to do need to be formalised, structured, and set owners, and those responsible for carrying them out. This is absolutely critical to the ongoing success of data management and turning your data into gold. One specific task that I wanted to highlight is that of cleaning your data. It links to the later area of data quality.

If you ensure your data is clean or, indeed, tidy (not messy), then a great deal else falls into place neatly. Analytics make more sense; reports and dashboards can be trusted, and staff and customers are happier.

Reach for quality: Having irrelevant data, old data, nonsensical data is simply not acceptable This is the cardinal sin of a great deal of companies across the world, in many sectors. It’s not something specific to social housing. We talk about rubbish in, rubbish out. Well, I certainly mention this regularly.

Imagine you need to make a decision based on the life span of a boiler part, but this information is messy, with some wrong dates in there. It may be that this was human error, or perhaps even the way your new system managed the data. The point here is that if you aren’t on it, and the data isn’t accurate, relevant, or, indeed, even spelled wrong then decisions based on this data can end in disaster, literally.

The main take away here is that there’s homework to be done. There’s maintenance required, and if you don’t do it, and don’t set it out in the strategies and policies, then expect a data swamp, expect mistakes, expect costs to rise and expect unhappy customers.

The next steps

So, above outlines three straightforward yet challenging areas to get sorted. To be fair, you can apply these to not just avoiding your data lake becoming a swamp — you can simply use them as guides to better manage your data, lake or not.