By Katie Wilburn, Head of Transformation, First Choice Homes Oldham, and Performance Group Lead for Greater Manchester Housing Providers

Like many performance and business improvement leads, the end of the financial year means a significant proportion of my time this month will be spent in a dark room, eyeballing year end performance, setting targets and strategies, and considering what needs to change in the coming 12 months to achieve them.

As I’m sure is the case for everyone else too, chief amongst the performance metrics I’ll be scrutinising in the coming weeks are the Tenant Satisfaction Measures (TSMs), and lately that’s left me with decidedly mixed feelings about the impact these ‘new’ regulatory indicators have had on service improvement across the sector since their introduction.

Now, I’m not trying to jump on the “not a fan of the TSMs” bandwagon which seems to be rolling through town at the minute. I was a pretty vocal advocate for their introduction, and I see real benefits in having a consistent set of sector-wide, publicly available metrics.

I am also a firm believer in the old adage that “what gets measured gets done”, and the TSMs provide a clearer message from the regulator than ever before that RPs need to do the basics well first: focus on customer experience alongside reliable business and financial acumen.

Anyone reading with interest the early C gradings could not have failed to notice, for example, that safety compliance is king, and 95% just isn’t good enough.

However, for all their worth, here is my problem – they don’t actually measure what they’re supposed to.

And more than that, we seem to be expected to subscribe to this collective delusion that they do, when everyone knows they don’t. It gets mentioned in meetings and quickly shot down as an excuse or a justification, not to be mentioned again for fear of looking like you’re not on board with driving forward improvements.

In the same way “listening to customer views and acting upon them” since the bygone days of STAR has never measured the quality or breadth of tenant engagement, “feeling safe in your home” doesn’t measure property safety compliance, and even “satisfaction with complaint handling” doesn’t really measure formal complaint handling (upwards of 75% of customers who think they’ve raised a formal complaint actually haven’t). I could go on. (For full transparency, I am focusing on the 12 perception rather than performance measures here).

And this is all part of a wider delusion within the customer feedback and service improvement industry generally. We all pretend satisfaction questions directly measure service delivery (possibly even against a benchmark of a published service standard if you’re truly delusional), when we all know what they actually measure is a customer’s perception of service delivery, balanced against their expectation of what that service delivery should look like (reasonable or not), with a little bit of their current mood, today’s weather and a vast array of other uncontrollable variables mixed in as seasoning.

So, what’s the answer? Am I saying we should discount the TSMs as an unreliable witness and stop trying to improve them?

No, I’m not. And, moreover, at the same time as highlighting the “inaccuracy” of TSMs I’m also saying they are totally valid feedback because we have to accept that perception is reality for the person who sees it that way.

If 20% of our customers believe their home is not safe that is a problem, even if your building safety compliance is 100%. If 75% of your customers believe they have made a complaint, but it wasn’t recorded as one, then the reality is they have made a complaint, and you haven’t recorded it. And that’s also a failure against the Housing Ombudsman’s code…. (“…an expression of dissatisfaction, however made…a resident does not have to use the word “complaint” for it be treated as such”).

What I am saying is that we need to stop pretending that TSMs are more forensic than they are. Because that means we can also stop pretending that improving delivery in a single, internally defined workstream such as ‘formal complaints handling’ will be enough to improve customer perceptions in this area. The real issue is broader, infinitely more complex, and probably to do with owning and responding to lower level service failures business wide, which in turn needs a broader, infinitely more complex, business-wide solution.

So, what could we do differently? Having completed my cathartic ranting (which, let’s be honest, we all need from time to time), here are my more practical thoughts on how, as a sector and individual providers, we could start to respond to TSM feedback more effectively:

1. Acceptance (Part 1): I’ve already touched on this above, but stage 1 for me is to accept that TSMs don’t directly measure the quality-of-service delivery in the nominated areas. They certainly don’t measure delivery against a published service standard. They are impacted by customer expectations and levels of customer awareness, and all of that is completely fine. Because once we accept and acknowledge this, we become open to a broader, more multifaceted, and complex range of approaches to improve.
2. Ask why: If we are accepting that the TSMs don’t just measure the service area they are supposed to, then the first logical challenge is to understand what they are actually measuring, right? Take a leaf out of your average toddler’s book and start asking why in your TSM surveys: Why is this good? Why is that bad? How could we make this better? Don’t ask why after every question though. Stretching the analogy a little further, as anyone with a toddler will tell you, what starts out as charmingly inquisitive transforms into remarkably irritating very quickly (or perhaps I was just a terrible parent).
3. Own the message: There is more than one way to change a perception. Obviously, I’m not advocating using comms to tell customers their views are wrong here, but equally it shouldn’t be taboo to tell customers that you were surprised by a poor score and what they maybe don’t know about the service being delivered. We can tell customers what we are doing to improve safety, how we are working with partners to improve local areas, and why there are some things we’ve made proactive decisions not to offer that they may have been expecting. If this reassures and informs customers, and demonstrates we are listening to their feedback, it may well improve perception scores too. As a minimum it starts an adult-to-adult conversation between tenant and provider.
4. Stop talking about apples and pears: The amount of time I spend listening to people talk about hypothetical fruit, you would honestly think I was a greengrocer not a business change specialist. That’s a little flippant, so I should probably concede that yes, all registered providers are different; we have different geographies, different tenure, different customer mixes and different challenges, but we all work to largely the same statutory and regulatory requirements and we all put roofs over people’s heads. We are more alike than we are different. We might be apples and pears, but we are all definitely ‘green fruit’. Far too much time and effort around the TSMs is absorbed by externally focused analysis, driven by justifying and explaining poor comparisons. Time which could have been better spent identifying internal improvement plans.
5. Benchmark for the right reasons: Linked to the above, the whole fruit debate is largely borne out of benchmarking being used as a stick to beat ‘underperforming’ teams with. If you want to benchmark, do it with the purpose of identifying practical and actionable improvements. If someone is scoring markedly better than you in one of your challenging areas, ask what operationally or structurally they are doing differently (not what their collection method and sample size is).
6. Play the long game: Customer perceptions are changeable, which should be a positive message for anyone who finds themselves at the bottom of whatever league tables are published in the coming months. We also need to keep this in mind when we track scores over time. A change of a few percentage points might reach a level of month-on-month ‘statistical difference’, but in the real world means nothing meaningful about a change in service delivery. Chasing and overanalysing day-to-day, week-to-week, or even month-to-month variations is often wasted effort. Focus instead on scrutinising the longer-term trends and direction of travel.
7. Change something! This might seem blindly obvious as a piece of practical advice, but I’m continually amazed by the level and longevity of ‘analysis paralysis’ the TSMs seem to have evoked across the sector. And, going full circle back to ‘acceptance’ I think at least in part this is driven by the fact we can’t get our collective heads around why they don’t measure what they pretend to, so now, we feel like we don’t have enough information to know what to change for the better. My advice is: try something. In the longer term, ask your customers what you should change but if you are not doing that already, ask your customer-facing teams – they see the problems too. We’ve spent two years now asking customers what they think of our service performance, it’s long past time to show we are listening, receptive, and willing to change.