Your employee experience metrics can look perfectly healthy while telling you almost nothing. Open rates climb and the dashboards glow green, and you still can’t prove the business is any better off. If you lead internal communications or employee experience at your organization, it’s a challenge you might be familiar with. This post by Interact’s Senior Content Marketing Executive Sophie Hamblett looks at why the usual numbers mislead, and which ones truly show your work is landing. 

Which employee experience metrics are vanity metrics? 

Vanity metrics are numbers that can go up and feel like progress, but that don’t directly connect to business results. In internal comms or employee experience, the classic examples are “activity metrics” such as email open rates, click-through rates, page and article views, logins or active users, total search volume, and video plays. 

You shouldn’t stop tracking activity altogether. Used in specific contexts, activity metrics can be important data points. Active users tell you whether people are showing up, email open rates tell you a message reached inboxes and had a compelling subject line, and video plays tell you a topic drew interest. 

The problem is that when used as a measure of overall performance, they become vanity metrics, flattering us into thinking we’re achieving more than we are.When an intranet user logs in, it doesn’t mean they’ve completed a task. When someone on your mailing list opens an email, it doesn’t mean they’ve read it. When an employee watches a video, it doesn’t mean they’ve understood the content.  

Most internal comms reports lead with the activity metrics in the table below because they’re quick to pull and easy to explain to stakeholders. They’re the metrics that fill most quarterly reports, and most intranet analytics dashboards put them front and center. The problem isn’t what they tell you, it’s what they don’t.  

Metric reported	What it shows	What it can hide
Email open rate	The message reached inboxes	Whether anyone read, understood, or acted on it
Click-through rate	People are interested in learning more	Whether the click led to any useful action
Page and article views	The content is popular	Whether the right people found what they needed
Logins or active users	People are adopting and using the platform	Whether people accomplished anything once inside
Total search volume	People are searching	Who is searching for what, and how successful they are
Video plays	People pressed play	Whether viewers watched and absorbed the whole thing

These numbers can be perfectly accurate and still mislead even seasoned internal comms and employee experience pros. They’re easy to grow without improving impact, and they can climb in the same quarter that your people grow more frustrated and less engaged. For employee experience metrics that mean something, you have to measure more than activity. 

How the wrong employee experience metrics hide what’s really happening: Lessons from AI Search 

Analyzing Interact customer data highlights this problem in the real business scenarios. For many customers, AI Search has replaced the old keyword search bar with a tool that understands natural language, so people can ask a question conversationally, the way they would ask a colleague, and get a direct answer rather than a list of links.  

We analyzed millions of real searches from before and after customers switched to AI Search. The result? Total search volume barely moved.  

Total search volume is a common activity metric, and if customers had stopped there, they might easily have assumed that after their investment and adoption of AI Search that nothing changed. The real story is very different. Underneath the flat volume, two groups of employees were moving in completely different directions, and both were better off. 

The first group were power users who already made regular use of the traditional search bar. With AI Search, their daily search count dropped by around 30 percent, but the work they completed stayed the same or increased. They were getting the right answer in a single query where keyword search used to take several attempts, so they searched less. Their search volume went down. 

The second group were the employees who barely searched before, running fewer than one search every ten days. They skew heavily toward frontline roles such as staff nurses, store-floor staff, loan processors, and branch managers. These are people who rarely knew the exact system or policy name the old keyword bar demanded. After they got access to AI Search, their search activity rose by around 121 percent. Roughly two-thirds of them now search more than they used to. 

Employee group	Search behavior before AI Search	Search behavior after AI Search	What the numbers show
Power users	At least one search a day	Around 30% fewer searches, with the same or more work done	They reach answers faster, so fewer searches now do more
Occasional users	Fewer than one search every ten days	Around 121% more search activity	A group that rarely searched now uses it regularly

If we judged the success of AI Search on overall volume, we’d see a flat line and move on, but when we look at who is searching and how, the story flips. The heaviest users are getting answers faster, and a large group that struggled with the old search bar is finally getting value from it, with natural-language questions scoring around 71 percent positive feedback. The same caution applies to almost any single number on your dashboard, because a flat or rising total can hide very different movements underneath. 

Why do vanity metrics put internal communications and employee experience success at risk? 

Relying on vanity metrics to shape your comms and employee experience can backfire in several ways: they make it hard to gain budget and stakeholder support, they tune your content strategy to the wrong signals, and they cause you to miss out on valuable opportunities to move your workplace forward. 

How vanity metrics impact internal comms budget 

The most important way vanity metrics impact internal comms and EX teams is budget. Executives fund outcomes, not opens, and the stakes are high. Gallup’s State of the Global Workplace estimates that low employee engagement now costs the world economy around $10 trillion a year.  

Many leaders are ready to put money behind programs that can help. The catch is that you need proof – and activity metrics like open rates and page visits don’t mean much to real ROI. If they can’t see what changed (or what could change) for the business as a whole because of your comms or employee experience, you’ll lose out on buy-in and budget, which makes it difficult to move your strategy forward. 

This is a common problem for communicators and EX pros. At Interact’s recent Measuring What Matters live online event, guest speaker and Forrester Principal Analyst Cheryl McKinnon described a trap she calls the data desert: teams on homegrown or manual systems with so little analytics that they cannot say whether they are improving or how they compare with their peers. Without clear data, those teams struggle to present the evidence leaders need to invest with confidence. 

How vanity metrics harm your content strategy 

Vanity metrics are also detrimental to your content strategy, because they don’t give you a great sense of what’s effective and what’s not. Using activity metrics as vanity metrics (i.e., treating email open rate as a measure of success rather than using it to see if subject lines are working) means you’ll likely repeat the same mistakes over and over again, with no meaningful improvements in strategy. 

For example, take a video promoting a company-wide policy update, pinned to the top of the intranet homepage. It records thousands of views because every employee lands on the homepage when they log in, but almost nobody clicks through to the detailed explanation, and if asked, most staff would not be able to say what changed or what’s needed from them. At the same time, the activity metric (view count) performs well, so the comms team considers it a success.  

A video records thousands of views because every employee lands on the homepage when they log in, but almost nobody clicks through to the detailed explanation.

Alternatively, tracking actual policy adoption tells you whether your campaign is working, whether you need to tweak your approach, and what you can learn for future campaigns.

Why vanity metrics lead to missed opportunities 

Judging success on vanity metrics alone means missing the hidden insights that make a big difference for employees. For example, a flat activity metric typically doesn’t tell you how a particular initiative is impacting different sub-sets of your audience. Take Interact’s own AI Search findings. Measuring success on search volume alone would have that adoption had stalled and that engagement hadn’t changed, instead of uncovering a meaningful improvement for an underserved segment of the workforce. 

Click-based activity is another category of vanity metrics due to how AI has changed how our workplaces operate. Today, AI assistants increasingly answer questions directly instead of sending people to a document to find the answer themselves. That means fewer clicks to content, but it doesn’t mean the content is any less useful. In fact, it’s likely that key information is reaching even more people despite a misleading decline in overall traffic. Looking at clicks alone would ignore the meaningful time savings AI is providing employees, and lead to counterproductive digital workplace changes. 

That means fewer clicks to content, but it doesn’t mean the content is any less useful.

There are plenty more examples like these that show the importance of looking beyond activity. But what should you measure instead? 

What employee experience metrics should you measure to prove value? 

This is the question guest speaker Cheryl McKinnon set out to answer at Measuring What Matters, and her recommendation, grounded in Forrester’s research into digital employee experience, was to shift attention to two measures that connect communication and EX to value. The first is whether employees complete the actions you ask of them. The second is the business outcomes those actions drive. Both move you from measuring activity to measuring impact, and here’s why.  

Call-to-action and task completion 

McKinnon borrowed this measurement tactic from marketing: every important communication should prompt an action, and we need to measure whether people take that action. In an internal comms context, that action might be registering for training, acknowledging a new policy, completing benefits enrollment, or finishing a compliance module. One of the strongest metrics she pointed to is help-desk deflection, where questions that used to become IT or HR tickets turn into self-service instead. 

Every important communication should prompt an action, and we need to measure whether people take that action.

The value of task completion is that it links a piece of internal communication or a change in the employee experience to what happens next – something open rates can’t do. McKinnon illustrates this well: a better onboarding process that brings a new salesperson up to speed 90 days faster than before. Because a quota-carrying rep generates revenue once they are productive, that speed carries a direct financial value, which is a far more persuasive number to put in front of leadership than a page view. 

Business outcomes 

The second measure is the set of outcomes the business already cares about, connected to what your communications influence. In the live session, McKinnon pointed to examples such as employee retention, fewer safety incidents, less shop-floor downtime, quicker adoption of new systems, and the everyday time savings that add up across thousands of people. 

The catch, she stressed, is that you cannot set these targets on your own. They call for cross-functional work – asking HR, sales, and operations how success is measured, and then building internal communication and employee experience improvements that help them hit those goals. When you tie your communications to numbers other leaders are already accountable for, employee experience metrics stop being a comms-only scorecard and become business evidence. 

Three tips to get the most out of your employee experience data 

Fast forward a few months. By now, you’ve decided which employee actions and business outcomes to measure, figured out how to track them, and started collecting data. Now, it’s time to start acting on the results. These three habits will help get the most out of your numbers: 

1. Look past the average. As The State of AI Search shows, we should always break results down. This could be by role, location, channel, device, or previous behavior. For example, if you’re tracking benefits enrollment, corporate employees may sign up quickly while the frontline needs a few extra nudges. Segmenting results helps you see who’s responding best, who’s being missed, and where to focus next.  

2. Use employee listening to add context. At Measuring What Matters, Cheryl McKinnon advised teams not to rely on quantitative data alone. Add employee listening, such as comments, pulse surveys, search feedback, or conversations, to explore what may be driving patterns in your findings.  

3. Make one focused change at a time. Upending your entire approach won’t tell you what finally led to improvement, so stick to one change at a time. Revise the content, target a different audience, switch the format, or remove a point of friction. Deliver the change and compare the same measure after an agreed period. Keep what worked, record what didn’t, and use the result to choose the next step.  

After you’ve got all of your data points, record and report the whole process rather than just your conclusion. This includes what the data showed, what you changed, what happened, and what you’ll do next. Over time, doing so creates stronger evidence of business impact and a more credible case for ROI. 

Your next step to tracking the employee experience metrics that count 

Any dashboard can show you a number that’s climbing, if a climbing number is what it’s set up to show. The more useful and more difficult question is whether employees are getting what they need, and whether the business can feel the difference. This is the best way to demonstrate the true impact of your comms and employee experience. 

If you want to go deeper, watch the on-demand recording of Measuring What Matters: Connecting Employee Experience and Communications to Business Outcomes, our live online event with Forrester Principal Analyst Cheryl McKinnon as guest speaker. It’s a research-backed playbook for defining and tracking the employee experience metrics that count.