Can technology provide more accurate analysis than humans?

A PublicRelay Partnership with MIT finds that the answer is no.

A study that set human media analysis head-to-head with MIT’s natural language technology processors found that automated solution only had:

• 9% accuracy on detecting key message presence
• 20% accuracy on allocating the correct sentiment
• 33% accuracy on highlighting the precise experience of the customer 

Background

Social media analysis is one of the fastest-growing areas of text analytics.

For efficiency and economical reasons, most media analytics providers in the space rely on automated technology to extract emotion and sentiment.  But can technology actually provide more accurate text analysis than a human?

In collaboration with Toyota Motor NA Energy & Environmental Research Group, PublicRelay and the intelligent minds at the Massachusetts Institute of Technology (MIT) Analytics Lab set out to find an answer.

The Project

Over the course of three months, the MIT Analytics Lab team tested various technologies to:

• Understand which topics car enthusiasts are discussing in relation to alternate-fuel vehicles on Twitter
• Identify “significant” tweets based on topic inference from above.

The goal was to identify tweets that demand further tracking or direct engagement, and formulate messaging that Toyota might use to drive social media conversations themselves.

The Technology

• MIT built two different modeling approaches to interpret the data– Latent Dirichlet Allocation (LDA) and a Biterm Topic Model (BTM).
• For both processes, the MIT team indicated to the machines which words were topically important.
• The processes would then clean up the tweet text by isolating only the most “important” words and analyze those for key messages, sentiment, and experience according to pre-set definitions.

The Results

• The accuracy of the technological solution versus the human in three key measured attributes was only 9% (key message), 20% (sentiment), and 33% (experience).
• While the technology may be useful in eliminating irrelevant posts before humans analyze them, the MIT team found that 80% of their machine-learning work was simply data clean-up — the computer algorithms had a hard time deciphering commonly used symbols in on Twitter, like @, #, and even text emojis.
• Lastly, the study showed how technology did not handle rapidly changing topics well:
  • Analysis over a short time frame fails to detect the quickly‐dissipating influence of one‐off news events and crises.
  • Rapidly emerging issues or breaking news such as the Tesla autopilot crash were slow to be reflected in a model.

The Client

Our client is a U.S. humanitarian organization that provides emergency assistance and disaster relief, including volunteer support, blood donation, and fundraising.

The Challenge

In 2020, the organization tasked its communications team with a tangible business goal: to boost the organization’s total fiscal fundraising and blood collection over four months.

For this to work, the team needed to implement a strategy for:

1. Affecting growth in fundraising and blood collection using earned media, and 
2. Demonstrating their role in achieving the results.

Our Solution

The communications team turned to PublicRelay for a data-backed PR attribution strategy to:

1. Demonstrate their impact on engagement – the audience behavior most often associated with increased donations. In other words, the team wanted their earned media coverage to facilitate increased interactions with the organization via their website, mobile app, and volunteering.
2. Establish their role in increasing brand credibility – the key dimension of brand health and reputation associated with audience engagement.
3. Pinpoint the reputational drivers that contribute to concepts associated with brand credibility within their earned media coverage, thereby enabling them to focus their messaging on the drivers with the most impact.

Using PublicRelay’s human-augmented technology, the communications team received a highly accurate and nuanced analysis of the organization’s earned media coverage during the four-month period.

Download Success Story

Download

Results

When interpreted alongside the organization’s reputation survey results, measures of engagement, total fiscal fundraising, and blood donations, PublicRelay’s insights revealed that:

• Earned media coverage in priority publications is positively correlated with improved brand credibility and increased engagement with the organization.
• Engagement stemming from the organization’s earned media coverage can be directly attributed to increased fundraising during the period.
• Though blood donations could not be directly tied to the organization’s earned media coverage in 100 priority publications, there is evidence to suggest that coverage in smaller local publications may have a greater influence on blood donations, performing 50x better than priority media on the blood services website.

With this approach, the communications team:

1. Achieved their business goal of increasing total fundraising,
2. Could present their CEO with data-backed evidence of their impact on the outcome, and
3. Attained nuanced insight into their earned media strategy that they could apply to future campaigns.

Want to learn more about how PublicRelay’s PR attribution strategy can help your team connect your work to business results? Get in touch with our team at solutions@publicrelay.com.

The value of competitive intelligence extends beyond informing strategic business operations. PR teams can also benefit from knowing how competitors perform by tracking their earned media coverage.

By including competitive intelligence in your media measurement program, your team can access crucial insights into your industry’s PR landscape and a deeper understanding of your own communications performance.

What is Competitive Intelligence?

Competitive intelligence is the practice of collecting and analyzing data on your competitors to inform your PR strategy. In communications, it requires tracking competitors’ earned media coverage using meaningful metrics that provide insight into various aspects of their strategies and performance. This data is then used to paint a complete picture of the competitive landscape your team operates within.

Why Is Competitive Intelligence Important?

Competitive intelligence is important because it allows you to benchmark your performance against industry peers. In addition to comparative analysis, it enables you to learn from competitors’ strategies, especially when it comes to their handling of communications crises.

You can also use this data to learn what stories and topics gain the most social traction, as well as identify opportunities in your market.

How to Use Competitive Intelligence to Inform Your PR Strategy

Once you’ve identified your key competitors and started measuring their media coverage, you can begin to analyze the data against your own to generate insights that can guide your communications going forward.

Here are a few ways you can use competitive intelligence to inform your PR strategy:

Benchmark Your Performance Against Peers

Benchmarking will help you to understand your performance in relation to peers and evaluate your progress over time. Collecting data on your competitors allows you to compare your communications on a variety of metrics. There are four metrics you can use as a starting point to conduct a benchmark analysis: Article Volume and Tone, Social Sharing, Share of Voice, and Key Message Penetration.

Article Volume and Tone

Measuring Article Volume and Tone is the simplest way to get a big picture insight into your competitors’ general brand awareness and sentiment as portrayed by the media. The tone of competitors’ coverage will reveal the brands that have generated the highest proportion of positive coverage, indicating that theirs are the strategies to learn from. Keep in mind that accurately evaluating tone is a challenge for AI-only media measurement tools, which is why we use human-augmented AI for reliable sentiment analysis.

Social Sharing

Social Sharing reveals which companies your audience is most engaged with. And, when analyzed alongside other metrics, like Tone or Key Message Penetration, it exposes the reach of that content. Consider Social Sharing a sign of the effectiveness of competitors’ coverage in resonating with your audience, for better or worse.

Share of Voice

How much attention is your brand receiving compared to your competitors? Which of your competitors dominates the media coverage relevant to your brand? Measuring Share of Voice is a broad indicator of brand awareness and your position within your market. This metric is useful for benchmarking your progress in increasing your company’s dominance in the industry conversation over time.

Key Message Penetration

While tracking the number of times your company was mentioned relative to competitors can show you who is leading your industry conversation, you can go deeper into the value of those mentions by measuring Key Message Penetration. In other words, how many of those media mentions reflect one of the core messages your brand wants to be known for? Not only will it show how effective peers’ messaging campaigns are, but you can learn from the outlets, authors, and strategies that give them the most success.

Identify Influencer Opportunities

Are there any journalists or publications that mention competitors often and favorably but aren’t covering your company? Do any of these authors generate outsized social sharing among your target audience?

Analyzing influencer data as a part of your competitor tracking will highlight the sources of their positive coverage and identify opportunities for your team to build relationships with industry influencers.

Learn From Competitor Crises

There is a lot you can learn from how your competitors handle communications crises relevant to your industry.

For instance, was their response proactive or reactive? Which messages resonated the most with your shared audience via social sharing? Which influencers covered their response positively? Negatively?

Analyzing their strategies in response to crises and the results will provide you with evidence-based recommendations in the event a similar crisis targets your company.

Use Predictive Analytics to Dominate the Conversation

A fully optimized media analytics program will not only allow you to learn from competitors’ past performance but will also help you to predict their future communications.

Using our predictive solutions, you can anticipate the messages your competitors plan to focus on in the coming weeks and the journalists they intend to target with that messaging. With this data, you can get a leg up on the narrative and dominate the conversations your competitors are vying for.

Strengthen Your PR Strategy with Competitive Intelligence

An in-depth understanding of your competitive landscape will help you develop a strong PR strategy. Detailed information on your competitors’ communications will highlight what works and what doesn’t with your shared audiences. Further, by benchmarking your communications performance against peers, you can identify opportunities for growth and measure your progress over time. PublicRelay supports your communications with advanced media analytics using a hybrid approach of cutting-edge technology and human analysts. Click here to learn more about how our solutions can help you to track your competitors!

Wednesday, May 4^th, 2022, US communications analytics and advisory firm PublicRelay has announced plans to establish a media analytics centre in Waterford City creating approximately 40 jobs over the next two years.

Founded in 2008 and with operations in Denver, San Diego, and Portland, PublicRelay is headquartered in Tyson’s Corner, Virginia and provides clients with insights based on their media coverage in print, online, broadcast, and social media. The company opened its first European office in Dublin in 2018, followed by its Cavan site in 2019.  PublicRelay’s expansion in Waterford City is now their third site in Ireland.
This project is supported by the Irish Government through IDA Ireland.

Tánaiste and Minister for Enterprise Trade & Employment Leo Varadkar said: “I’m really pleased PublicRelay has chosen Waterford City as the location for its new media analytics centre, which will create 40 new jobs over the next couple of years. It’s a real sign of the company’s confidence in Ireland and the wealth of talent we have here. Thank you to the team for their continued commitment and investment and the very best of luck with this latest expansion.”

PublicRelay will operate on a hybrid basis with its South-East Hub located in BoxWorks 2, a co-working space in Waterford City.

“Our experience in Ireland has been outstanding since we opened our first office here in 2018.  We’ve been able to access a hardworking, highly educated, and friendly work force, and our Irish team has been a big factor in our company success over the past four years. I’m delighted that we are now in a position to open our third office in Ireland, and I am looking forward to visiting Waterford very soon” said Eric Koefoot, President & CEO.

“We are excited to be further expanding our operations in Ireland.  We looked at a number of locations around the country, but the proximity of the university combined with a great facility in such a vibrant area of the city won us over.  I was impressed with Boxworks’ flexibility and willingness to adapt to our needs as we built out the team” said Karl Finn, Director of Irish Operations.

PublicRelay is hiring Associate Analysts for its south-east media analytics centre. The company also has a number of open roles at its EU HQ in Dublin. For more information on how to join the team at PublicRelay, visit www.PublicRelay.com

Executive Director of IDA Ireland Mary Buckley said: “PublicRelay’s decision to open a third Irish site in Waterford following the success of their Cavan and Dublin operations, demonstrates the company’s strong commitment to Ireland and confidence in the country’s highly skilled and talented workforce. “This further investment exemplifies IDA Ireland’s continued commitment to winning jobs and investment for the regions.  I’d like to wish PublicRelay every success with this latest expansion.” 

##

For media enquiries, please contact – karl.finn@publicrelay.com

Alison Nulty, Regional Communications Executive, IDA Ireland, alison.nulty@ida.ie 087 3492590

About PublicRelay

PublicRelay delivers quality analytics that increase the value and influence of communicators in their organization. PublicRelay’s unique human-augmented technology provides data and insights that prove communications’ impact on business goals, predict media outcomes, and guide future strategy. Known for continual innovation, superior data-driven insights, and exceptional partnership, PublicRelay elevates communications data to the standards of today’s C-suite leaders.  For more information, please visit www.publicrelay.com.

Editor’s Note: Chief Insights Officer Mark Weiner’s commitment to inspiring PR teams of every size to begin measuring their communications work led to his partnership with PublicRelay. Like Mark, we strive to support communications teams at every stage of PR Measurement – from simple, entry-level evaluation to more advanced PR attribution programs. And what better way than to join forces?

One of the biggest and most pervasive myths in public relations is that good measurement must be expensive and complicated. Years ago, speaking at a public relations conference, I shared stories from a variety of Fortune 500 clients. This seemed like the best way to make an impact on the importance of communications research, after all, if the biggest PR departments invested in research, everyone should. Then, a member of the audience commented, “I know what you’re saying is right, but I don’t have a Fortune 500 budget. So, rather than doing it wrong, I choose not to measure at all.” At that moment, I realized that in promoting my position, I perpetuated the myth I’m committed to dispelling.   

In fact, research is more accessible now to more professional communicators than ever before. The Institute for Public Relations and AMEC offer all the guidance you’ll need including case studies, methodology, and instructional frameworks. What’s more, communications technology platforms are now ubiquitous and at many price points.

Start With Simple PR Measurement

Even if management isn’t demanding PR measurement, you should consider what can be done and the positive effects of simple measurement. As you’ll certainly see on the IPR and AMEC websites, the measurati speak of four types of measures:

1. Inputs
2. Outputs
3. Outcomes
4. Business Results

Inputs track your expenditure in terms of time and money. By monitoring the levels of your investment, you can show how wisely you spent your budget and the efficiency of your actions. When combined with output, outcome, and business measures, your efficiency equation will show your “Cost per ‘X’” where “X” may mean “cost-per-thousand circulation” or “cost-per-positive story” or “cost-per-percentage point of increased awareness.” It’s a good and easy way to begin.

Outputs measures what you put out, such as news coverage and social media activity. In addition to simple quantitative tabulations, you can look at qualitative measures like the tone of coverage and the presence of key messages. Keep in mind that technology alone has trouble accurately assessing quality, so be prepared to review the computer’s calculations. Finally, the technology enables competitive analysis to track share-of-voice as well as other comparative measures. 

Outcomes measure the effects of your communications on the awareness, understanding, preferences, and attitudes of your target audience. These answers reside in people’s minds. There are two ways to measure inexpensively: survey technology and social media. Low-cost survey technology allows you to ask respondents the questions that reveal the extent to which – or even if – their positions changed because of your work. Social media enables you to gauge awareness, attitudes, and behavior as evident in posts where people share their opinions about and experiences with your product or service. Social media listening is like a giant unmoderated focus group, and you can learn a lot by simply listening.  

The purpose of connecting public relations activity to business results is to demonstrate PR’s ability to affect sales, cost efficiency, and risk mitigation. While certain methods are complicated and expensive, making the “PR-to-sales connection” can happen during times when PR operates in isolation (so there’s no other way to explain the result). You can also study social media results to uncover references to purchase behavior (“I just bought the new improved detergent, and it really works!”). While linking PR with sales is sexy, the most accessible approach to PR’s effect on business outcomes is efficiency. This requires linking PR Inputs with Outcomes to show that you’re doing more with less and for less. By lowering the cost on each positive story (budget divided by positive stories), you’re improving the return of the organization’s investment in PR. Finally, avoiding catastrophic costs is a measure of your good counsel. Compare your crisis averted with a competitor’s crisis and see what it cost them by doing it wrong. Measure their stock performance, their market share, or other data, much of which is publicly available through trade associations and trade media.

Low-Cost Best Practice PR Measurement

One of the best examples of PR research I’ve seen was entered in the PRSA Silver Anvil Awards. A small town submitted an entry that quantified inputs, outputs, and behavioral results – and the campaign cost nothing! The town sought to reduce the number of stolen cars by reminding trusting citizens to remove their keys from the ignition while running routine errands. The program began by tabulating the number of car thefts in the town and comparing their town data with neighboring communities. Once the behavioral benchmark was sent, the town began a media campaign in the local newspaper. Within a few months, car thefts were eliminated. 

Communicate Your PR Performance with Data

Over the past few years, I’ve adopted a new measurement motto: “Begin simply. Simply begin.” By committing fully to PR measurement, even in simple ways, you communicate PR performance in the language of the boardroom. Data transcends language barriers to demonstrate the effect of your good work. What’s more, fundamental measures create an appetite among senior executives for more PR programs, more good results, and more measurement. You, in turn, position yourself for more: bigger budgets, more resources, and greater acclaim.

Ready to show your impact? Download our PR attribution whitepaper to learn how you can connect your communications work to business results now!

# # #

Mark Weiner is Chief Insights Officer for PublicRelay and the author of “PR Technology, Data and Insights: Igniting a Positive Return on Your Communications Investment.”

Originally published in PRNEWS April 2021 issue.

The Client

Founded in 1875, Prudential Financial has grown into a multinational financial services leader that now operates in over 40 countries. The Fortune 100 company specializes in insurance and investment management with a mission of helping its customers to grow and protect their wealth.

The Challenge

For years, the financial services industry has tried to reach affluent Black consumers by promoting CSR and DEI efforts.

It wasn’t working.

A study conducted over two years revealed a decline in brand health among Black Adults not just for Prudential, but across the entire financial services industry.

Company leaders suspected that different messages may resonate more with affluent Black consumers, but what were they? And how could their PR team develop a data-driven strategy for reaching Black audiences going forward?

Our Solution

Prudential’s PR team decided to use PublicRelay’s PR attribution strategy to generate insights from the company’s earned media coverage that would:

• Uncover the themes and messages that have the greatest impact on brand perceptions among affluent Black Adults,
• Connect the dots between their work, their earned media coverage, and changes in brand health among affluent Black audiences, and
• Develop a framework for developing data-driven approaches to reaching specific audiences in the future.

Download Success Story

Download

Results

Trust emerged as the aspect of brand health that mattered most in Black Adults’ decision to become customers of a financial services company.

And, it turns out the company leaders were right: Trust among affluent Black consumers was less impacted by CSR and DEI and more affected by companies’ business performance and thought leadership coverage.

The findings revealed that, while Citizenship and CSR coverage could improve Favorability among highly invested Black Adults, it does not translate into Trust for that group. Research coverage, on the other hand, correlated with Trust for both high-income and lower-income Black Adults.

With these insights, the PR team was able to intelligently pivot its strategy and maximize the impact of its initiative to target Black audiences.

Want to learn more about how PublicRelay’s PR attribution strategy can help your team focus your messaging? Get in touch with our team at solutions@publicrelay.com.