What is advertising effectiveness measurement? A thorough explanation from media-specific metrics to practical points for achieving results.

"Advertising effectiveness measurement" is essential for accurately understanding the results of advertising and using that information to inform future strategies. Which advertisements are contributing to sales? Which channels should we increase investment in? Effectiveness measurement provides the foundation for answering these questions.

However, in today's world, we live in a time where all kinds of channels coexist, including mass advertising such as TV commercials and transit advertising, web advertising, and social media promotions. If you make a mistake in the measurement method, you risk making wrong decisions and failing to optimize your advertising investment.

This article systematically explains everything from the basic concepts of measuring advertising effectiveness, key metrics for each media type, the practical aspects of brand lift surveys to measure psychological effects, why effectiveness measurement is said to be difficult, how to choose tools, and practical points for achieving results.

Free download of case study

Gain knowledge of "statistics" and accelerate your marketing
- Introducing the basics and "MMM," ​​a statistical analysis method that has recently been gaining attention again -

Download Now

What is advertising effectiveness measurement?

Advertising effectiveness measurement refers to the measurement of online and offline advertising.A process of visualizing and evaluating the degree of effectiveness using numerical values ​​and indicators.This is the point. To achieve maximum results within a limited advertising budget, it is essential to rely on objective data-based analysis rather than intuition or experience.

The role of measuring advertising effectiveness is not limited to simply confirming whether an advertisement was effective or not. For successful patterns, it's about understanding "why it was effective," and for unsuccessful patterns, it's about understanding "why it wasn't effective."Indicators for analyzing causal relationshipsWorks as well.

Companies that thoroughly implement a rapid PDCA cycle for their advertising campaigns are the ones that consistently achieve results through advertising.

Three areas where advertising effectiveness can be measured

The effectiveness of advertising is multifaceted, and simply judging whether it "led to sales" doesn't capture the whole picture. Evaluating from the following three perspectives makes it easier to determine the intrinsic value of each initiative.

① Contact effect (reach and visibility)

These are metrics that show how many people an advertisement reached and how many people saw it. For TV commercials, this would include GRP and reach rate, while for web advertisements, it would include impressions and viewability.

② Psychological effects (changes in attitude and cognition)

This measures how viewers' awareness, interest, and favorability change as a result of exposure to advertising. This includes brand recall, ad awareness, and changes in purchase intent. While often overlooked because it doesn't directly impact sales, it's essential for evaluating brand campaigns and awareness-building strategies. As a method for quantitatively measuring psychological effects,Brand lift surveyThis is widely used (details will be explained in the next section).

③ Sales effect (actions/results)

This evaluates how many actual actions, such as purchases or sign-ups, were triggered by advertising. This is visualized using metrics such as CPA (Cost Per Acquisition) and ROAS (Return on Advertising Spend).

Brand Lift Surveys: A Method for Quantifying Psychological Effects

The ability to quantify the psychological effect of an advertisement—how much it influences people's attention—which cannot be measured by click counts or conversion rates, is achieved through a method called "advertising."Brand lift survey.

What is a brand lift study?

A brand lift study involves dividing a group into two groups: one that has been exposed to advertising (exposure group) and one that has not been exposed (non-exposure group/control group).A survey-type measurement method for comparing changes in brand perception..

By statistically extracting the "change in awareness = lift" caused by advertising, we can visualize the effects of branding measures that are difficult to see in sales metrics alone.

Key metrics that can be measured by brand lift

These indicators are compared between the exposed and unexposed groups to measure the difference (lift) between them.

Main methods for conducting brand lift surveys

① Brand lift function within the platform

Major advertising platforms such as Google, Meta, and YouTube offer brand lift measurement features linked to the ads you run. Lift values ​​are calculated through automated surveys of users who saw the ads and those who didn't. These features are relatively low-cost to implement and easy to integrate with your advertising data.

② Panel survey conducted by an external research company

This method involves utilizing panels owned by research companies such as Intage and Macromill to conduct more precise survey designs. It has the advantage of being able to design surveys that take into account exposure to offline measures that cannot be measured by the platform's functions, such as TV commercials and newspaper advertisements, and also offers high statistical reliability.

③ Before-and-after comparison based on our own survey

This method involves conducting surveys with target users in-house and comparing metrics before and after advertising. While it has challenges such as difficulty in designing a control group and difficulty in eliminating the influence of external factors, it offers a high degree of flexibility in survey design.

Points to note when using brand lift surveys

Brand lift surveys only measure "changes in awareness" and do not prove a direct contribution to sales. To bridge the gap between psychological effects and sales effects, it is important to interpret brand lift results in combination with business performance indicators such as ROAS and ROI.

Furthermore, the quality of the survey design significantly impacts the reliability of the results. If the conditions for the exposed and unexposed groups are not consistent, the lift value cannot be calculated accurately. When using an external research company, it is recommended to verify the validity of the sample size and survey methodology in advance.

The basics of measuring advertising effectiveness: The concept of "relative evaluation"

The basics of measuring advertising effectiveness areRelative evaluationTherefore, we clearly define what we want to achieve through advertising beforehand, and measure its effectiveness by comparing the changes in metrics before and after the campaign launch.

The important thing is not to be satisfied with the fact that "the numbers have improved",Focus on whether you achieved your goal.Therefore, when comparing results before and after advertising, it is necessary to keep conditions such as the period, target users, and channels as consistent as possible.

Furthermore, the effectiveness of advertising is also influenced by overall market trends and competition. Therefore, it is essential to establish assumptions based on the external environment in order to correctly interpret the measurement results.

Mass advertising effectiveness measurement indicators

Mass advertising is an advertising method that delivers information to a large, unspecified audience simultaneously, such as television commercials, newspaper ads, magazine ads, and radio commercials. While it has broad reach and can promote awareness among many people in a short period,It is difficult to quantitatively assess the effects.This presents a challenge.

GRP (gross viewing rating)

GRP (Gross Rating Point)) is a representative effectiveness measurement metric used in television commercials.The total viewership rate of TV commercials aired during a certain periodThis represents the average viewership rating of a television program multiplied by the number of commercials aired during the program.

[ 例：平均視聴率10%の番組でCMを3本放送した場合 → 10 × 3 ＝ 30GRP ]

A higher GRP indicates that the ad reached a larger audience, but it doesn't tell us whether people actually watched the ad. It only shows how many people the ad was seen by.displayedThis is an indicator that shows that.

GAP (Total Gaze Volume)

GAP (Gross Attention Point) was proposed as an indicator to address the shortcomings of GRP. Using facial recognition with a sensor camera,How much attention viewers paid to TV commercialsThis is measured in units of seconds (e.g., 5 seconds of gaze = 5 GAP).

By combining GRP and GAP, it's possible to determine whether the target audience actually saw the commercial. GAP is still an evolving metric, but it has the potential to become the mainstream alternative to GRP in the future.

Advertisement equivalent

Through public relations and PR activities, we aim to highlight the value of our products and services when they are featured in news programs, newspapers, magazines, and online media.Metrics to understand by converting them into advertising costs.In recent years, it has become important to set advertising equivalent value targets in advance and strategically advance PR activities.

Increase in number of name searches

This method measures changes in the number of searches for specific company or product names on Google, Yahoo!, and other search engines. An increase in branded searches after a TV commercial airs suggests a certain level of advertising effectiveness. However, interpretation must take into account other factors such as seasonality and trends.

Increase in site access

Changes in website traffic, such as page views (PV) and unique users (UU), are also important metrics for measuring the effectiveness of offline advertising. Even if branded searches haven't increased, an increase in site traffic suggests there may have been a potential branding effect.

Metrics for newspaper and radio advertising

Web advertising effectiveness measurement index

Web advertising (digital advertising) is advertising placed on websites, social media, video streaming platforms, etc. It allows for detailed targeting, and user behavior is recorded numerically.Tracking the entire process from ad display → click → purchase.Available

The effectiveness of web advertising is measured at the following three stages of the funnel:

Impressions (awareness and reach)

Traffic (Inflow)

A higher CTR indicates that the ad creative resonated with the target audience. However, it's important to evaluate ad effectiveness not solely based on CTR and CPC, but in combination with other metrics.

Conversion (results)

A high CPA doesn't necessarily mean low advertising effectiveness. If the repeat purchase rate is high or the LTV (Lifetime Value), which will be discussed later, is large, then even with a high CPA, the advertising can be considered effective.

(*2) The action you want users to take after coming into contact with your ad (such as requesting information or purchasing a product)
(*3) "LP = Landing Page": The first page you reach after clicking on an ad

Four reasons why measuring advertising effectiveness is considered "difficult"

Simply understanding the metrics is not enough. Understanding why measuring advertising effectiveness is difficult will significantly improve the accuracy of your measurements and the quality of your interpretations.

1. You can’t visualize the effectiveness of offline advertising

Offline advertising, such as TV commercials and newspaper/magazine ads, relies on limited metrics like GRP, making it impossible to precisely measure CTR and CPA like online advertising. This leaves a fundamental problem: "It's difficult to visualize the effectiveness."

2. The effect continues even after the ad ends

The effects of advertising continue even after it has finished being published or broadcast. Ripple effects, residual effects, and brand accumulation effects are believed to exist in all advertisements, and accurate evaluation is impossible without incorporating these long-term effects into the measurement.

(*5) This is a score (249-2017) created from statistically processed analytical data (2021 cases) owned by XICA, so it is not common to all cases.
(*6) In this study, “long-term” is defined as a period of 3 to 5 years.

3. User touchpoints are becoming more diverse

Online advertising has become increasingly fragmented, encompassing search engines, display ads, social media, video streaming, and e-commerce. This makes it difficult to accurately track which ads customers encounter at which point in the purchasing process and through what channels they ultimately convert.

4. We tend to ignore indirect effects

We tend to only evaluate the "final ad" that contributed to the final conversion. However, we must not overlook the contributions of TV commercials and other advertisements that nurtured user interest before that. Indirect effects are difficult to visualize and are therefore often underestimated, which is a major factor that makes measuring effectiveness difficult.

Metrics for measuring business performance: ROAS, ROI, LTV

We will outline three key metrics for evaluating advertising effectiveness and ultimately linking it to "business results."

ROAS (Return on Advertising Spend)

ROAS (Return on Advertising Spend) isHow much revenue was generated relative to the advertising expenses incurred?This is an indicator that shows:

[ ROAS（%）＝（広告による売上 ÷ 広告費）× 100 ]

A ROAS exceeding 100% does not necessarily mean profitability. This is because it does not take into account expenses such as cost of goods sold and personnel expenses. Also, the figure will change depending on whether you define it as "new sales during the advertising period" or "sales from online inquiries," so it is important to set the appropriate criteria for each company.

ROI (Return on Investment)

ROI (Return on Investment)It is,How much profit was gained from the investment in advertising costs?This is an indicator that shows the rate of return on investment, and is also called the rate of return on investment or investment return rate.

[ ROI（%）＝（広告による売上 － 売上原価）÷ 広告費 × 100 ]

While ROAS is an indicator that measures "sales," ROI is an indicator that measures "profit." Since ROI tends to be low in the initial stages of advertising, it is important to avoid judging solely on short-term figures and to evaluate it based on medium- to long-term trends.

Calculate LTV (Customer Lifetime Value) for each advertisement.

LTV (Life Time Value) isThe cumulative profit that one customer (or company) brings to a company.This is a numerical representation of that.

[ BtoCのLTV ＝ 購入単価 × 購入回数 × 継続期間 ]

[ BtoBのLTV ＝ 年間取引額 × 利益率 × 継続年数 ]

[ サブスクのLTV ＝ 平均継続期間（1÷解約率）× ユーザー平均単価（ARPU） ]

In modern approaches to measuring advertising effectiveness, LTV is used.Calculated "per advertisement"This is important because even with two ads that have the same CTR and CPA, the LTV (Lifetime Value) of customers acquired through each ad often differs. By using LTV as a criterion, it becomes possible to make investment decisions that take into account not only the cost-effectiveness in the short term, but also future profits.

Types of advertising effectiveness measurement tools and how to choose the right one

To properly measure the effectiveness of advertising, it's crucial to choose a tool that matches the type of effect you want to measure, the scale of your campaign, and your budget. This document outlines the main tool categories and their respective suitable uses.

Category ①: Web Analytics Tools

Representative examples: Google Analytics 4 (GA4), Adobe Analytics

This tool measures user behavior on websites. It can track metrics such as traffic from ads, bounce rate, conversion rates, and page dwell time, providing a foundation for understanding the effectiveness of web advertising. GA4 is free to use, which is why many companies running web advertising campaigns have adopted it.

Suitable uses: Measuring traffic, behavior, and conversions across all web advertising platforms.

Category ②: Advertising management platform (management screen for each media platform)

Representative examples: Google Ads, Meta Ads Manager, Yahoo! Ads

Each advertising platform provides its own management screen where you can check metrics such as CTR, CPC, CVR, and CPA in real time. This is specialized for measuring effectiveness within the media where your ads are placed; cross-platform comparisons and integrated evaluations with offline advertising must be performed separately.

Suitable uses: Single-platform ad performance management and daily PDCA cycles

Category 3: Attribution Tools / App Ad Effectiveness Measurement Tools

Representative examples: AD EBiS, Adjust, AppsFlyer

This tool analyzes the contribution of multiple advertising channels to conversions. It visualizes "which ad was effective at which stage of the purchasing decision-making process," making it an important option for moving away from an over-reliance on last-click results. It is particularly gaining popularity among D2C and e-commerce brands that operate multiple channels simultaneously.

Suitable uses: Evaluating the cross-channel contribution of online advertising in multi-channel operations.

Category ④: Brand Lift Survey Tools

Representative examples: Google Brand Lift Survey, Meta Brand Lift Test, Intage, Macromill

This is a tool and service for conducting the aforementioned brand lift survey. It quantifies how much exposure to advertising has influenced purchase intent and brand awareness. It is essential for evaluating measures aimed at improving awareness and favorability, such as TV commercials and YouTube ads.

Suitable uses: Measuring the psychological effects of brand campaigns (awareness, favorability, and purchase intent)

Category 5: MMM (Marketing Mix Modeling) Tools and Services

Representative examples: XICA MMM (MAGELLAN), Intage MMM, Robyn (Meta), Google Meridian

As a method for measuring the effectiveness of advertising,MMM (Marketing Mix Consulting)This approach statistically breaks down the impact of multiple factors, such as TV commercials, web advertising, social media, seasonal fluctuations, and competitor trends, on sales. Its greatest feature is the ability to compare all measures, both online and offline, on a common axis: "contribution to sales." Provided by XICA.MMM analysis platform MAGELLANIn this case, we use a patented path analysis method to determine brand lift andAttribution AnalysisIt cannot be fully grasped.Long-term ripple effects, residual effects, and brand accumulation effectsThis can also be visualized.

Suitable uses: Integrated evaluation of multiple channels, including TV commercials, and optimization of advertising budget allocation.

The MMM approach can be broadly divided into two categories: "self-implementation of open source" and "utilization of managed services."

Open Source

Meta's Robyn and Google's Meridian are frameworks that have released MMM (Multiple Modeling Memory) as open source. Their emergence democratized access to MMM methodologies, which were previously only available to large advertisers and specialized vendors, creating an environment where more companies can experiment with MMM. They are also valued for their lack of licensing costs and high transparency of their methodologies.

However, being able to use a tool and being able to use it correctly to achieve results are two different things. Implementing and using Robyn or Meridian in-house requires the presence of data scientists. This involves designing the variables to be incorporated, collecting and preprocessing data, tuning model parameters, and statistically interpreting the output results. In addition to these, consistent expertise is required, all the way to translating this into marketing strategy decisions, such as "how to change budget allocation." The tool itself cannot make these decisions for you. It is a suitable option for companies with sufficient data science and marketing science capabilities in-house, or for use in research.

Managed services (supportive work by experts)

The practical value of MMM (Model-Based Modeling) lies in what happens after the model is run. This includes deciding which variables to incorporate into the model, how to collect, integrate, and cleanse the data, how to interpret the output results to optimize budget allocation, and how to design a continuous measurement and improvement cycle.Data Science and Marketing StrategyA consulting function with expertise in both areas is essential.

Managed MMM services provide comprehensive support, from model design and data implementation to regular reporting and improvement suggestions. For companies without in-house data scientists or those wanting to implement MMM but lacking the operational expertise, it's a practical and immediately effective option.

The MMM offered at XICA is thisConsulting-based supportThis is a typical solution.

For information on how to implement MMM and how to do it properly, please see this article:A Marketer's Guide to Modern MMM

Tool Selection Guide for Different Situations

Implementing a tool isn't a "one-times-and-done" process. Accuracy improves through continuous measurement and data accumulation. We recommend starting with a tool you can easily use and expanding your approach as your operations mature.

Four practical points to translate advertising effectiveness measurement into results

Measuring effectiveness is meaningless if it only involves collecting data. To use it for marketing decision-making, it is essential to clearly define "what to measure, why, and how to measure it."

1. Set metrics according to the type and purpose of the advertisement.

The metrics to track differ fundamentally between video ads aimed at brand awareness and search ads aimed at purchases. Measuring multiple ads as a single unit makes it difficult to see the contribution of each individual campaign. Setting KPIs for each campaign and measuring and analyzing them individually leads to better decision-making.

2. Evaluate the process with attribution analysis

Many conversions occur after exposure to multiple advertisements. Even if the last clicked ad is the deciding factor, the contributions of the preceding ads that nurtured the user's interest should not be overlooked.

Attribution analysis allows you to visualize the role of each measure in a series of touchpoints. Analysis that focuses only on online advertising tends to overlook the contributions of offline measures such as TV commercials and OOH."Integrated Attribution"Let's consider using it.

3. Regularly repeat the measurement of effectiveness.

Advertising effectiveness fluctuates depending on competitor actions, market changes, and seasonal factors. Even strategies that were successful in the past may become ineffective over time. Regularly measuring effectiveness and adjusting advertising content and channel allocation accordingly is key to sustained results.

·Related article:Three major changes in measuring advertising effectiveness

4. Don't stop at measurement; translate it into improvement actions.

The purpose of measuring advertising effectiveness is to identify "what is working well and what needs improvement" and to derive the next course of action. By continuously implementing the PDCA cycle, you can improve the quality of your advertising operations.

1. Set clear goals(Examples: improve CVR by 10%, increase cognitive lift by 15%)
2. Measure and analyze the effectiveness of each ad
3. Develop and test improvement measures to address the problem.
4. Check the cost-effectiveness again and verify the effectiveness of the improvements

Summary

It's not enough to simply run an advertising campaign; it's crucial to measure its effectiveness and continuously maximize its return on investment.

The essence of measuring advertising effectiveness lies in correctly understanding the different metrics used for each medium, grasping the true nature of the "difficulty" of measurement, and then implementing a practical cycle that leads to results.

By combining web analytics, attribution, brand lift surveys, and MMM (Marketing Management Model) according to their respective purposes, measurement tools enable a more comprehensive evaluation.

·Related article:The four pillars of measuring the effectiveness of marketing investments: tracking, experimentation, research, and modeling.

By correctly understanding the reasons why measuring advertising effectiveness is said to be difficult—such as the invisibility of offline effects, their persistence, the diversification of touchpoints, and the neglect of indirect effects—and adopting an integrated approach, measurement accuracy can be significantly improved.

In an increasingly complex advertising environment, XICA uses the power of data science to support companies struggling with the challenge of "measuring the advertising effectiveness of their entire marketing activities and maximizing their ROI."

Frequently Asked Questions: Measuring the Effectiveness of Advertisements

Q1. When should we start measuring the effectiveness of our advertising?

Ideally, you should start preparing before you even launch your ads. Recording baseline metrics (such as website traffic, brand awareness, and sales) before launching your ads allows for comparison with post-launch data. If you wait until after launch to measure, you'll lack a basis for comparison and won't be able to accurately evaluate your campaign. It's also crucial to complete the setting of key performance indicators (KPIs) before launching your ads.

Q2. Is it possible to measure advertising effectiveness even with a small budget?

Yes, you can. GA4 (Google Analytics 4) and the management screens of each advertising platform are free to use, and for web advertising, basic metrics such as clicks, conversion rates (CVR), and cost per acquisition (CPA) can be measured immediately. While brand lift surveys and the use of external research companies incur certain costs, the priority is to first establish a habit of measurement using free tools. Regardless of budget size, the basic approach of "setting goals and comparing results before and after advertising" remains the same.

Q3. Can the effectiveness of TV commercials be compared using the same criteria as web advertisements?

This is difficult with conventional measurement tools. Television commercials measure exposure effectiveness using GRP and GAP, while web advertising measures behavioral effectiveness using CTR and CVR, so the types of metrics are fundamentally different. To evaluate both side-by-side using a common criterion of "contribution to sales," it is effective to use MMM (Marketing Mix Modeling). By using MMM, the sales contribution of all measures, regardless of whether they are offline or online, can be compared using a unified metric, and this can be used to optimize budget allocation.

Q4. If the conversion rate (CVR) and return on advertising spend (ROAS) are high, can we say that the advertising was effective?

That's not always the case. While CVR and ROAS are metrics that measure behavioral and sales effects, advertising effectiveness also includes exposure effects (reach) and psychological effects (changes in awareness and favorability). For example, if you evaluate TV commercials or social media video ads aimed at gaining awareness using CVR, there is a risk of misjudging them as "ineffective" because the objective and the metric are mismatched. Using metrics that are appropriate to the advertising objective is a prerequisite for correct evaluation.

Q5. Can anyone conduct a brand lift survey?

Advertising platforms like Google and Meta offer free brand lift surveys for campaigns that meet a certain delivery volume (number of impressions). Therefore, it is relatively easy for companies that operate digital advertising on a certain scale to conduct these surveys. On the other hand, for integrated surveys that include offline measures such as TV commercials and OOH, it is necessary to commission an external research company.

Q6. How will advertising effectiveness measurement change in a cookie-less environment?

Multi-touch attribution (MTA) and some retargeting measures that heavily rely on third-party cookies will inevitably suffer from reduced accuracy. On the other hand, marketing mix modeling (MMM), which uses aggregated data,Cookie-independentTherefore, the effectiveness of all channels can be evaluated without being affected even in a cookie-less environment. The same applies to brand lift surveys, as they are also survey-based. In the future, it is expected that there will be a shift from "individual tracking" to "aggregate and statistical" measurement methods.

Q7. How often should advertising effectiveness be measured?

For digital advertising, daily and weekly monitoring is standard practice. However, overreacting to short-term fluctuations in numbers and drastically changing strategies can make it impossible to verify their effectiveness. Strategic decisions (such as reviewing budget allocation or changing channels) should be made on a monthly or quarterly basis, while daily and weekly monitoring should be limited to detecting anomalies and making small improvements. Strategies that yield results over the medium to long term, such as TV commercials and OOH (out-of-home) advertising, require evaluation over a longer period.

Author information

XICA Co., Ltd. Marketing Team