In the previous report (※1), we discussed the importance of "data collection." In this report, we will discuss how to select an analysis method to visualize marketing effects.

As the data that companies can obtain becomes more diverse, the frameworks for analytical methods are also becoming more diverse. There are probably many companies that are unsure of the optimal analytical method at this time, and as a result are unable to take the plunge into data-driven decision-making. This report will focus on what analytical methods are generally available for visualizing marketing effectiveness, and what should be considered when selecting one.

(※1) Previous report:Best practices for data collection: What we learned from working with clients

Process for selecting analytical methods

If the selection of an analysis method is not done through the correct process, even if you have put a lot of time and effort into obtaining analysis results, you will likely find yourself in a situation where you are left wondering "What did you want to know?" or "I can't put it to use..." It's something you've heard so many times, but before rushing to adopt a trendy analysis method or blindly introducing analysis tools, as mentioned in the previous report (※1), you should first clarify the purpose of the analysis and then select an analysis method.

Below is the "data collection process flow" that we recommend, which includes the process of ① reaching a consensus on the analysis purpose and then ② designing a model that matches the analysis purpose.

Even when choosing an analysis method that matches the analysis purpose, it is difficult to choose the best analysis method based on that alone. In addition to the above, we will now discuss other constraints that should be considered.

Constraints on analytical method selection

When selecting a specific analysis method, there are three constraints to consider:

1. Man-hour constraints

The accuracy of data analysis results is basically proportional to the amount of work required. To improve accuracy, it is necessary to either expand the range of factors (variables) related to the analysis model, or to steadily adjust the analysis model by checking the various parameters and results calculated from the analysis model.

From a business perspective, delivery time is also an important consideration. Since you need to consider the cost-effectiveness in addition to the schedule and the amount of work you can put into it, select a realistic analysis model after estimating the working hours of the project members.

2. Constraints due to analytical know-how (skills)

Some data analysis methods can be easily carried out in Excel, while others require highly specialized analysis methods (requiring programming, mathematical knowledge, etc.). It will be difficult for project members to choose an analysis method they have no experience with. As with man-hour constraints, try to choose a realistic method based on the knowledge of project members and your company.

3. Constraints due to data collection availability

When identifying factors (variables) related to the analytical model, it may not be possible to collect data essential to the constructed analytical model, and the agreed-upon analytical objectives may not be achieved. When selecting an analytical method, it is more efficient to proceed while considering whether important data related to the analytical model based on it can be obtained. If this is not possible, it is necessary to select an analytical method that can proceed without obtaining the relevant data, or to return to the phase of agreeing on the analytical objectives. In particular, regulations on cookie data have recently become stricter from the perspective of personal information protection, so caution is required when discussing the possibility of collecting data.

If it is essential to overcome the above constraints in order to achieve the agreed upon original analysis objectives, you can turn to an external data analysis specialist. In some cases, constraints due to labor hours and analytical know-how can be avoided. In addition, if the data is not first-party data, it may be possible to obtain it by utilizing external resources, so relying on a service that collects specific data is also an option.

Furthermore, these are things that must be taken into consideration when proceeding with any analysis project, not just when conducting marketing effectiveness visualization analysis.

Analysis methods for visualizing marketing effects

There are a wide variety of analytical methods related to marketing. Therefore, this time we will focus on the visualization of marketing effects and discuss the use cases and features of each analytical method. Please note that the following are general methods for visualizing marketing effects and do not cover all methods.

1. A/B Testing

• Man-hours: ★　
• Skills: ★　
• Difficulty of data collection: ★

*The more stars there are, the more restrictions and difficulty there are. Same below.

It is the most commonly used, simple and powerful method for visualizing marketing effects. It is a method to visualize the effects by looking at the difference between subjects who have undergone a certain marketing activity (intervention group) and those who have not (control group). If you pay attention to making the intervention group and control group as homogeneous as possible and to ensuring a certain number of samples, you can quickly go through the PDCA cycle. An easy example to imagine would be changing the page design of an organic site. By looking at the difference between users who have changed the page design (intervention group) and those who have not (non-intervention group), you can determine whether the change in page design was effective.

Whether or not a specific intervention has produced results is generally determined by conducting hypothesis testing. However, A/B testing is not an all-purpose method, and it has a crucial weakness in visualizing the effects of marketing activities, in that it only targets marketing activities that have personal data and can be carried out experimentally.

For example, it would be difficult to use A/B testing to measure the effectiveness of marketing activities that cannot track personal data, such as the effectiveness of TV commercials. In addition, major changes in marketing strategies, specifically the impact of new product launches and price changes, require continued tracing of some customers, and are difficult to apply to only some customers, making these areas unsuitable for A/B testing.

2. DID (Difference of Differences)

• Man-hours: ★★
• Skills: ★★
• Difficulty of data collection: ★

As an extension of A/B testing, there is also something called DID (difference in differences). This compares the difference in sales between an intervention group and a non-intervention group before and after implementing a marketing measure, and visualizes the effects of implementing a marketing measure. If you run an experimental TV commercial in separate areas, you can visualize the effects of a marketing measure without tracking the personal data mentioned earlier.

On the other hand, it is necessary to make the assumption that time-series trends will not change except when the marketing measures being surveyed are implemented, and it is difficult to find cases that meet the condition that "time-series trends will not change except when the marketing measures being surveyed are implemented." To do this rigorously, it is necessary to use specialized skills to design the experiment, and it is not something that can be done easily.

3. MTA (Multi-Touch Attribution)

• Man-hours: ★
• Skills: ★
• Difficulty of data collection: ★★

This is also a standard method now. It is a method that uses cookies and pixel technology to track customer behavior in real time. Since data can be obtained and checked in very fine units and in real time, there is a lot of useful information just by looking at the data. For example, if sales are used as a conversion indicator, ROAS can be measured without the need for specific analysis, making it a very powerful tool for visualizing marketing effectiveness.

However, since "personal data" is the key to MTA, it is difficult to visualize the effectiveness of TV commercials or clarify the impact of new product launches and price changes, as mentioned earlier. In addition, regulations on cookies have become strict recently, and some data may be lost. Another disadvantage is that it is necessary to introduce tools, which incurs a certain amount of cost.

4. Regression Analysis

• Man-hours: ★★
• Skills: ★★
• Difficulty of data collection: ★★

This is also a method that has been often used to visualize marketing effects. It is often used in such a way that sales data is set as the objective variable, and data related to marketing activities is used as the explanatory variables to perform regression analysis to estimate how much a certain marketing activity contributed to sales. When there is one explanatory variable, it is called simple regression analysis, and when there are multiple explanatory variables, it is called multiple regression analysis. Although a certain level of statistical knowledge is required, it can be carried out in Excel, so it is a relatively easy method.

Unlike the A/B testing and MTA mentioned above, it is possible to track fluctuations over time without requiring personal data, so it can also be used to measure the effectiveness of TV commercials and visualize the impact of changes in marketing strategies.

On the other hand, it is necessary to be careful of "spurious regressions," especially in regression analysis of time-series data. Even if the parameters statistically calculated from the analysis model show that "marketing activities are effective," this may be a "spurious effect" produced by the characteristics of the time-series data.

　In addition, multiple regression analysis can have problems with multicollinearity, and the pursuit of high accuracy requires specialized knowledge, such as securing the necessary amount of data.

Free downloads of related materials

A guide to multiple regression analysis in Excel that empowers marketers
~ Understand the correlation between marketing measures and business results ~

Download Now

5. MMM (Marketing Mix Modeling)

This analysis makes use of various statistical methods such as the regression analysis mentioned above to visualize how each factor (including marketing measures and external factors) influences the final KPI. Because it does not require personal data, it is possible to comprehensively track the effectiveness of TV commercials, the impact of changes in marketing strategies, and the impact of external factors (economic fluctuations and changes in market trends). For marketing measures for which data can be obtained, it is possible to comprehensively visualize the impact of external factors and the impact of each advertisement on each other, making it an extremely powerful analysis method that can also be applied to comprehensive budget allocation and upstream marketing strategy decisions.

On the other hand, since it is necessary to build an analytical model that correctly reflects marketing hypotheses, in addition to statistical knowledge that can process multiple models, a wide range of skills are required, including statistical knowledge, industry knowledge, and marketing knowledge. Also, since it is a very large-scale method, it is not cost-effective to follow small fluctuations, so for example, it is often recommended to measure the effect of changing the page design of an organic site with A/B testing rather than MMM.

The analysis methods mentioned so far are only a part of the methods for visualizing marketing effects, but they are powerful and versatile. The appropriate means of visualizing effects will vary depending on the purpose of the analysis and the marketing measures in question, so select an analysis method that suits the required effort and characteristics.

The media mix model is a statistical analysis method to evaluate how to allocate advertising budgets to multiple advertising media. In marketing, it is important to effectively combine multiple advertising media. The media mix model allows you to evaluate the contribution of each advertising medium and determine the allocation of advertising budgets that maximizes advertising effectiveness.

Free download of case study

What is "MMM" that all marketers should know about?
~ Benefits of Implementation: Insights from Three Company Case Studies ~

Download Now

Summary

By establishing a system for visualizing marketing effects, you can turn a very powerful PDCA cycle. In order to achieve higher-level goals, you should select the best analytical method based on a thorough understanding of the points that need to be considered.

• When selecting an analysis method, it is a good idea to consider the purpose and three constraints (manpower, skills, and data) before making your selection.
• There are many different analytical methods for visualizing marketing effectiveness, but there are five common ones:
  • A / B test
  • DID (Difference of Differences)
  • MTA (Multi-Touch Attribution)
  • regression analysis
  • MMM (Marketing Mix Modeling)
• By establishing a system for visualizing marketing effects, you will be able to achieve higher-level goals.

Integrated on-off analysis allows you to continuously understand acquisition efficiency and carry out the most effective marketing operations.

From next time onwards, we plan to explain data analysis methods for visualizing specific marketing effects.

XICA's MMM Solution

Our company, XICA, has been providing consulting and services in the field of data science in marketing for over 10 years. In particular, we have a track record of supporting over 250 companies, mainly large corporations, through our in-house developed MMM solution (MAGELLAN).

Our specialized data scientists and consultants will support our clients in making optimal use of MMM in line with their needs and objectives, and in maximizing their marketing effectiveness.

Please take this opportunity to learn more about the various industries that use MAGELLAN.Interviews with companies that use our servicePlease also read: