What is bias? Its meaning and types [including points to note when analyzing data]
Bias means "bias," "prejudice," "preconception," etc.Distorted or biased perceptionIt is used as a word to express.
Bias often comes into play in everyday business situations and in data analysis. If bias is not recognized and corrected, it can lead to illogical decision-making, not deriving the full value of data, and many other problems.
There are many types of bias. In this article,10 types of bias that business people should be aware of on a daily basisWhen,Four types of biases that people new to data analysis should be especially aware ofI will explain it as simply and clearly as possible.
Business people who want to use data as a weapon should aim for correct data analysis while being aware of the biases explained in this article.
table of contents
What is bias?
Bias is a Japanese word that means bias, prejudice, and preconception.
For example, in English-speaking countries, the word bias is used daily, such as in the phrase "I might be biased but I think..."
In the context of statistics or psychology, "some kind of bias" is also called bias.
Differences between statistical and psychological biases
- Statistics: Bias that occurs in data populations, samples, and hypotheses made from analysis results
- Psychology: Biased opinions about people and things that arise from personal experiences and societal conventions
In statistics, more accurate suggestions can be derived by analyzing data while eliminating bias as much as possible.
The difference between stereotypes and bias
A stereotype is "a fixed idea, image, preconception, concept, or pattern of thought that is pervasive among many people."
Although this may seem similar to bias in psychology, there is a clear difference between bias and stereotypes.
The difference between bias and stereotypes isWhat is a stereotype? A clear explanation of the difference between stereotypes and biases, as well as their advantages and disadvantages' is explained in detail in.
If you read this article together with the previous one, you will gain a deeper understanding of the bias that can easily occur in data analysis, so be sure to check it out.
Advantages and disadvantages of bias
The words "bias" and "prejudice" tend to emphasize the negative impression of bias. However, bias has both advantages and disadvantages.
Benefits of Bias
- Compressing the amount of information the brain processes to make quick decisions
- By recognizing the various biases, it is possible to conduct data analysis that eliminates unconscious bias.
Disadvantages of Bias
- By simplifying people and things, we miss their essential characteristics and properties.
- Accurate analysis cannot be performed due to bias in collected data and hypotheses
10 types of bias that business people should remember
In this section, we will introduce 10 types of biases that business people should be aware of in their daily lives. If these biases are not recognized and corrected, they can lead to irrational decision-making.
Be careful, as bias can intrude into any situation, including small everyday decisions and personnel evaluations.
Additionally, this is a bias that you should be aware of when conducting data analysis, so please read on while thinking about what kind of biases you have (or are prone to have).
1. Cognitive biases
Bias caused by preconceptions formed from past experiences and intuitionThese are called "cognitive biases." These biases involve preconceptions and intuition, and can easily lead to irrational decision-making.
Moreover, because this bias is likely to occur in everyday life, it is one that requires caution in all business situations.
For more information on cognitive biases, seeThe Importance of Understanding Cognitive Bias in Marketing: Consumer Insights Beyond Data"Please refer to the.
2. Confirmation Bias
Confirmation bias is one type of cognitive bias.
When people judge whether a hypothesis they have made or a hypothesis given to them by others is correct,Focus on and emphasize information that confirms a hypothesis, rather than information that disproves it.There is a trend.
This is called "confirmation bias."
Simply put, it is the psychology of "focusing only on information that is convenient for you."
3. Authority Bias
The authority of a person or organization influences people, leading them to blindly believe the information they give out.This is called "authority bias."
In the field of data analysis, there are cases where people blindly believe the data, thinking, "The data is reliable because it was published by that well-known research organization."
Just because a data is authoritative does not mean it is correct, and it is essential to verify any data beforehand.
4. Normalcy Bias
When data occurs that exceeds a predefined threshold,The bias that makes us think, "It's okay, this is within the normal range"This is called "normalcy bias."
For example, it has been pointed out that in past disasters, "there have been cases where normalcy bias rather than panic led to people being late in escaping."
Even in the field of data analysis, there are many cases where inconvenient data is ignored or underestimated.
5. Scarcity Bias
The bias towards things that are hard to obtainThis is called "scarcity bias." Many people are vulnerable to phrases such as "only one left" and "limited time only" because of the scarcity bias.
In data analysis, there are cases where people assume that "because the data was obtained with great difficulty, it must be highly valuable."
But the value of data isn't determined by how difficult it is to obtain.
6. In-group bias
Bias caused by peer sentimentThis is called "in-group bias," or what we call "favoritism toward one's own people."
In-group bias is not just a bias directed at one's group and its members, but can also arise from small ties such as being from the same hometown.
7. Hindsight Bias
When the results come out,The bias of thinking, "I knew it would turn out like this"This is called "hindsight bias."
Hindsight bias itself is common and doesn't have any direct impact on business or data analysis.
However, people who have hindsight bias may also have other biases, so it is important to be aware of the existence of biases when analyzing data.
8. Self-Serving Bias
The bias of blaming oneself for success and others or the environment for failureThis is called the "self-serving bias."
For example, if you blame the results of a failed data analysis on the data or the surrounding environment, self-serving bias is at work. Data analysis is not something you do once and then do. It is rather rare to get correct analysis results from just one data analysis. Therefore, it is essential for correct data analysis to continue to turn the PDCA cycle and improve the accuracy of the actions you derive.
Even if your data analysis does not go well, it is important to avoid self-serving bias and identify areas for improvement from an objective perspective, asking yourself, "Was there a problem with me, such as the way I collected or analyzed the data?"
Also, be careful not to overestimate yourself just because you have been successful in data analysis, as this can cause you to overlook important factors for success.
9. Actor-observer bias
When a mistake or problem occurs,A bias in which we consider external factors when it comes to our own behavior, but only focus on internal factors when it comes to the behavior of othersThis is called the actor-observer bias.
To put it simply, it is the psychology of thinking, "My mistakes are the fault of the environment, and others' mistakes are the fault of the individual."
If you fall into this bias of being lenient on yourself and harsh on others, it can cause problems not only in data analysis but also in interpersonal relationships.
10. Unconscious bias
Unconsciously formed prejudices and prejudicesThis is called "unconscious bias."
When analyzing data, unconscious preconceptions such as "this is what we usually do" or "this analytical method is appropriate in this case" can lead to misguided analysis.
Unconscious bias is also a social problem that occurs in all business situations.What is unconscious bias? Learn about factors and how to improve them through specific examples, and the difference from bias', so please take a look.
Bias to be aware of when dealing with data
From here, I will explain four biases that you should be careful of, especially when working with data.
In addition, as background knowledge, we will explain the relationship between populations and samples, and systematic error and random error. These background knowledge are very important in eliminating bias in data analysis.
Relationship between population and sample
A population is the group (collection of data) that is the subject of data analysis.
For example, to determine the accurate average annual income of business people living in Japan, it is necessary to collect annual income data from the population of approximately 6,723 million employed people in Japan (*1).
By calculating the average value from all the annual income data, we can observe the population. In data analysis, this is called"Descriptive Statistics"called.
However, collecting annual income data for all of Japan's approximately 6,723 million employed people would require enormous cost and time.
Since this is not a realistic analytical method, a method is often used in which a portion of the population is extracted and analyzed to clarify the characteristics of the population."Inferential Statistics"called.
When extracting a sample from a population, various biases can occur, and there is a risk of obtaining incorrect analysis results. Therefore, it is very important to consider "what criteria should be used to extract the sample?".
(*1) *Quoted from the 2022 Labor Force Survey https://www.stat.go.jp/data/roudou/
Systematic and random errors
Data analysis is always accompanied by "data errors." Data errors can be broadly divided into two types: systematic errors and random errors.
Systematic errorThis refers to the "difference in analytical results" that arise due to differences in the data collected, the way it is processed, and the method of analysis.
For example, when investigating the average annual income of business people, the results will differ depending on whether the survey is conducted across the whole of Japan or by prefecture. In this way, systematic errors arise depending on the conditions of data analysis.
Random errorThis refers to the "variability in analytical results" that occurs when data analysis is performed under the same conditions.
For example, suppose you want to research the average annual income of business people living in Tokyo. Even if you extract the same number of samples (sample data) from the same population and perform data analysis, the results will vary each time you analyze it. This is because the characteristics of each sample extracted from the population are different (in this case, the annual income).
In order to obtain accurate results from data analysis, both systematic and random errors must be taken into consideration from the planning stage.
1. Survivorship bias
A common bias when obtaining data analysis results: "Only the surviving data is emphasized".
In data analysis, only the collected (selected) data will be analyzed; discarded data will not be analyzed.
This may seem like a very obvious thing, but the analytical results obtained from collected data are not necessarily correct.
If you do not analyze data while constantly asking yourself, "Is the truth hidden in the discarded data?", you will end up making decisions based only on superficial facts.
2. Applicant bias
Bias caused by personal choice in sample selection.
For example, suppose you obtain annual income data (sample) for 1,000 people to find out the average annual income of business people in Tokyo.
The 1,000 people may be a sample taken from business people with relatively high annual incomes who would not be embarrassed to disclose their annual income.
Because volunteers (participants) for data analysis have a strong interest in the research topic, bias in the results of the analysis is often introduced.
3. Sampling bias
Bias caused by limiting data analysis to a specific sample.
For example, if you extract annual income data (sample) from 1,000 full-time employees to find out the average annual income of people living in Tokyo, you will not obtain accurate analytical results.
The result is simply the "average annual income of regular employees living in Tokyo."
There is no problem if the purpose is to find out the "average annual income of regular employees living in Tokyo," but if you want to find out the "average annual income of people living in Tokyo," you will also need to collect income data on non-regular employees.
4. Algorithm bias
Bias can occur in machine learning algorithms due to biased data being fed to AI or due to developer bias..
However, it is ultimately humans who provide biased data and develop biased programs. The algorithm itself is not bad.
If you see algorithmic bias, there’s a good chance the developers themselves have some bias.
Beware of "Misleading bias" when measuring results
In addition to the four types of bias mentioned above, business people involved in data analysis should be aware of "misleading due to bias."
This often happens when measuring the effectiveness of digital initiatives such as marketing or business improvement.
Compared to non-digital measures, digital measures allow for more precise targeting and richer data collection. In other words,Digital measures are more likely to be influenced by bias
For example, in the case of TV commercials, sponsored programs are decided based on broad targets such as gender, age, region, etc. On the other hand, YouTube ads allow for more precise targeting, such as gender, age, interests, and life events.
The more detailed the targeting options, the greater the opportunity for marketer bias to come into play.In addition, when measuring the effectiveness,It is easy to introduce bias when deciding which data to analyze.
Problems like this, where bias leads to implementing the wrong digital initiatives or measuring their effectiveness incorrectly, are called "misleading due to bias."
In conclusion
Bias exists everywhere in the business world and can lead to various misunderstandings.
In addition, when analyzing data,You must be conscious of "eliminating bias" from start to finish of the data analysis process, including determining the population, sampling the sample, data processing methods, analytical methods, and hypothesis formulation.
It is no exaggeration to say that people live their lives ruled by bias.
First, recognize what biases you have or are prone to have. Then, think about what kind of thinking and action process you need to eliminate those biases.
