Introduction to Data Modeling | Difficult technical terms explained in an easy-to-understand way

Update date:2024.05.15 Column

In addition to the three major management resources of people, things, and money,"Data (information) is the fourth important resource."This recognition has spread widely, and many companies are now aiming to achieve data-driven management.

However, while many companies understand the value of data as a resource, not many are aware of the importance of data modeling.

Therefore, in this article, we will explain data modeling in an easy-to-understand manner, while simplifying difficult technical terms, as an "Introduction to Data Modeling." Data modeling is a concept and task that is essential for correct data analysis and data organization, so please take a look at this article.

What is Data Modeling?

The data held by a company is scattered throughout the organization. For example, the human resources department has employee information and associated personnel evaluation data, while the sales department has customer information and case data. All of these are important resources for a company.

Resources are "materials that serve as the basis of productive activities," and it is easy to understand if you describe them as "data = oil."

Since petroleum cannot be used as it is, it must be refined and converted into fuels such as gasoline, or into plastic, and consumed as a daily commodity.

The same goes for data. Simply collecting data from each department is not enough. Just like refining oil, data needs to be cleansed, processed, and then analyzed before it can have value.

However, in analyzing the data,"Data modeling" is important before collecting and cleansing data..

Modeling = visualization, diagramming

In Japanese, the word "model" means "example, model." In other words, modeling means "to create (an example or model)."

It depends on what you are modeling. In the world of graphic design, modeling refers to creating a 3D design.

So, what does modeling involve in the world of data analysis? It involves visualizing and diagramming the type and amount of data stored in each database, as well as the relationships between databases.

Modeling the Data

The term "data modeling" may sound difficult, but as mentioned above,The task of visualizing and diagramming the data held in each database and the relationships between them.."Create a blueprint for your data"There's no problem with thinking of it in a simple way.

A blueprint is a drawing used as a reference for a design or architecture. It is made up of a myriad of straight lines and curves, and also contains various information necessary for the design or architecture.

Data modeling creates blueprints in a similar way: they are used to help build a database by showing lines, curves, and various pieces of information.

By the way, the database design completed through data modeling is called"Data Architecture (Data Structure)"called.

Why is data modeling important?

The ratio of preparation to work in a job is sometimes expressed as "Dandori 8% (preparation is 2%)." The same can be said for data analysis, with data analysis making up the remaining XNUMX%.

On the other hand, data modeling is part of the "8% of preparation" and is essential for correct data analysis. Here are five reasons why it is specifically important.

To efficiently collect, store and process data

As mentioned earlier, data is scattered throughout an organization.

Unless you have already established an integrated database such as an ERP system, you will need to collect, store, and process the necessary data from multiple databases in order to perform data analysis.

To make these tasks more efficient, data modeling is required to clarify the location, type, amount of data, relationship with the database, and even the processing process.

To improve the accuracy and efficiency of database development

Databases that store the vast amounts of data generated by companies must be reliable. An unreliable database not only impedes operational efficiency but can also cause disruption to your business.

To develop a reliable database, it is necessary to ensure the accuracy of the work from the development stage onwards. However, since we cannot afford infinite time and cost, efficiency is also necessary in addition to accuracy.

In order to ensure and improve the accuracy and efficiency of database development, data modeling, which is a preliminary step, is essential.

To facilitate smooth communication between development and operations

The database that forms the basis of data-driven management is regularly improved and continuously operated.

Since development and operations are typically handled by different teams, a data architecture that serves as a common understanding between the two is required.

If data architecture can facilitate communication between development and operations, the role of the database and processing processes can be flexibly changed in response to changes in the business.

To ensure consistency of system design across the organization

Data modeling ensures consistency in data architecture and system design across your organization.

Conversely, companies that lack consistency and have fragmented data architectures and systems can experience fatal problems for their business, including processing delays, business slowdowns, and misguided data analysis.

A few years ago, maintaining consistency in system design and building integrated databases were dismissed as "IT industry trends." However, today, they are essential issues to keep up with the ever-increasing pace of business change.

To achieve a sustainable database

A "sustainable database" is one that can continue to be optimized while ensuring reliability, even as the team members involved in development, operations, and data analysis change or increase in number.

This cannot be achieved by relying on system integrators to build databases as in the past because database architecture and related technologies have become black boxes. This is a natural business strategy for system integrators to prevent customers from leaving, but as a result, many companies are struggling with this black box nature.

In order to build a sustainable database, it is necessary to re-implement your company's data modeling and reconsider the design of your entire system, including the database.

Types of Data Modeling

There are three stages in data modeling: conceptual data model, logical data model, and physical data model. Let's explain them one by one.

Conceptual Data Model

A conceptual data model is a data modeling method to represent the "whole picture of data". Each element that makes up the whole picture of data is called"entity"called.

You may feel a little confused because of all the technical terms that suddenly pop up,"Entity = major classification of data"The organization of these entities and the modeling of their relationships is the conceptual data model.

^{Conceptual data model = A model of the major categories of data and their relationships}

Logical Data Model

The logical data model further organizes the entities organized in the conceptual data model."attribute"called.

In actual data modeling, entities, their relationships, and attributes are modeled in more detail.

^{Logical data model = A more detailed model of entities, their relationships, and their attributes}

Physical Data Model

In the physical data model, we will incorporate the entities and attributes we have organized so far into the database design. More specifically, we will define how to manage the data organized above in the database tables.

In databases, attribute names cannot be registered in Japanese, such as "customer code." Therefore, it is common to convert them to English, such as "customer_id."

^{Physical database = A definition of how data organized in a logical database is managed in database tables}

The physical data model also includes organizing which data will be managed in which database, and how the systems will be connected.

Steps for Data Modeling

When performing data modeling, it is a reliable and efficient method to perform it using commonly known procedures. Here, we will explain the procedure for performing data modeling in a simple manner.

1. Representing entities and their relationships

The first step is to represent the entities and their relationships. This corresponds to the "conceptual data model" mentioned earlier."ER Diagram (Entity Relationship Diagram)"For example, an ER diagram can be created as follows:

When creating an ER diagram,"IE notation"or sending us a message on"IDEF1X notation"It is common to follow these rules (notational differences will be omitted):

IE notation

IDEF1X notation

The reason for following one of the rules is to ensure that the data architecture is shared among all parties involved. If the rules for creating ER diagrams are inconsistent, not only will work efficiency decrease, but there is also a risk of misunderstandings leading to incorrect database design.

2. Create an attribute list

Next, create an attribute list for each entity. This is the step to organize the attributes in the "logical data model" mentioned above.

When creating an attribute list, you should keep in mind the following:"MECE (Mutually, Exclusive, Collectively, Exhaustive)"In Japanese, it means "no omissions, no duplications," and is something that is always kept in mind in the field where data needs to be organized.

If there are gaps or overlaps in the attributes of each entity, problems such as the database not functioning properly, the system not fulfilling its purpose, and inability to perform sufficient data analysis can occur.

It's a delicate process and takes a lot of time,Creating an attribute list with MECE in mind is an essential preparation for a reliable database.

3. Decide on a data modeling approach

There are three main approaches to data modeling:

Relational Data Modeling	Although it is an old modeling technique, it is still commonly used due to its simple description. It is suitable for modeling structured data.
Hierarchical Data Modeling	This is a modeling technique that describes the relationships between entities in a tree structure. Parent entities and child entities have a "1:n" relationship, and is used when you want to speed up specific database processing.
Graph Data Modeling	This is a modeling method that allows you to describe "1 to n" or "n to n" even if the entity does not have the concept of parent and child. It is also used in AI development and machine learning.

It is important to choose the appropriate method depending on the purpose of data modeling and the business goals you want to achieve through data.

4. Create a physical data model

Finally, create a physical data model. However, data modeling does not end with creating a physical data model.

As mentioned above, the database that forms the basis of data-driven management requires regular improvement and continuous operation. In other words, physical data models require constant optimization.

Just because you go through a lengthy process to create a physical data model doesn't mean the resulting data architecture is the right one.

To use data to achieve your business goals,Regular optimization is required to keep the physical data model close to the correct answer and keep improving it.

So what is data modeling?

So far, we have explained the overview, importance, types, and steps of data modeling. Some business people who will be involved in data analysis in the future may still be wondering, "What exactly is data modeling?"

But it's not that hard. After all, data modeling is"Designing a database".

The technical aspects of data modeling are the domain of data scientists and database engineers. However, if business people involved in data analysis are aware of the basic knowledge mentioned above, they will be able to communicate more smoothly and build databases that lead to better business results.

While reflecting on your daily work or imagining the data you need, work with data scientists and database engineers to create a good data architecture.

From Data Modeling to Business Strategy: Practical Applications of Statistical and Predictive Models

We have understood the basics of data modeling and seen the importance of structuring data effectively, but this is not just theoretical; it has practical applications in various business areas, especially marketing.

To dig deeper into how data modeling can support an effective marketing strategy,List of statistical models in marketingとPredictive Models in MarketingBe sure to also read the article about.

In these articles, you’ll learn about the specific statistical models used in marketing analytics and how predictive modeling can enhance your marketing efforts by forecasting future trends and customer behavior.