What is context engineering? Why generative AI answers become mediocre: The difference between context design and prompting.

"We want to use generative AI in our company's strategy, but it's not working as well as we hoped."
"The answers are all bland and unhelpful; they're useless for decision-making."
"No matter how much you refine the prompt, all you'll get back is generalities."

When you feel that frustration, you might be tempted to suspect that there's a problem with the "prompt" (how you give instructions). However, that might not be the real cause. In fact, it's highly likely that the "context" you're providing to the AI ​​is insufficient.

Why can't AI offer suggestions that truly align with a company's strategy? It's because AI isn't given sufficient context in the following two areas:

• Internal context:Brand guidelines, past success stories, meeting minutes, proprietary company knowledge, etc.
• External context:Real customer voices, the latest survey results, and market realities.

Using AI without conveying these points is equivalent to asking an external consultant, who lacks sufficient information, to suddenly come up with a "game-changing strategy." Without context, even the most brilliant AI cannot produce a strategy optimized for your company.

Therefore, this article will provide a clear explanation of "context engineering," a way of thinking that goes beyond what can be achieved through prompt optimization alone.

What is Context Engineering? From "Instructions" to "Creating the Right Environment"

In short, context engineering is an approach to designing and managing the "information environment itself" (data, rules, history, etc. that the AI ​​references) that is provided to AI (especially LLM/Large-Scale Language Models).

Rather than being an entirely new concept, it's a way of thinking that was articulated and organized in 2025, and it can be described as an extension of prompt engineering to design not only the input text but also the entire information surrounding the AI.

Let's summarize the differences between the two.

The biggest difference lies in the "approach." Prompt engineering focuses on "individual input skills," while context engineering targets the "data environment itself" (what to show the AI ​​and what to have it remember), such as background information.

Of course, these are not contradictory. Excellent prompting is incorporated as part of context engineering. The important thing is not to leave the skill of listening to individuals, but to establish organizational frameworks for thinking and decision-making. This is what creates sustainable value in the business world.

Why is context engineering necessary now? 3 benefits

As the use of generative AI expands, the performance differences between the models themselves are rapidly narrowing. As model performance becomes commoditized, the difference is shifting to "what data is provided and how."

The value of an organization engaging in context engineering goes beyond simply improving operational efficiency. It's also a process of developing AI into a "trusted partner" and enhancing the organization's competitiveness.

1. Proprietary data becomes a competitive advantage.

The key to differentiation lies in providing AI with "unique company information (knowledge)" that is not available on the internet.

For example, in a marketing setting, data obtained from CRM, purchase history, customer feedback, and effectiveness measurement analyses such as MMM (Marketing Mix Modeling) are not merely reports, but become "decision-making data" for AI.

Providing this context transforms textbook frameworks and generalities into usable strategies tailored to your company's specific circumstances. While excellent prompts may be copied, assets accumulated over many years cannot be replicated.

2. Towards an organization that does not rely on "prompt specialists"

A situation where "we can't use AI effectively without that person" is a risk for an organization. By setting up the environment through context engineering, anyone can obtain the same quality of answers.

Furthermore, the information accumulated as context remains as an asset of the organization. Even if an employee is transferred or leaves the company, the "work context" they possessed is less likely to be lost, which is a secondary effect that should not be overlooked.

3. Prevent AI from pretending to know everything.

The phenomenon in which AI generates information that is not factual is called hallucination, and its main causes are a lack of information for judgment and inconsistencies in the information. Providing AI with "reliable primary information" as context is the most effective way to prevent errors in AI inferences and increase reliability.

Realistic challenges you should know before you start

Context engineering is a powerful approach, but implementation doesn't always go according to theory. This article explains the practical challenges you might face during implementation and operation, as well as the realistic points you should understand beforehand.

If the data quality is poor, the accuracy of the AI's responses will also decrease.

No matter how high-performance a model is, if the given context is of low quality, the output will also be of low quality. As the principle of "Garbage In, Garbage Out" suggests, data organization is essential from the start.

In particular, the following three cases require special attention.

• Information inconsistencies:For example, what would happen if you gave the AI ​​both a manual from three years ago and the latest manual? The AI ​​might get confused about which one to consider correct and end up mixing up the information to give an answer.
• Raw data:"Unreadable data," such as scanned PDFs or handwritten notes, contains a lot of noise for AI. It's necessary to convert this data into text and format it in a way that AI can understand.
• Missing context:If you feed an AI meeting minutes that use company-specific abbreviations (e.g., "PJT-X") without definitions, it may not understand the context and may make inaccurate interpretations.

This isn't a project that ends with implementation; it's a project that we nurture together.

Contextual engineering is not something that is completed simply by implementing a system. Rather, it is a long-term project, similar to R&D (research and development), that tackles problems for which there are no easy answers.

Therefore, continuous investment in the following "invisible costs" is necessary.

• Search cost (effort involved in trial and error):There is no single correct way to provide context. To find out which materials to provide to the AI, how long they should be, and in what format to get the best answer, you need to repeatedly formulate hypotheses and test them.
• Evolution cost (continuous improvement):Maintenance is essential to keep the system constantly evolving, including tuning based on feedback from the field, coordinating with ever-changing internal information, and frequently updating the AI ​​model.
• Human resource costs (investment in high-level decision-making skills):Running these processes requires personnel with a deep understanding of both the "business context" and the "characteristics of AI." Developing and hiring experts capable of making sophisticated judgments in data selection and design, as well as investing in reliable external partners, are crucial factors that determine the success or failure of a project.

Understanding AI's strengths and weaknesses and having appropriate expectations.

AI technology is evolving at an astonishing pace, but it also has structural limitations at present. Understanding these limitations is crucial for designing what tasks AI should handle and what roles humans should take on.

For example, while AI can provide excellent information and options, it cannot assume responsibility for the final decision. In particular, in the following situations, a human must always make the final decision:

• Responsibility for decision-makingAI can present compelling "options," but the responsibility for decisions that require accountability to stakeholders, as well as for final decisions that involve legal and ethical risks, always rests with humans.
• Data governanceHandling personal information (names, addresses, purchase history) and confidential information (employee personnel data and contract details with business partners) always requires caution. Information that should not be shown in the human world should not be shown via AI either.
• The boundaries of creativityAI is currently good at "combining" and "extending" existing data, but unprecedented, innovative ideas and intuitive thinking remain in the realm of humans.

Practical Guide: Contextual Engineering in 4 Steps

So, how can we implement context engineering in practical work?

For example, when you commission work from a consultant, you would likely conduct a briefing beforehand. This would include the background, objectives, constraints, reference materials, etc. By organizing and providing this information in advance, you can receive truly useful proposals. The approach to AI is the same.

Elements of context engineering

As proposed by Philipp Schmid of Google DeepMind, modern AI development is increasingly focused on designing the "entire context" rather than just individual prompts. He has systematically organized the elements that make up context engineering (see reference:The New Skill in AI is Not Prompting, It's Context EngineeringBased on this idea, we have restructured it into the following five elements to make it easier to implement in a business setting.

• Instructions: Define your role and behavior, such as "You are an experienced strategy consultant."
• Knowledge: Provide reference materials such as manuals and success stories that serve as the basis for decision-making.
• Memory: Organize past decision-making processes and discussions to ensure a continuous flow of context.
• Tools: Connect to external tools such as inventory management systems and CRMs as needed.
• State: We share our current status on tasks, such as "We're currently in the idea generation phase" and "Next is the summarization phase."

The "internal context" and "external context" mentioned at the beginning primarily correspond to the elements of "knowledge," "memory," and "tools." In other words, much of the frustration illustrated stems from a lack of these three elements.

Furthermore, all elements are fundamentally complementary to one another, and the quality of the answer will decrease if even one element is missing. In practical terms, this can be put into the following four steps.

STEP 1: Define roles and goals (instructions)

First, let's define the AI's "role." Before giving individual, detailed instructions (user prompts), clearly defining its foundational "role (system prompts)" is key to obtaining consistent answers.

Role assignment:This clarifies what "perspective" or "stance" the AI ​​should adopt when thinking.

• ✕ Bad example: "Think of a strategy"
• 〇 Good example: "You are an experienced strategy consultant."

By assigning roles, AI can develop a framework for decision-making, determining "from what perspective and what to prioritize."

Setting the goal:We will share a common understanding of what constitutes "success" and what the final deliverable should look like.

• ✕ Bad example: "To wrap it up nicely"
• 〇 Good example: "Please provide a 'concrete action plan' that can be implemented immediately."

If the goal remains vague, the AI ​​will return a "plausible answer," but it may not reach a level that is usable in practical applications.

Setting constraints:We pre-set the "rules" that we want you to follow.

• ✕ Bad example: "Answer as briefly as possible."
• 〇 Good examples: "Always structure your answers in the order of conclusion, reason, and specific example," and "Do not use anything other than company jargon."

By defining constraints in advance, the number of instructions given each time is reduced, and the consistency of responses is increased.

Example output:I will illustrate the desired format for your response with a real-world example.

• ✕ Bad example: "Please write it more clearly."
• 〇 Good example: "Please follow the following format for your answer: [Heading]: [2-3 sentence explanation], [Specific example]" + Attach 1-2 examples of actual output.

By providing examples, you can accurately ensure that the "tone and level of detail in responses," which cannot be fully conveyed through instructions alone, are consistent.

STEP 2: Gather and organize information (knowledge)

Next, you select the data to reference. Taking a marketing example, the main subjects of this "knowledge" would be internally accumulated data such as CRM, purchase history, customer feedback, and marketing effectiveness measurement (MMM), which were also mentioned in the benefits chapter.

For small-scale tests, you can start by simply manually selecting 5 to 10 relevant documents.

However, as an organization's data becomes more extensive, manual selection by humans becomes subject to limitations in comprehensiveness and bias. Furthermore, instead of simply "handing over" the vast amount of raw data, it becomes necessary to "format it in a way that is easy for AI to handle."

This is where data science techniques come into play in processes such as data organization and extraction. By using appropriate methods, it becomes possible to design a context that is effective for AI while minimizing human bias.

Approach 1: Reduce the quantity(Extraction of representative samples)

When selecting and filtering information, techniques such as clustering are used to group data according to similar patterns, and mathematically fair "representative samples" are extracted from each group. This allows for both comprehensive information and an appropriate amount of data.

However, simply reducing the quantity is not enough. For AI to correctly understand the "content" of the remaining data, the next step, structuring, is necessary.

Approach ②: Structure(Organizing and understanding information)

Instead of providing raw, context-less data, we organize the overall picture of the data by extracting frequently occurring themes using "topic modeling" and quantifying positive and negative sentiment tendencies using "sentiment analysis." This minimizes the risk of the AI ​​overlooking important elements and enables logical summarization.

STEP 3: Choose how to convey information (memory/tools)

Choose how to have the AI ​​memorize the information you've prepared. Select the method that best suits your needs.

Approach 1: Deliver the entire amount at once(In-Context Learning)

This method loads all the information at once from the beginning and processes it based on the entire context. While it allows for an immediate grasp of the overall picture, it has capacity limitations, making it suitable for limited amounts of information, such as specific project documents.

Approach ②: Only perform searches when necessary.(RAG)

The system searches for relevant information from internal databases and tools and provides only the necessary content. There is no limit to the amount of information, but initial system construction is required, including database integration and the development of a search engine.

If the data being handled is limited to documents of a few dozen pages or less, option ① is the more practical choice. If you want to access a large, company-wide database, option ② is the more realistic choice.

STEP 4: Tell us where you are right now (status)

Finally, we share the task's "current state." By letting the AI ​​recognize "which phase of the project we are currently in," we can improve the accuracy of the output at each stage.

Let's consider a specific example. For instance, creating a sales proposal involves the following steps:

• Phase 1: Organizing the information gathered from customer interviews.
• Phase 2: Problem Analysis and Hypothesis Formulation
• Phase 3: Creating a solution proposal
• Stage 4: Creating the final proposal

Many AIs do not retain memories across conversations or chats, so unless the current state is explicitly stated, they tend to start each time by judging "what should be done," leading to divergent responses. However, by defining "We are currently in Stage 2: Analysis Phase," the AI ​​can focus on actions appropriate to that phase, such as "referencing the interview results (deliverables from Stage 1) and prioritizing the tasks." This allows for highly accurate answers that build upon information from previous stages while focusing on the tasks currently needed.

You don't need to perfect all four steps at once. Even just starting with Steps 1 and 2 will significantly improve the quality of your answer.

In conclusion: Small steps you can take starting today

When switching to contextual engineering, there's no need to suddenly build a large-scale, company-wide system. Let's start by experiencing the changes in our own daily operations.

• Focus on one issue.First, define just one specific problem you want to solve, such as "coming up with catchy slogans" or "improving the accuracy of customer support FAQs." By narrowing the target, it becomes easier to measure the effectiveness of the solution.
• Select high-quality materials.Select 5 to 10 relevant and high-quality resources (such as the latest manuals, past success stories, and project proposals) that can help solve the problem. In the first half of the article (STEP 2), we mentioned that using data science is ideal, but for the initial testing phase, you can select them manually.
• Test at a practical level.The selected materials are fed into the AI, and prompts are issued. While testing whether the expected output is obtained, the prompts are also adjusted (instructions are made clearer) to confirm whether the AI ​​is properly taking the content of the materials into account when responding. At this time, try comparing the responses with and without context. By doing this comparison, you should be able to clearly see how much the accuracy changes just by adding context.

The "context engineering" we've introduced here isn't a special technique for unlocking AI's potential; it's simply an extension of the basic idea of ​​organizing and appropriately delivering necessary information. By focusing not only on what to ask the AI, but also on what background information to provide, the quality of the output changes dramatically.

Start by experimenting with small themes to experience the difference firsthand. Through this accumulation of experiences, AI will evolve from a "convenient tool" to a "powerful partner that provides strategic insights."

XICA Co., Ltd. Marketing Team