Google's Agent white paper (Chinese version)

Reviewers and contributors

Evan Huang

Emily Xue

Olcan Sercinoglu

Sebastian Riedel

Satinder Baveja

Antonio Gulli

Anant Nawalgaria

Curators and editors

Antonio Gulli

Anant Nawalgaria

Grace Mollison

Technical writer

Joey Heimok

Designer

Michael Langning

GetGoogleintelligentbodyAgentOriginal PDF file of the white paperScan the QR code to follow us and reply with: 20250108

Introduction

Humans excel at handling complex pattern recognition tasks. However, they often rely on tools – before drawing conclusions.artificialintelligentModels can supplement their existing knowledge using tools such as books, Google searches, or calculators. Just like humans, generative models...artificialintelligentModels can also be trained to use tools to acquire real-time information or suggest actions in the real world. For example, a model can leverage database retrieval tools to access specific information, such as a customer's purchase history, to generate personalized shopping recommendations. Or, based on a user's query, a model can make various API calls to send email replies to colleagues or complete financial transactions on your behalf. To do this, a model must not only have access to a range of external tools but also the ability to plan and execute any task in a self-directed manner. This combination of reasoning, logic, and access to external information is all related to generative programming.artificialintelligentThe correlation between models gives rise to the concept of surrogate, or rather, a concept that transcends generative models.artificialintelligentThe program for model independence. This white paper will provide a more detailed overview of all these aspects and related areas.

What isAgent

In its most basic form, generative formulasartificialintelligentAn agent can be defined as an application that attempts to achieve its goals by observing the world and using the tools at its disposal. Agents are autonomous and can act independently without human intervention, especially when they have appropriate goals or objectives. Agents can also proactively achieve their goals. Even without explicit instructions from humans, agents can reason about what to do next to achieve their ultimate objective. AlthoughartificialintelligentThe concept of a proxy is very common and functional.powerfulHowever, this white paper primarily focused on generative programming at the time of its publication.artificialintelligentThe model can build specific types of agents.

To understand the inner workings of an agent, we'll first introduce the fundamental components that drive its behavior, actions, and decisions. The combination of these components can be described as a cognitive architecture, and many such architectures can be implemented through the mixing and matching of these components. Focusing on core functionality, as shown in Figure 1, the agent's cognitive architecture has three basic components.

Figure 1. General proxy architecture and components

Within the scope of the agent, the model refers to a Language Model (LM), which will be used as a centralized decision-maker in the agent's processes. The agent can use one or more LMs of any size (small/large) that can follow instruction-based reasoning and logic frameworks such as ReAct, Chain-of-Thought, or Tree-of-Thoughts. The model can be general...MultimodalThe model can be fine-tuned to suit the specific agent architecture. For optimal production performance, you should use the model best suited to your desired end application, ideally trained on data features associated with the tools you plan to use in your cognitive architecture. It's important to note that models are typically not trained based on the agent's specific configuration settings (such as tool selection, coordination/inference settings). However, the model can be further refined by providing examples demonstrating the agent's capabilities, including instances of the agent using specific tools or inference steps in different contexts.

While the underlying model can generate impressive text and images, it remains limited by its inability to interact with the external world. Tools overcome this limitation, enabling agents to interact with external data and services, thus unleashing more action beyond the underlying model. Tools come in various forms and have different functionalities. Their complexity varies, but they generally align with common web API methods such as GET, POST, PATCH, and DELETE. For example, a tool can update customer information in a database or retrieve weather data to influence travel recommendations provided by the agent to users. Through tools, agents can access and manipulate real-world information. This allows them to support more specialized systems, such as Retrieval Augmented Generation (RAG), significantly expanding the agent's capabilities beyond what the underlying model itself can achieve. We will discuss tools in detail below, but most importantly, we need to understand that tools are the bridge between the agent's internal capabilities and the external world, opening up a wider range of possibilities.

The coordination layer describes a cyclical process that controls how the agent receives information, performs internal reasoning, and uses this reasoning to inform the next action or decision. Generally, this cycle continues until the agent reaches its goal or a stopping point. The complexity of the coordination layer can vary greatly depending on the agent and the tasks it performs. Some cycles may include decision rules.SimpleThe calculations, and some loops may contain chained logic, involving additional...Machine LearningAlgorithms, or the implementation of other probabilistic reasoning techniques. We will discuss the implementation of the agent coordination layer in detail in the cognitive architecture section.

To better understand the difference between an agent and a model, please see the following Chafi:

Cognitive Architecture: How Agents Work

Imagine a chef in a busy kitchen. Their goal is to create delicious food for restaurant customers, which involves a cycle of planning, execution, and adjustment.

They collect information such as customer orders and ingredients in pantry and refrigerator.
They will do some internal reasoning based on the information they have just gathered to determine which dishes and flavors can be made.
They took matters into their own hands and prepared the dishes: chopping vegetables, seasoning, and grilling meat.

At each stage of this process, chefs adjust as needed, refining their plans when ingredients run out or customer feedback is received, and using a series of previous results to determine their next course of action. This cycle of information intake, planning, execution, and adjustment describes the unique cognitive structure a chef employs to achieve their goal. Like chefs, agents can use cognitive architectures to achieve their ultimate goals by iteratively processing information, making informed decisions, and refining their next actions based on previous outputs. At the heart of an agent's cognitive architecture is the coordination layer, responsible for maintaining memory, state, reasoning, and planning. It utilizes…fastThe field of cue engineering and related frameworks are evolving to guide reasoning and planning, enabling agents to interact more effectively with their environment and accomplish tasks. Research on cue engineering frameworks and task planning for language models is rapidly developing, yielding a variety of promising approaches. While this is not an exhaustive list, these are some of the most popular frameworks and reasoning techniques at the time of this report's publication:

ReAct is a prompting engineering framework that provides a thought process strategy for language models to reason about and act on user queries, with or without contextual examples. ReAct prompts have proven to outperform several state-of-the-art (SOTA) baselines and improve upon them. LLM Human interoperability and trustworthiness.
Thought Chains (CoT) are a hinting engineering framework that enables reasoning through intermediate steps. CoT has several sub-technologies, including self-consistent, proactive hinting, and multi-modal CoT, each with its own advantages and disadvantages depending on the specific application.
Mind Tree (ToT) is a prompting engineering framework well-suited for exploratory or strategically forward-looking tasks. It generalizes thought chain prompts, allowing models to explore various thought chains as intermediate steps in solving general problems using language models.

Agents can utilize one or more of the aforementioned reasoning techniques to select the next best action for a given user request. For example, let's consider an agent programmed to use the ReAct framework to select the correct action and tool for a user query. The sequence of events might be as follows:

User sends query to agent
Agent starts ReAct sequence
The agent prompts the model to generate the next ReAct step and its corresponding output:
- a. Problem/Issue: Providing hints for user input issues in queries.
- b. Reflection: The model's consideration of the next action.
- c. Action: The model determines the next action to be taken.
  - i. This is where you choose your tools.
  - ii. For example, an operation can be one of [fly, search, code, none], where the first three represent known tools that the model can choose, and the last one represents "no tool selection".
  - d. Action Input: The model determines which inputs (if any) to provide to the tool.
  - e. Observations: Results of action/action input sequence
    - i. This thinking/action/action input/observation can be repeated N times as needed.
    Figure 2. Example of an agent using ReAct inference in the coordination layer.
    
    As shown in Figure 2, the model, tools, and agent configuration work together to provide the user with a well-founded and concise response based on the user's original query. While the model can guess the answer based on its prior knowledge (illusion), it uses a tool (flying) to search for real-time external information. This additional information provided to the model enables it to make more informed decisions based on real-world data and then summarizes and feeds this information back to the user.
    
    In summary, the quality of an agent's response is directly linked to the model's reasoning and action capabilities for these different tasks, including its ability to select the right tools and the completeness of those tool definitions. Just as a chef uses fresh ingredients to prepare dishes and pays attention to customer feedback, an agent relies on sound reasoning and reliable information to deliver optimal results. In the next section, we will explore the various ways agents connect with fresh data.
    
    Tools: Our keys to the outside world
    
    While language models excel at processing information, they lack the ability to directly perceive and influence the real world. This limits their effectiveness in situations requiring interaction with external systems or data. In a sense, the quality of a language model depends on what it learns from its training data. But no matter how much data we feed the model, it still lacks the fundamental ability to interact with the external world. So, how can we enable our models to interact with external systems in real-time, context-aware ways? Functions, extensions, data storage, and plugins are all methods to provide models with this crucial capability.
    
    Tools have many names, but they are the link between our underlying model and the outside world. This connection to external systems and data enables our agents to perform a wider variety of tasks and with greater accuracy and reliability. For example, tools can allow agents to adjust SMAF homepage settings, update calendars, retrieve user information from databases, or send emails based on specific instructions.
    
    As of the date of this publication, Google Models can interact with three main types of tools: extensions, functions, and data stores. By equipping agents with these tools, we have unlocked enormous potential in them, enabling them not only to understand the world but also to act upon it, opening doors to countless new applications and possibilities.
    
    Understanding the extensionSimpleThe approach is to view it as a standardized bridge between the API and the agent, enabling the agent to seamlessly execute the API regardless of its underlying implementation. For example, you create an agent aimed at helping users book flights. You know you need to use the Google Flights API to retrieve flight information, but you're unsure how to get the agent to call that API endpoint.
    
    Figure 3. How does the proxy interact with external APIs?
    
    One approach is to execute custom code that receives the incoming user query, parses the relevant information, and then makes the API call. For example, in a flight booking use case, a user might say, "I want to book a flight from Austin to Zurich." In this case, our custom code solution would need to extract "Austin" and "Zurich" as relevant entities from the user query before attempting the API call. But what happens if the user says, "I want to book a flight to Zurich," without providing the destination city? Without the required data, the API call will fail, and even more code would need to be executed to catch such edge and corner cases. This approach lacks scalability and is prone to errors in any situation that exceeds the scope of the custom code implementation.
    
    A more flexible approach is to use extensions. Extensions bridge the gap between the agent and the application interface in the following ways:
    1. Use examples to teach agents how to use API endpoints.
    2. Tell the agent what parameters are needed for a successful API endpoint call.
    Figure 4. Extensions connect the agent to external application interfaces.
    
    Extensions can be designed independently of the agent, but should be provided as part of the agent's configuration. At runtime, the agent uses models and instances to determine which extension (if any) is best suited to resolve the user's query. This highlights a key advantage of extensions: their built-in instance types allow the agent to dynamically select the most appropriate extension for the task.
    
    Figure 5. One-to-many relationship between proxies, extensions, and application programming interfaces (APIs)
    
    Just as software developers decide which API endpoint to use when solving a user's problem, if a user wants to book a flight, the developer might use the Google Flights API. If a user wants to know where the nearest coffee shop is, the developer might use the Google Maps API. Similarly, the agent/model stack uses a known set of extensions to determine which one is best suited for the user's query. If you want to see how the extensions are performing, you can go to Settings > Extensions in the Gemini app and enable any extension you want to test. For example, you could enable the Google Flights extension and ask Gemini, "Show me flights from Austin to Zurich next Friday."
    
    To simplify the use of extensions, Google provides several out-of-the-box extensions that require minimal configuration.fastIntegrate it into your project and use it. The code interpreter extension in Snippet 1 allows you to generate and run Python code based on natural language descriptions.
    
    Python 导入 vertexai 导入 pprint project_id= "your_project_id" REGION = "us-central1" vertexai.init(project=PROJECT_ID, location=REGION) from vertexai.preview.extensions import Extension extension_code_interpreter= Extension.from_hub("code_interpreter") CODE_QUERY= """Write a python method to invert a binary tree in O(n) time.""" response= extension_code_interpreter.execute( operation_id = "generate_and_execute", operation_params = {"query"：CODE_QUERY} ) print("Generated Code:") pprint.pprint({response['generated_code']}) #上述代码段将生成以下代码。
```
生成代码：
类 TreeNode：
def init（self，val=0，left=None，right=None）：
self.val = val self.left = left self.right= right

def invert_binary_tree(root)：
"""
反转二叉树参数
根：二叉树的根
返回：
倒置二叉树的根。
"""
如果不是 root：
返回 None
# 递归交换左右子代 root.left、root.right =
invert_binary_tree(root.right), invert_binary_tree(root.left)
返回根
# 示例用法：
# 构建二叉树样本 root = TreeNode(4)
root.left = TreeNode(2) root.right = TreeNode(7) root.left.left = TreeNode(1) root.left.right = TreeNode(3) root.right.left = TreeNode(6) root.right.right= TreeNode(9)
# 反转二叉树
inverted_root= invert_binary_tree(root)
 ```
```
Code snippet 1. Code interpreter extensions can generate and run Python code.

In summary, extensions provide agents with a way to perceive, interact with, and influence the external world. The selection and invocation of these extension functions are guided by instances, all of which are defined as PAFIs of the extension configuration.

In software engineering, a function is defined as an independent module of code that performs a specific task and can be reused as needed. Software developers typically create many functions to accomplish various tasks when writing programs. They also define the logic for when to call function_a and function_b, as well as the expected inputs and outputs.

Functions work very similarly in the proxy world, but we can use models instead of software developers. A model accepts a set of known functions and, based on its specification, decides when to use each function and what parameters it requires. Functions differ from extensions in several ways, the most obvious being...

1. The model will output a function and its parameters, but will not call the real-time API.

2. Functionality is executed on the client side, while extensions are executed on the proxy side.

Taking Google Flights as another example,SimpleThe functionality may be similar to the example in Figure 7.

Figure 7. How do functions interact with external APIs?

Please note that the key difference here is that neither the functionality nor the agent interacts directly with the Google Flights API. So, how exactly do API calls occur?

As shown in Figures 8 and 9, using functions offloads the logic and execution of calls to the actual application interface endpoints from the proxy to the client application. This allows developers to have finer-grained control over the data flow in the application. There are many reasons why developers choose to use functions instead of extensions, but common use cases include the following:
- Application programming interface (API) calls need to be made at another layer of the application stack outside the direct proxy architecture process (such as middleware systems, front-end frameworks, etc.).
- Security or authentication restrictions that prevent the proxy from directly calling the application interface (e.g., the application interface is not exposed to the internet, or the proxy infrastructure cannot access it).
- Time or operation sequence restrictions that prevent the proxy from making real-time API calls (i.e., batch operations, manual in-loop review, etc.).
- Additional data transformation logic is needed for API responses that cannot be performed by the application proxy. For example, consider that API endpoints do not provide filtering mechanisms to limit the number of results returned. Using functions on the client side provides developers with additional opportunities to perform these transformations.
- Developers want to iterate on proxy development without deploying additional infrastructure for application interface endpoints (e.g., function calls are like "stubs" for the application interface).
As shown in Figure 8, the differences between the two methods in terms of internal architecture are subtle, but the additional control and decoupling from external infrastructure make function calls an attractive option for developers.

Figure 8. Client-side and proxy-side control division for extensions and function calls.

Models can be used to call functions to handle complex client-side execution flows for end users. In this case, agent developers might not want a language model to manage API execution (this is the case with extensions). Let's look at the following example: An agent is being trained as a travel concierge to interact with users who want to book a vacation. Our goal is for the agent to generate a list of cities so we can use it in our middleware application to download images, data, etc., for the user's travel plans. The user might say...

I want to go skiing with my family, but I don't know where to go. A typical hint for the model might result in the following output: Of course, here's a list of cities you can consider for a family ski trip:
- Krestbutt, Colorado, USA
- Whistler, British Columbia, Canada
- Zermatt, Switzerland
While the output above contains the data we need (city names), its format isn't suitable for parsing. Through function calls, we can teach the model to format the output in a structured style (such as JSON), making it easier for other systems to parse. An example of the function's JSON output might look like this, given the same user input.

Code snippet 5. Example of a function call payload to display a list of cities and user preferences.

This JSON payload is generated by the model and then sent to our client server to accomplish whatever we want. In this specific case, we will call the Google Places API to retrieve the city provided by the model and find images, then feed them back to our users as formatted rich content. The sequence diagram in Figure 9 illustrates the above interaction process in detail.

Figure 9. Sequence diagram showing the lifecycle of a function call.

The example in Figure 9 shows how the model is used to "fill in" the parameters required by the client-side user interface to call the Google Places API. The client-side user interface uses the parameters provided by the model in the returned Function to manage the actual API call. This is just one use case for function calls, but there are many other scenarios worth considering, such as...
- You want the language model to suggest functions you use in your code, but you don't want to include credentials in the code. Because function calls don't execute functions, you don't need to include certificate and function information in your code.
- You are running an asynchronous operation, which may take more than a few seconds. Since function calls are asynchronous operations, these situations are handled well.
- You want to run the function on a device that is different from the system that generated the function call and its arguments.
A key point to remember regarding functions is that they not only give developers more control over the execution of API calls, but also over the entire data flow throughout the application. In the example in Figure 9, the developer chose not to return API information to the agent because this information is not important to the agent's future actions. However, depending on the application's architecture, returning external API call data to the agent to influence future reasoning, logic, and operational choices may make sense. Ultimately, application developers must choose the appropriate approach based on the specific application.

Functional Sample Code

To achieve the above output in a ski resort scenario, let's use the gemini-1.5-flash-001 model to build the components that enable this goal.

First, we define the display_cities function as aSimplePython methods.

Code snippet 6. Example of a Python method for displaying a list of cities.

Next, we will instantiate the model, build the tool, and then pass the user's query and the tool into the model. Executing the code below will produce the output at the bottom of the code snippet.

Code snippet 7. Create a tool to send user queries to the model and allow function calls.

In short, a function providesSimpleThe clear framework enables application developers to have fine-grained control over data flow and system execution, while effectively leveraging agents/models to generate key inputs. Developers can selectively choose whether to have agents "participate in the loop" by returning external data, or omit agents altogether, depending on the specific application architecture requirements.

Imagine a language model as a well-stocked library containing its training data. However, unlike a library that constantly acquires new books, this model remains static, storing only the knowledge from its initial training. This presents a challenge because real-world knowledge is constantly evolving. Data storage addresses this by providing more dynamic and...up to dateThis information addresses this limitation and ensures that the model's response is always based on facts and relevance. Consider a common scenario where developers might need to provide the model with a small amount of additional data, perhaps in spreadsheet or PDF format.

Figure 10. How do agents interact with structured and unstructured data?

Data storage allows developers to provide additional data to the agent in its original format, eliminating time-consuming data transformations, model retraining, or fine-tuning. Data storage transforms incoming documents into a set of vector database embeddings, which the agent can use to extract the necessary information to supplement its next steps or responses to users.

Figure 11. Data storage connects the agent to various types of new real-time data sources.

In generative formulaartificialintelligentIn the context of proxies, data storage is typically implemented in the form of vector databases, which developers expect the proxy to access at runtime. While we won't delve into vector databases here, a key point to understand is that they store data as vector embeddings, which are high-dimensional vector or mathematical embeddings.

The representation of the provided data. One of the most recent examples of combining data storage with language models is the implementation of Retrieval Augmented language models.

Generative algorithms (RAG) are the basis for these applications. These applications aim to expand the breadth and depth of a model's knowledge beyond its basic training data by allowing it to access data in various formats.
- Website content
- Structured data formats, such as PDF, Word documents, CSV, spreadsheets, etc.
- Unstructured data in formats such as HTML, PDF, and TXT.
Figure 12. A one-to-many relationship between the agent and the data store, which can represent various types of pre-indexed data.

The basic process modeling for each user request and proxy response loop is generally shown in Figure 13.
1. User queries are sent to the embedding model to generate query embedding information.
2. Then, matching algorithms such as SCaNN are used to match the query embedding with the content of the vector database.
3. The matched content is retrieved from the vector database in text format and sent back to the agent.
4. The agent receives user queries and retrieved content, and then formulates a response or action.
5. Send final reply to user
Figure 13. Lifecycle of user requests and proxy responses in a RAG-based application

The ultimate application result is that the agent uses vector search to match user queries with known data stores, retrieves the raw content, and provides it to the coordination layer and model for further processing. The next step might be to provide the user with the final answer or to perform additional vector searches to further refine the results.

Figure 14 shows an example of interacting with an agent that implements RAG using ReAct reasoning/planning capabilities.

In summary, extensions, functions, and data stores constitute several different tool types for the agent to use at runtime. Each tool has its own purpose, and they can be used together or independently, at the discretion of the agent developers.

Improve model performance through targeted learning

A crucial aspect of effectively using a model is its ability to select the right tools when generating output, especially when tools are used at scale in production. While general training helps models develop this skill, real-world scenarios often require knowledge beyond the training data. Think of it as the difference between basic cooking skills and mastering a specific dish. Both require basic culinary knowledge, but the latter requires targeted learning to achieve more nuanced results.

To help the model acquire this specific type of knowledge, several methods are available:
- Contextual learningThis approach provides hints, tools, and a few examples to a general model during inference, enabling it to learn "on the fly" how and when to use these tools in a specific task. The ReAct framework is an example of this approach applied to natural language processing.
- Retrieval-based context learningThis technology dynamically populates model hints by retrieving the most relevant information, tools, and related examples from external storage. For example, Vefiex... artificialintelligentThe "Example Storage" in the extension or the previously mentioned RAG-based data storage.
- Learning based on fine-tuningThis approach involves training the model using a large dataset of specific examples before inference. This helps the model understand when and how to apply the cefiain tool before receiving any user queries.
To further understand each targeted learning method, let's revisit the analogy of cooking.
- Imagine a chef receiving a specific recipe (a hint), some key ingredients (related tools), and a few sample dishes from a customer. Based on this limited information and the chef's general culinary knowledge, they need to figure out how to cook a dish that best matches the recipe and the customer's preferences. This is contextual learning.
- Now, let's imagine a chef in a kitchen with a well-stocked pantry (external data storage) filled with various ingredients and cookbooks (examples and tools). The chef can now dynamically select ingredients and cookbooks from the pantry and better adapt them to the client's recipes and preferences. In this way, the chef can utilize existing and new knowledge to create more intelligent and refined dishes. This is context-based learning based on retrieval.
- Finally, let's imagine sending chefs back to school to learn a new dish or set of dishes (pre-trained on a larger dataset of specific examples). This allows the chefs to handle recipes from future customers with a deeper understanding. This approach would be ideal if we wanted chefs to excel in a specific dish (knowledge domain). This is what fine-tuning-based learning is all about.
These methods each have their advantages and disadvantages in terms of speed, cost, and latency. However, by combining these techniques into a single proxy framework, we can fully leverage their strengths and minimize their weaknesses, thereby achieving greater [advantages/benefits].powerfulA more adaptable solution.

Using LangChain's proxyfast Stafi

To provide examples of real-world executable proxy operations, we will build one using the LangChain and LangGraph libraries.fastPrototypes. These popular onesopen sourceThe library allows users to build client agents to answer user queries by "chaining" sequences of logic, reasoning, and tool calls. We will use the gemini-1.5-flash-001 model and some...SimpleTools are used to answer users' multi-stage queries, as shown in fragment 8.

The tools we used are the SerpAPI (for Google Search) and the Google Places API. After executing the program in code snippet 8, you can see the example output in code snippet 9.

Code snippet 8. Proxy examples and tools based on LangChain and LangGraph

Code segment 9. Output of the program in Figure 8

Although this is quiteSimpleThis example demonstrates how fundamental components such as models, coordination, and tools work together to achieve a specific goal. In the final section, we'll explore how these components are integrated into hosting products at Google's scale (such as Vefiex). artificialintelligentIt combines proxy and generative game elements.

Using Vefiex artificialintelligentProduction applications of agents

This white paper explores the core components of an agent, but building production-ready applications requires integrating the agent with other tools such as user interfaces, evaluation frameworks, and continuous improvement mechanisms. Google's Vekex... artificialintelligentThe platform provides a fully manageable environment, encompassing all the essential elements mentioned above, thus simplifying the process. Developers can utilize a natural language interface.fastUsers can easily build the desired system behavior by defining key elements of their agents—goals, task instructions, tools, task delegation sub-agents, and examples. Furthermore, the platform comes with a suite of development tools for testing, evaluating, measuring agent performance, debugging, and improving the overall quality of the developed agents. This allows developers to focus on building and refining their agents, while the platform itself manages the complex infrastructure, deployment, and maintenance.

In Figure 15, we provide a diagram of Vefiex. artificialintelligentAn example of a proxy architecture built on the platform, which uses Vefiex. Agent Builder, Vefiex Extensions, Vefiex AI Agent It includes various features such as a Builder, function calls, and Vefiex sample storage. The architecture encompasses a wide range of components required for many production-ready applications.

Figure 15. Based on Vefiex artificialintelligentExample of an end-to-end proxy architecture built on the platform

You can try out examples of this pre-built proxy architecture from our official documentation.

In this white paper, we discuss generative formulas.artificialintelligentThe fundamental building blocks of agents, their composition, and effective methods for implementing them in the form of a cognitive architecture. Some of the key contents of this white paper include:
1. Agents can leverage one or more language models to determine when and how to perform state transitions and use external tools to accomplish any number of complex tasks that are difficult or impossible for the model to perform independently.
2. At the heart of agent operation is the coordination layer, a cognitive architecture used to construct reasoning, planning, and decision-making, and to guide agent actions. Various reasoning techniques, such as ReAct, Chain-of-Thought, and Tree-of-Thoughts, provide the coordination layer with a framework for receiving information, performing internal reasoning, and generating informed decisions or responses.
3. Tools such as extensions, functions, and data stores are the keys for agents to access the outside world, enabling them to interact with external systems and acquire knowledge beyond their training data. Extensions bridge the gap between agents and external application programming interfaces (APIs), making it possible to execute API calls and retrieve real-time information. They generate function parameters that can be executed on the client side. Data stores allow agents to access structured or unstructured data, enabling data-driven applications.
The future of agent technology promises exciting advancements, and we are only beginning to grasp the surface of its potential. As tools become more sophisticated and reasoning capabilities more powerful, agents will be able to solve increasingly complex problems. Furthermore, the strategic approach of "agent chains" will continue to be utilized. By combining specialized agents (each adept at a specific domain or task), we can create a "hybrid agent experience" approach that can achieve outstanding results across various industries and problem domains.

It's important to remember that building complex agent architectures requires an iterative approach. Experimentation and refinement are key to finding solutions for specific business cases and organizational needs. Because the underlying model underpinning the agent architecture is generative, no two agents are the same. However, by leveraging the strengths of each fundamental component, we can create impactful applications that extend the capabilities of language models and drive real-world value.
1. Shafran, I., Cao, Y. et al., 2022, 'ReAct: Collaboration of Reasoning and Action in Language Models'. Available at:hflps://arxiv.org/abs/2210.03629
2. Wei, J., Wang, X. et al., 2023, "Chain-of-Thought" Prompting Elicits Reasoning in Large Language Models. See hflps://arxiv.org/pdf/2201.11903.pdf.
3. Wang, X. et al., 2022, 'Self-Consistency Improves Chain of Thought Reasoning in Language Models'. See hflps://arxiv.org/abs/2203.11171.
4. Diao, S. et al., 2023, "Active Prompting with Chain-of-Thought for Large Language Models". See hflps://arxiv.org/pdf/2302.12246.pdf.
5. Zhang, H. et al., 2023, 'Multimodal Chain-of-Thought Reasoning in Language Models'. See hflps://arxiv.org/abs/2302.00923.
6. Yao, S. et al., 2023, "Tree of Thoughts: Carefully Solving Problems with Large Language Models". Available at:hflps://arxiv.org/abs/2305.10601.
7. Long, X., 2023, "Large Language Model Guided Tree-of-Thought". See hflps://arxiv.org/abs/2305.08291.
8. Google. Google Gemini app. URL:hflp://gemini.google.com.
9. Swagger. OpenAPI Specification. URL:hflps://swagger.io/specification/.
10. Xie, M., 2022, "How Does Situated Learning Work? A Framework for Understanding the Differences from Traditional Supervised Learning." See [link to article/reference]. hflps://ai.stanford.edu/blog/understanding-incontext/.
11. Google Research. 'ScaNN (Scalable Nearest Neighbors)'. Please visit [link to Google Research]. hflps://github.com/google-research/google-research/tree/master/scann.
12. LangChain.LangChain. (For more information, please refer to:)hflps://python.langchain.com/v0.2/docs/introduction/.
GetGoogleintelligentbodyAgentOriginal PDF file of the white paperScan the QR code to follow us and reply with: 20250108

All Categories

Introduction

What isAgent

Cognitive Architecture: How Agents Work

Tools: Our keys to the outside world

Functional Sample Code

Improve model performance through targeted learning

Using LangChain's proxyfast Stafi

Using Vefiex artificialintelligentProduction applications of agents