DynaSaur - A large language model proxy framework from Adobe
DynaSaur is a large language model (LLM) agent framework from Adobe Research that breaks through the limitations of traditional LLM agent systems that are restricted by predefined action sets. The framework supports dynamic creation and composition of agent actions, based on generative...
What is DynaSaur?
DynaSaur, developed by Adobe Research, is a large language model (LLM) agent framework that breaks through the limitations of traditional LLM agent systems that are restricted to predefined sets of actions. The framework supports the dynamic creation and composition of actions by agents, enabling more flexible problem-solving by generating and executing Python code to interact with the environment. DynaSaur can accumulate generated actions to build reusable function libraries, improving efficiency and adaptability for future tasks. In GAIA benchmark tests, DynaSaur demonstrates significant flexibility, especially excelling in handling complex and long-term tasks.
Main functions of DynaSaur
- Dynamic Action CreationBased on the environment and task requirements, new Python functions are dynamically generated as actions, without relying on a predefined set of actions.
- Action accumulation and reuseThe framework accumulates the generated actions, builds a reusable function library, and uses it in future tasks to improve problem-solving efficiency.
- Environmental InteractionIt is based on generating and executing Python code to interact with the environment, defining new actions or calling existing actions.
- Flexibility and adaptabilityIt can adapt flexibly to complex and long-term tasks, especially when predefined actions are not applicable or fail, and can recover and complete the task.
DynaSaur's technical principles
- Action representationEach action is represented as a Python function, leveraging Python's versatility and LLM's code generation capabilities.
- Action SearchBased on the action retrieval function, the most relevant previously generated actions are retrieved from the accumulated action library according to the query and context.
- Accumulation of movementsDuring task execution, newly generated actions are added to the action library, and these actions are used for future tasks.
- Partially Observable Markov Decision Processes (POMDPs)The agent's behavior is modeled as a POMDP, which includes task space, action space, state space, observation space, state transition function, and observation function.
- Dynamic motion space: Introduce a potentially infinite set of actions A*, which allows the agent to propose any action to solve the task at each time step.
DynaSaur project address
- GitHub repository:https://github.com/adobe-research/dynasaur
- arXiv technical paper:https://arxiv.org/pdf/2411.01747
Application scenarios of DynaSaur
- Automated customer serviceAs a chatbot, it dynamically generates solutions to handle customer queries and questions, providing more flexible and personalized services.
- Intelligent personal assistantAs a personal assistant, it helps users manage schedules, perform tasks, and search for information, and dynamically learns new skills based on users' needs.
- Software development assistanceIt helps software developers automatically generate code, solve programming problems, or act as an automated test agent in software testing.
- Education and trainingIn the field of education, it serves as a teaching aid, dynamically adjusting the content and difficulty of teaching according to students' learning progress and needs.
- Data Science and AnalyticsIt handles complex data analysis tasks, dynamically generates data processing workflows, and helps researchers and analysts explore data and identify patterns more efficiently.