Agent-E - An AI-powered browser automation system built on the AutoGen agent framework.
Agent-E is an intelligent automation system built on the AutoGen proxy framework, focusing on automated operations within the browser. Based on natural language interaction, Agent-E can perform tasks such as filling out forms, searching and ranking e-commerce products, and locating webpages...
What is Agent-E?
Agent-E is an intelligent automation system built on the AutoGen agent framework, focusing on automated operations within the browser. Based on natural language interaction, Agent-E can perform a variety of complex tasks, such as filling out forms, searching and ranking e-commerce products, locating web page content, managing online media playback, conducting deep web searches, automating project management tasks, and providing personalized shopping assistance. Agent-E improves online efficiency, reduces repetitive tasks, and allows users to focus on more important matters.
Main functions of Agent-E
- Form fillingAutomatically fills out online forms, including personal information input.
- E-commerce search and rankingSearching for and sorting products on e-commerce websites such as Amazon based on criteria such as sales volume or price.
- Content positioning: To find specific content on a website, such as sports scores or university contact information.
- Media InteractionInteract with web-based media, including playing YouTube videos and managing playback settings.
- Web searchPerform comprehensive web searches and collect information on a wide range of topics.
- Project Management AutomationFilter issues and automate workflows on project management platforms such as JIRA.
Agent-E's technical principles
- Proxy-based architectureBased on the AutoGen proxy framework, it performs tasks using proxies (such as user agents and browser navigation proxies).
- Skills LibraryThe core functionality revolves around a skill library, which contains a series of predefined actions (skills), divided into perception skills and action skills.
- Natural Language InteractionIt supports users interacting with the browser using natural language, making task execution more intuitive.
- DOM distillationBased on DOM distillation technology, Agent-E simplifies the HTML DOM into relevant JSON snapshots, focusing on elements relevant to the user's task.
- Change ObservationAfter executing the action, Agent-E monitors the status changes and provides them to LLM in the form of verbal feedback to guide more accurate performance.
- Hierarchical planningHierarchical planning is adopted to decompose complex tasks into subtasks, which are then handled by agents at different levels.
Agent-E project address
- GitHub repository:https://github.com/EmergenceAI/Agent-E
- arXiv technical paper:https://arxiv.org/pdf/2407.13032
Application scenarios of Agent-E
- Online shoppingAutomatically searches for products, sorts the results, and adds them to the shopping cart, simplifying the shopping process.
- Information retrievalIt can quickly collect specific information from the internet, such as news or academic materials, to improve research efficiency.
- Form automationAutomatically fills out and submits online forms, reducing manual input and saving time.
- Personal AssistantWe offer personalized suggestions based on individual preferences, such as restaurant recommendations, to improve your quality of life.
- Media Playback ManagementAutomatically play and control music and video content to enhance the entertainment experience.