Data Formulator - An open-source AI data visualization tool from Microsoft Research.

What is Data Formulator?

Data Formulator is an open-source, AI-driven data visualization tool from Microsoft Research that helps users quickly create rich data visualizations through simple interactions and commands. Combining a graphical user interface (GUI) and natural language input (NL), users can design charts by dragging and dropping or directly inputting their requirements, while AI handles complex data transformations.

Main functions of Data Formulator

• Combining graphical interface with natural language inputUsers can drag and drop data fields into chart properties, or describe their needs in natural language. The AI will then perform data transformation and visualization based on the instructions.
• Supports complex data transformationUsers can enter the name of a non-existent data field in the coding field, and the AI will perform data calculations and transformations based on natural language prompts to generate new visualizations.
• Iterative Visual DesignData Formulator offers a "data thread" feature, allowing users to perform further operations on existing charts, and the AI will update the charts based on natural language instructions.
• Results Validation and Error CorrectionUsers can view AI-generated transformation data, visualizations, and code, and understand the data transformation process through the code explanation module. If errors are found, they can be corrected using the iterative mechanism of the data thread.
• Flexible chart style adjustmentUsers can directly adjust chart styles (such as color schemes, axis order, etc.) on the concept coding shelf without performing additional data transformations and see immediate visual feedback.

The technical principles of Data Formulator

• Multimodal interactive interfaceData Formulator combines a graphical user interface (GUI) with natural language input (NL), allowing users to define their visualization needs through drag-and-drop operations or by directly entering natural language commands. This dual approach enables users to choose their preferred method of operation, efficiently conveying their requirements through either interface interaction or verbal description.
• Concept binding and data transformationUsers first define the data concepts they plan to visualize using natural language or examples, and then bind these concepts to visualization channels (such as x-axis, y-axis, color, etc.). The Data Formulator automatically transforms the input data into the required format and generates the desired visualization through its AI agent.
• AI Agents and Code GenerationThe Data Formulator's backend uses the Flask framework and receives frontend requests via a RESTful API. When a user clicks the "Formulate" button, the frontend sends a... POST Request to the backend /derive-data Interface. The backend calls the AI agent (such as...) based on the user's input commands and data. DataTransformationAgentV2Generate Python code and execute it to complete the data transformation.
• Data processing and feedback mechanismData Formulator offers a data threading feature, allowing users to perform further operations on existing charts, with the AI updating the charts based on natural language instructions. Data Formulator also provides a feedback mechanism, allowing users to view the AI-generated transformation data, visualizations, and code to ensure the results meet expectations.
• Open source and flexibilityData Formulator is an open-source project that users can install and run locally via Python pip, or use directly from GitHub Codespaces.

The project address for Data Formulator

• Github repository:https://github.com/microsoft/data-formulator
• arXiv technical paper:https://arxiv.org/pdf/2408.16119

Application scenarios of Data Formulator

• Data Analysis and VisualizationData Formulator helps users quickly transform complex data into intuitive visual charts, enabling them to rapidly discover trends and patterns within the data.
• Data Concept Extension and ComputationUsers can define non-existent data concepts by inputting them in natural language. For example, when analyzing sustainable energy data, users can add the data concept of "sustainable energy percentage" to the y-axis. Even if the original data does not directly provide a percentage value, Data Formulator will automatically calculate and generate the corresponding visualization.
• Iteration and optimizationData Formulator supports iterative design based on existing charts. Users can modify and optimize existing charts using natural language commands, without having to describe the entire design from scratch.
• Multimodal interactionUsers can define their visualization needs through a graphical interface (drag and drop) or natural language input, making Data Formulator suitable for users of different skill levels.