Kiln AI - An open-source AI prototyping and dataset collaboration tool for fine-tuning custom models.

What is Kiln AI?

Kiln AI is an open-source AI development tool that simplifies fine-tuning of large language models (LLMs), synthetic data generation, and dataset collaboration. Kiln AI offers an intuitive desktop application supporting Windows, macOS, and Linux, allowing users to fine-tune various models (such as Llama, GPT4o, and Mixtral) with zero-code support and automated deployment. Kiln AI provides interactive tools for generating training data, supports Git-based version control, and facilitates team collaboration. Kiln AI supports automatic data generation, multi-model integration, and prioritizes privacy to ensure user data security. Kiln AI's Python library is open-source, making it easy for developers to integrate into existing workflows.

Kiln AI's main functions

• Intuitive desktop applicationsSupports Windows, macOS, and Linux systems, offering one-click installation and use, with a simple and intuitive design.
• Zero-code fine-tuningSupports multiple language models, such as Llama, GPT4o, and Mixtral, with automatic serverless deployment.
• Synthetic data generationProvides interactive visualization tools for generating training data.
• TeamworkIt uses Git-based version control, supports multi-user collaboration, and is suitable for QA, PM and domain experts to jointly participate in the construction of datasets.
• Automatic prompt generationIt supports automatically generating prompts from data, including chain thinking, few-sample and many-sample prompts.
• Wide range of supported models and providers:Supports models based on Ollama, OpenAI, OpenRouter, Fireworks, Groq, AWS, or any OpenAI API compatible.

Kiln AI's Technical Principles

• Git-based version controlIt uses Git as the underlying version control system, supporting multi-user collaboration and version management of datasets. Dataset files are stored in JSON format, supporting parallel collaboration and conflict resolution.
• Serverless DeploymentThe finely tuned model supports automatic deployment to the cloud or local storage without requiring manual server configuration. It supports multiple cloud platforms and local environments.
• Interactive data generation toolIt provides an interactive interface to help users generate high-quality synthetic data based on visualization tools. It supports various data generation strategies, such as few-shot learning and many-shot learning.
• Python library integrationProvides an open-source Python library that makes it easy for developers to integrate datasets into existing workflows. Supports use in Jupyter Notebooks, facilitating in-depth analysis for data scientists.
• Multi-model supportBased on the adapter pattern, it supports multiple language models and platforms, and provides a unified API interface, making it easy for users to switch between different models and providers.

Kiln AI's project address

• GitHub repository:https://github.com/Kiln-AI/Kiln

Kiln AI Quick Start

• Download and install:
  • Desktop ApplicationsDownload and install the free desktop application on macOS, Windows, and Linux using the provided download links.
  • Python librariesUse the command `pip install kiln-ai` to install Python libraries, integrate datasets into your workflow, build fine-tuned models, use Kiln AI in Notebooks, build custom tools, and more.
• Launch application:
  • After Android is completed, launch the application and follow the instructions to create projects and tasks, and connect to AI providers (such as Ollama, OpenAI, OpenRouter, etc.).
  • Try it out quickly with the sample tasks, or define custom tasks according to your needs.

Kiln AI's model and AI provider

• AI models and providers supported by Kiln:
  • Supported providersKiln AI is compatible with various cloud service providers and AI models, such as OpenAI, Groq, OpenRouter, AWS, and Fireworks. Users need to provide their own API keys; the system does not access users' datasets.
  • Compatible serversUsers can connect to any OpenAI-compatible server, such as LiteLLM, for vLLM, etc.
• Set up AI providers:
  • Initial settingsWhen you run Kiln for the first time, the application prompts you to set up one or more AI providers. At least one provider is required to use Kiln AI's core functionality.
  • Add or remove providersUsers can add new providers in "Settings > AI Providers and Models" or delete existing providers by editing the ~/.kiln_ai/settings.yaml file.
• Model usage and addition:
  • Recommended built-in modelsThe model has been tested and is compatible with all of Kiln AI's features. It is very easy to use and error-free. Users simply need to connect to any AI provider on the settings page and select the model on the run screen.
  • Custom ModelIf a user wants to use a model that is not listed, but is an AI provider supported by Kiln AI, the user needs to use the model in "Settings > AI Providers and Models" under "Add Model". The model will then appear in the "Untested" section of the model dropdown menu.
  • Custom OpenAI compatible serverIf a user has an OpenAI-compatible server (such as LiteLLM, vLLM, etc.), they can use it with Kiln AI. Users need to add a "Custom API" in "Settings > AI Providers and Models". All models supported by this API will appear in the "Untested" section of the Model dropdown menu.

Synthetic Data Generation by Kiln AI

• Working principle:
  • Synthetic data methods
    • Zero-sample data generationData is generated directly based on the task definition, without any additional configuration.
    • Topic Tree Data GenerationIt quickly generates extensive data based on a generated topic tree (including nested topics). Users can choose automatic topic generation or manually add topics.
    • Structured data generationIf the task requires structured input or output, the generated data will follow a user-defined JSON schema. All generated values will be validated, and data that does not conform to the schema will not be saved.
  • Generate options:
    • Model selectionIt supports multiple models (such as OpenAI, Anthropic, Llama, Google, Mistral, etc.) and multiple hosts (such as Ollama). Users can choose the appropriate model according to their needs.
    • Prompt OptionsAfter scoring some examples, unlock more powerful hint options, such as fewer samples, more samples, and mind chain hints, to improve the quality of the generated data.
  • Human guidance:
    • It generates content covering global topics, not just those within the United States.
    • Generate an example in Spanish.
    • Generate specific data for problems that are difficult for models to classify (such as satirical information).
• Iteration:
  • Open the Synthetic Dataset tab.
  • Choose high-quality models (including production models that are not fast enough and have high costs).
  • Start generating data to demonstrate the problem, using human guidance and better models to ensure output quality.
  • Manually delete examples with incorrect styles.
  • Synthetic data tools can generate accurate data (using a combination of models and guidance), scaling up the generation to hundreds of samples.
  • Save the new synthetic dataset.

Kiln AI Tuning Guide

• Step 1: Define the task and goal
  • Task definitionCreate a new task in the Kiln UI, including initial prompts, requirements, and input/output modes.
  • Example TaskGenerate news headlines: Given a summary of a news topic, generate news headlines in different styles.
• Step 2: Generate training data using synthetic data
  • Synthetic data generationProvides an interactive interface for quickly generating high-quality synthetic datasets.
  • Generation processGenerates 920 training samples in 9 minutes, supporting multiple models and multiple prompting strategies.
  • Tips and tricksImprove data quality by using high-quality models and detailed hints (such as multi-sample hints and chain thinking).
• Step 3: Select the model to be fine-tuned
  • Supported models:
    • OpenAIGPT-4o and 4o-Mini
    • MistralMixtral 8x7b MoE
    • other: Llama 3.2 (1b/3b/11b), Llama 3.1 (8b/70b), etc.
• Step 4: Start the fine-tuning task
  • Fine-tuning operationIn the Kiln UI, on the "Fine-tuning" tab, select the model, dataset, and training parameters.
  • Dataset splittingIt is recommended to create test and validation sets to evaluate model performance after fine-tuning.
• Step 5: Deploy and run the model
  • Automated deploymentAfter fine-tuning, the system automatically deploys the model without requiring additional configuration.
  • How to useSelect and use the "Run" tab based on the Kiln UI.
• Step 6 (optional): Train on your own infrastructure
  • Export DatasetExport the dataset to a common format for fine-tuning on your own infrastructure.
  • Recommendation PlatformUnsloth and Axolotl support multiple open-source models.
  • ExampleUse Unsloth notebooks to load datasets exported from Kiln AI and fine-tune them locally or in Google Colab.
• Next steps
  • Evaluation ModelUse the test set and validation set to evaluate the performance of the fine-tuned model.
  • Iterative improvementAdjust hyperparameters, optimize prompts, fix errors, and fine-tune based on evaluation results.
  • Data StrategyThe approach uses a "step-by-step" data strategy, gradually expanding from a small number of high-quality samples to a large amount of synthetic data.
• costFireworks and OpenAI's fine-tuning models are deployed in a "serverless" manner, with charges based on usage and no fixed costs.

Kiln AI Training Inference Model Guide

• Key steps in training an inference model:
  • Ensure that the training data includes "inference".Use inference models or chain-like thinking prompts to generate training data, ensuring that the dataset contains inference content.
  • Create a training dataset that includes inference.When creating a dataset, filter samples that contain reasoning/thinking.
  • Choose the right training strategySelect the "Final Response and Intermediate Inference" training strategy, which includes inference data.
  • Use appropriate hints when calling the fine-tuning modelFor best results, it is recommended to use the same prompts as during training.
• The choice between reasoning and chain thinking:
  • Inference ModelSuitable for scenarios requiring cross-domain reasoning capabilities, it creates smaller, faster models based on fine-tuning large inference models (such as Deepseek R1).
  • Chain thinkingImprove output quality based on simple "step-by-step thinking" prompts, or generate training sets for specific tasks by customizing thinking prompts.

Application Scenarios of Kiln AI

• Intelligent Customer ServiceThe intelligent customer service system generates customer service dialogue datasets and fine-tunes the language model to improve the accuracy and relevance of responses.
• medical fieldIn AI projects in the medical field, doctors (domain experts) generate medical datasets, data scientists fine-tune the models, and the QA team is responsible for verifying data quality.
• Rapid prototyping and experimentation:When developing text generation tools, use few-sample hints and multi-model support to quickly experiment with the generation effects of different models.
• educate:Edtech companies are building educational datasets, including student questions and answers, to fine-tune educational AI models.
• Financial industryThe financial industry fine-tunes its risk assessment models, processes all data locally, and ensures that customer data is not leaked.