General365 - a general-purpose inference benchmark open-sourced by the Meituan LongCat team.

What is General365?

General365 is an open-source general reasoning benchmark developed by Meituan's LongCat team. It includes 365 original seed questions and 1095 extended variations, covering eight dimensions of reasoning challenges. The benchmark strictly limits the knowledge scope to the K-12 level, decoupling the reasoning ability of large models from their reliance on specialized knowledge, and realistically assessing the models' general logical reasoning level in everyday scenarios. Real-world testing shows that among 26 mainstream large models, only Gemini 3 Pro achieved an accuracy rate of 62.8%, with the vast majority of models failing to reach the 60% passing mark.

General365's main functions

• High diversity assessment365 original seed problems created by human engineers + 1095 variations, covering eight dimensions: complex constraints, branch enumeration, spatiotemporal reasoning, recursive backtracking, semantic interference, implicit information, optimal strategy, and probabilistic uncertainty.
• Decoupling of Reasoning and KnowledgeLimiting background knowledge to the K-12 range, purely measuring logical reasoning ability rather than knowledge retrieval.
• Hybrid scoring systemCombining rule-based scoring with model scoring (GPT-4.1), the accuracy of manual verification scoring reached 99.6%.
• Public subsets and hidden test sets180 seed problems and variations (out of a total of 720 problems) have been released. The remaining problems are used as a hidden test set to prevent data corruption.
• Multi-model cross-sectional evaluationIt supports standardized evaluation of the inference capabilities of 26+ models, including OpenAI, Gemini, Anthropic, DeepSeek, Qwen, GLM, Kimi, and LongCat.

General365's technical principles

• Eight dimensions to break down general reasoningThe general reasoning problem is broken down into 8 core challenge types, with each question corresponding to at least one of them, and nearly 70% of the questions have more than two category tags.
• Avoid templates and rote memorizationBy using t-SNE semantic distribution verification and Gemini 3 Pro inference path similarity scoring, we ensure the logical independence of the questions and prevent the model from scoring by "memorizing templates".
• Difficulty filtering and manual reviewAll questions have undergone difficulty filtering, diversity expansion, model expansion, and manual review, resulting in 1460 high-quality questions.
• Hybrid scoring frameworkNumerical questions were analyzed and verified using the math-verify tool; multiple-choice and text-based questions were scored using the GPT-4.1 model.

General365's core advantages

• Focus on real reasoning abilityUnlike subject-based competitions such as AIME and IMO, General365 focuses on everyday general logic, exposing the shortcomings of large models that are "high-scoring but low-performance".
• High discriminationThe state-of-the-art (SOTA) model is barely passable, avoiding the performance saturation and indistinguishability issues of existing benchmarks (such as BBH and BBEH).
• Diversity that can withstand scrutinyThe semantic distribution is evenly distributed, and the logical independence is much higher than that of BBH and BBEH.
• Open source and reproducibleThe GitHub repository provides complete evaluation code, supporting rapid integration and reproduction by the community.

General365's project address

• Project official websitehttps://general365.github.io/
• GitHub repository: https://github.com/meituan-longcat/General365
• HuggingFace model libraryhttps://huggingface.co/datasets/meituan-longcat/General365_Public
• arXiv technical paper: https://arxiv.org/pdf/2604.11778

Comparison of General365 products with similar competitors

Application scenarios of General365

• Large Model Development and EvaluationIt helps model developers identify weaknesses in reasoning capabilities and optimize weak dimensions such as complex constraints, semantic interference, and optimal strategies.
• Model Selection ReferenceIt provides objective data support for enterprise users to select commercial or open-source models with stronger reasoning capabilities.
• academic research: To provide standardized evaluation tools for the general reasoning field and promote the evolution of LLM from "subject matter experts" to "general reasoners".
• Inference efficiency analysisIt supports analyzing the relationship between model accuracy and the number of output tokens, and evaluating inference efficiency.