Chronicles-OCR - A visual perception evaluation benchmark launched by Tencent in collaboration with universities and other institutions. - AiBoss

What is Chronicles-OCR?

Chronicles-OCR is the industry's first cross-temporal visual perception evaluation benchmark jointly launched by Tencent Hunyuan, the Institute of Information Engineering of the Chinese Academy of Sciences, Anyang Normal University, Nankai University, and the Palace Museum. It covers the complete evolutionary trajectory of the "seven styles" of Chinese characters. The benchmark contains 2,800 high-quality images and the system covers seven major scripts: oracle bone script, bronze script, seal script, clerical script, regular script, running script, and cursive script. Through a stage-adaptive annotation paradigm and four evaluation tasks, it evaluates the perceptual robustness of the visual large language model in the face of the continuous evolution of Chinese character forms over thousands of years.

Chronicles-OCR Main Functions

Seven-body comprehensive evaluationIt fully covers seven major Chinese scripts: Oracle Bone Script, Bronze Script, Seal Script, Clerical Script, Regular Script, Running Script, and Cursive Script, constructing a cross-time evaluation system from the Shang Dynasty to modern times.
Stage-adaptive annotationFor ancient scripts (oracle bone script/bronze script/seal script), it provides single-character-level bounding boxes + modern Chinese character mapping annotations; for mature fonts (clerical script/regular script/running script/cursive script), it provides sequence-level layout understanding annotations to adapt to the morphological differences of different historical stages.
Four core evaluation tasksIt supports cross-period character localization, fine-grained ancient character recognition, ancient text parsing, and font classification, providing a comprehensive evaluation of the model's capabilities.
Visual reference mechanismBy marking target characters in an image with colored boxes, the model can be made to recognize them, thus achieving a decoupled evaluation of character deciphering ability and spatial localization ability.
High-quality expert annotationMulti-level cross-annotation was performed by PhDs, graduate students, and experts in paleography to ensure high fidelity in font classification, bounding box positioning, and character transcription.
Authoritative data source integrationThe oracle bone script comes from the Key Laboratory of Oracle Bone Script Information Processing at Anyang Normal University; the bronze script/seal script was compiled by the paleography team; and the clerical script/regular script/running script/cursive script comes from the handwritten script recognition test dataset of cultural relics from the Palace Museum.
Model robustness diagnosisThis study systematically evaluates the perceptual capabilities of a visual large language model in the process of thousands of years of Chinese character morphological evolution, revealing the key bottleneck in the generalization of modern document parsing to historical writing systems.

How to use Chronicles-OCR

Access open source repositoryGo to the GitHub repository https://github.com/VirtualLUOUCAS/Chronicles-OCR to get the dataset, paper, and evaluation code.
Download datasetObtain the complete dataset containing 2,800 images, including 400 images each of oracle bone script, bronze script, seal script, clerical script, regular script, running script, and cursive script, along with annotation files in JSON or XML format.
Understanding data formatsAdaptive annotation standards for the reading stage: For the ancient script stage (oracle bone script/bronze script/seal script), it is single-character-level bounding box coordinates + modern Chinese character mapping; for the mature script stage (clerical script/regular script/running script/cursive script), it is sequence-level text layout and reading order annotation.
Select evaluation taskBased on the research objectives, the following four tasks were selected: cross-period character localization, fine-grained ancient character recognition, ancient text parsing, and font classification.
Prepare the model to be testedDeploy the Visual Large Language Model (VLLM) to be evaluated, ensuring that the model supports both image input and text output capabilities.
Execution Model InferenceInput the dataset images into the model and output the corresponding results according to the task requirements (such as bounding box coordinates, modern Chinese character transcription, font category, etc.).
Run the evaluation scriptUsing the official evaluation code provided by the repository, the model output is compared with the standard annotation, and the metrics for each task (such as positioning accuracy, NED edit distance, classification accuracy, etc.) are automatically calculated.
Analysis and evaluation resultsBy comparing the model performance data in the benchmark report, we can diagnose the limitations and deficiencies of the tested model in terms of ancient character perception, historical layout understanding, and font evolution cognition.
Citations and FeedbackPlease cite arXiv:2605.11960v1 in relevant research papers, and provide feedback on usage issues or suggestions for improvement to the author team via GitHub Issues.

The core advantages of Chronicles-OCR

Industry-first cross-time evaluation systemThe first system to cover the complete evolutionary trajectory of the "seven styles" of Chinese characters (oracle bone script, bronze script, seal script, clerical script, regular script, running script, and cursive script) provides a visual perception evaluation benchmark, filling the gap in evaluation across the time span from the Shang Dynasty to modern times.
Top academic institutions jointly endorseThis product was jointly launched by Tencent Hunyuan, the Institute of Information Engineering of the Chinese Academy of Sciences, the Key Laboratory of Oracle Bone Inscription Information Processing of Anyang Normal University, Nankai University, and the Palace Museum. The data source and annotation quality have undergone rigorous academic review.
Pioneering a phased adaptive annotation paradigmTo address the dramatic morphological changes in fonts across different historical periods, this approach differentiates between single-character positioning and mapping in ancient script stages and sequence-level layout understanding in mature font stages, resulting in more precise evaluation dimensions.
Deeply revealing the limits of model capabilitiesThis study not only provides evaluation data but also systematically reveals the disastrous failures of current mainstream VLLMs in fine-grained spatial localization and semantic deciphering of ancient characters, providing a clear direction for model optimization.
Authoritative data sources and expert annotationOracle bone script was sourced from a professional laboratory at Anyang Normal University; bronze inscriptions/seal script were compiled by a team of PhDs in paleography; and clerical, regular, running, and cursive scripts were taken from a dataset of cultural relics from the Palace Museum. High fidelity was ensured through multi-level cross-annotation.
Visual reference decoupling evaluation mechanismBy using colored boxes for visual reference, character-level decoding capabilities and spatial positioning capabilities are decoupled and evaluated, allowing for precise diagnosis of specific model shortcomings.
Fully open source and reproducibleBoth the paper and the dataset have been open-sourced, facilitating reproduction, verification, comparison, evaluation, and further research by the academic community.

Comparison of Chronicles-OCR with similar competing products

Comparison Dimensions	Chronicles-OCR	AncientDoc	ACCID
Issuing agency	Tencent Hunyuan, Institute of Information Engineering, Chinese Academy of Sciences, Anyang Normal University, Nankai University, Palace Museum	ByteDance, Beijing Jiaotong University, etc.	Academic research institution (the paper does not specify a single institution).
Dataset size	2,800 strictly balanced images, 400 for each of the seven bodies.	Approximately 3,000 pages of images from over 100 ancient books.	2,892 character category images, including radical-level annotations.
Coverage	The seven major script styles: Oracle Bone Script, Bronze Script, Seal Script, Clerical Script, Regular Script, Running Script, and Cursive Script.	14 categories of ancient books and documents (classics, history, philosophy, literature, medicine, astronomy and calendrical calculations, Chu Ci, etc.)	Images of ancient Chinese characters (emphasizing radical structure)
Time span	Approximately three thousand years (from the Shang Dynasty to modern times).	From the Warring States Period, Qin and Han Dynasties to the Ming and Qing Dynasties	Ancient Chinese characters (without a clearly defined period)
Evaluation task	Cross-period character localization, fine-grained ancient character recognition, ancient text parsing, and font classification	Page-level OCR, Plain Language Translation, Reasoning QA, Knowledge QA, Language Variation QA	Zero-sample character recognition, radical recognition
labeling granularity	Phase-based adaptive design: Ancient Chinese characters at the single-character level bounding box + modern Chinese character mapping; mature font sequence-level layout.	Page-level text, question-and-answer pairs, translation pairs	Character level + radical level (including coordinates and structure)
Target Model	Visual Large Language Models (VLLMs)	Visual Language Models (VLMs)	Zero-shot OCR / Deep learning model
Core Innovation	It pioneered a cross-time Chinese character evolution evaluation, a stage-adaptive annotation paradigm, and a visual reference decoupling assessment.	The first multi-task evaluation benchmark for ancient books, extending from OCR to knowledge reasoning.	Pioneering fine-grained annotation at the departmental level, supporting zero-shot learning.

Application scenarios of Chronicles-OCR

Intelligent recognition of ancient charactersIt assists in the automated identification and expert verification of incompletely deciphered ancient scripts such as oracle bone script and bronze script, lowering the threshold for paleography research.
Ancient Books Digitization ProjectHigh-precision OCR and structured analysis of calligraphy, rubbings, and documents from various dynasties will promote the digital archiving of cultural heritage.
Font Evolution ResearchUsing AI models to analyze the morphological evolution of Chinese characters from oracle bone script to regular script, supporting interdisciplinary research in philology and history.
VLLM Capability AssessmentAs a standardized benchmark, it evaluates and compares the cross-temporal perception capabilities and robustness of different visual large language models.
Cultural Relics Information ExtractionExtracting textual information from images of cultural relics such as bronze inscriptions, bamboo slips, and silk manuscripts to assist in archaeological research and artifact authentication.
Educational and popular science applicationsThis tool is used in teaching Chinese character culture to demonstrate ancient character recognition and visualize the evolution of Chinese characters, thereby enhancing the public's understanding of the history of Chinese characters.