News
optstop, an open-source LLM benchmarking tool from AISI (UK).
The UK AI Security Institute has released an open-source Python tool, optstop, which can stop sampling early when the evaluation results reach a preset accuracy or tend to stabilize.
The UK-based AI Security Institute has released an open-source Python tool, optstop.This tool can stop sampling early when the LLM evaluation results reach the preset accuracy or tend to stabilize, so that computing power can continue to be invested in tasks with higher uncertainty.
AISI states that in initial tests covering different scoring types and model performance levels, optstop saved 57% to 97% of program runs without affecting score estimation. This percentage is based on results obtained under specific testing conditions.
refer to:UK AISI Explanation