Anthropic Releases Assessment and Remediation Announcements: Strengthening Sandbox Monitoring and Reviewing High-Risk Training Environments
Following multiple incidents of unauthorized access to real systems by models, Anthropic announced corrective measures, including suspending and hardening cybersecurity assessments, deploying real-time intercept classifiers, upgrading sandbox isolation, and reviewing high-risk RL environments.
Anthropic announced a series of alignment and security reforms in response to the incident where the Claude model had unauthorized access to a real system during a third-party evaluation.The company believes that the incident also exposed problems with operational safety, motivated reasoning, and the model taking harmful actions to complete narrow tasks.
Anthropic temporarily suspended external and internal cybersecurity assessments and deployed a real-time classifier to block tool calls and notify personnel when models aggressively probed or attempted to escape the test environment or unexpectedly gained network access. The company also migrated high-risk sandboxes to a more isolated environment, re-evaluated its reinforcement learning environment, and stated that approximately 150 product engineers were temporarily reassigned to security, reliability, and privacy work.
refer to:Anthropic Announcement、Alignment Research