ASAM - An AI image segmentation model developed by vivo.

What is ASAM?

ASAM (Adversarial Adjustment of Segment Anything Model) is an AI image segmentation model launched by vivo. It enhances the performance of the original SAM model through adversarial adjustments. ASAM uses natural adversarial examples and a stable diffusion model to augment the dataset, generating adversarial instances that represent natural variations. These instances maintain photorealistic quality and are aligned with the original mask annotations, ensuring the integrity of the segmentation task.

ASAM's main functions

1. Enhance capabilitiesASAM makes the original SAM model perform better on image segmentation tasks.
2. No need to changeIt doesn't require changing the basic structure of the model, just like changing a car's tires while keeping the body the same.
3. Increase the challengeBy adding some special images (adversarial samples), the model learns to work in more different situations.
4. More realisticThese special images look just like real ones, allowing the model to learn to be more realistic.
5. No new data neededNo additional pictures or data are needed; you can learn new knowledge just like reviewing an old book.
6. Overall improvementASAM enables models to perform better on a variety of different image segmentation tasks.

ASAM's technical principles

1. competitive trainingASAM utilizes the concept of adversarial training, a method that improves the generalization ability of a model by inputting carefully designed perturbations into the model.
2. Examples of natural antagonismASAM pays special attention to natural adversarial examples, which are visually similar to normal samples and are located near the model's decision boundary.
3. Stable diffusion modelASAM uses a stable diffusion model to augment a subset of the SA-1B dataset, generating adversarial instances. This method introduces subtle perturbations while maintaining image quality.
4. PhotorealisticThe generated adversarial samples need to maintain the same visual realism as the original images so that the model will not learn incorrect features due to unrealistic samples during training.
5. Mask annotation alignmentAdversarial samples need to be aligned with the original mask annotations to ensure that the model can correctly identify and segment target objects in the segmentation task.
6. Fine-tuning processASAM improves model performance by fine-tuning the original SAM model using these adversarial examples without changing the original architecture.

ASAM's project address

ASAM Application Scenarios

• Medical image analysisIn the medical field, ASAM can be used to improve the accuracy of medical image segmentation, assisting doctors in disease diagnosis and treatment planning.
• self-driving carsIn autonomous driving technology, ASAM can be used to improve the segmentation accuracy of roads and objects, and enhance the vehicle's understanding of its surrounding environment.
• Robot VisionIn robotics, ASAM can help robots better understand their working environment and perform accurate object recognition and manipulation.
• Augmented Reality (AR)In AR applications, ASAM can enhance the naturalness of the integration of virtual objects with the real world, providing a more immersive experience.