Segment Anything SAM: 2026

SAM (Segment Anything Model) by Meta, released in 2023, revolutionized image segmentation. SAM 2 (2024) extends capability to videos with object tracking. In 2026, it's the standard for accelerated annotation and interactive segmentation.

TL;DR

- SAM: universal image segmentation via prompts.

- SAM 2 (2024): extends to videos with tracking.

- Generalist model, works without specific training.

- Apache 2.0 open source, free models.

The problem SAM solves

Before SAM, image segmentation required a specific model per domain (medical, automotive, satellite), trained on laboriously annotated dataset (10K-100K masks). SAM is a generalist model trained on 1.1B masks that can segment anything with simple prompts:

• A point clicked on the object
• A bounding box
• A rough mask
• Text (via CLIP, SAM 2)

SAM architecture

Three components:

• Image encoder: ViT (Vision Transformer), generates dense image embedding
• Prompt encoder: encodes prompts (points, boxes, masks)
• Mask decoder: combines both to generate segmentation mask

Image encoder is heavy (~600 MB ViT-H), takes ~1 second on GPU. Prompts + decoder are ultra-fast (~50 ms). So encode image once, then iterate quickly with different prompts.

SAM 2 (2024): video

SAM 2 extends SAM to videos with object tracking:

• Click on an object in a frame
• SAM 2 propagates mask through video, tracking the object
• Memory module to handle occlusions and disappearances
• Streaming inference for long videos

2026 production use cases

• Accelerated annotation: 10x faster than manual mask annotation (SAM click = mask)
• Image editing: Photoshop, Pixelmator use SAM for magic selection
• Medical imaging: tumor, organ segmentation (with fine-tune)
• Satellite / drone: agricultural plot, building segmentation
• Robotics: segmentation for object manipulation
• Video editing: Adobe Premiere, DaVinci Resolve with SAM 2 for rotoscoping
• VFX: actor / object masking for effects

Model sizes

• SAM ViT-B: ~375 MB, faster
• SAM ViT-L: ~1.2 GB, balanced
• SAM ViT-H: ~2.5 GB, maximum quality (default)
• SAM 2: ~150-450 MB per variant (Hiera architecture)

Deployment

• Cloud: NVIDIA A10/A100 GPU recommended for low latency
• Edge: SAM Mobile (~30 MB) for smartphone via TFLite
• Offline: Roboflow, Encord integrate SAM in their annotation interfaces

FAQ

Q: Is SAM really generalist?

A: Yes for standard visual domains. Very specialized domains (medical MRI, electron microscopy) require fine-tuning.

Q: SAM vs U-Net / Mask R-CNN difference?

A: U-Net / Mask R-CNN = trained on specific dataset. SAM = promptable generalist. Often SAM in pre-annotation + fine-tuned U-Net in final prod.

Conclusion

SAM and SAM 2 in 2026 deeply transformed segmentation. Accelerated annotation, interactive segmentation, video tracking — all now accessible without training a specific model. For any production vision team, SAM became an indispensable tool, either in preprocessing (annotation) or directly in inference.