Photoshop’s AI tools have evolved far beyond the one-click Select Subject. Whilst that feature is great for quick selections, advanced masking—think intricate hair, complex backgrounds, or multiple subjects—requires tapping into deeper AI capabilities like the Select and Mask workspace, Object Selection Tool, and Generative Fill refinements. In this tutorial, we’ll mask a person with flyaway hair against a busy background, pushing Photoshop’s AI to its limits. Let’s unlock precision masking like never before!
What You’ll Need
- Adobe Photoshop (2025 version recommended for latest AI updates)
- A sample image (e.g., a person with detailed hair against a textured backdrop like trees or a crowd)
Step-by-Step Guide
Step 1: Open Your Image and Convert to a Smart Object
Load your image in Photoshop. Right-click the Background layer in the Layers panel and select “Convert to Smart Object.” This keeps edits non-destructive, letting you tweak AI filters later.

Step 2: Start with Object Selection Tool (Not Just Select Subject)
Go to the Toolbar, pick the Object Selection Tool (nested under Quick Selection, shortcut W
). In the Options bar, set Mode to “Lasso” and draw a rough outline around your subject.

Photoshop’s AI will snap to edges, offering more control than Select Subject’s blanket approach.

Step 3: Enter Select and Mask Workspace
With your initial selection active, click “Select and Mask” in the Options bar (or Ctrl+Alt+R
/ Cmd+Option+R
). This opens the AI-powered workspace.

In the Properties panel, set View Mode to “On Black” (shortcut A
) to spot edge issues clearly.

Step 4: Refine with AI-Driven Edge Detection
In Select and Mask, click “Refine Hair” in the Options bar—Photoshop’s AI kicks in to tackle tricky hair strands.

Then, grab the Refine Edge Brush Tool (shortcut R
) and paint over flyaway hair or fuzzy edges.

Adjust the Radius slider (e.g., 3-5 px) in the Edge Detection section for precision.

Step 5: Use AI Masking in Camera Raw for Extra Control
Exit Select and Mask by outputting to a “New Layer with Layer Mask.” Now, go to Filter > Camera Raw Filter
.
In the Camera Raw workspace, click the Masking icon (right sidebar), then choose “Select People.” Photoshop’s AI detects facial features, hair, or clothing—pick “Hair” to refine further. Adjust sliders like Feather or Contrast.


Step 6: Fix Gaps with Generative Fill
Back in Photoshop, inspect your mask. If gaps or halos remain (e.g., background peeking through hair), select the masked layer, grab the Lasso Tool (L
), and draw around problem areas. Right-click, choose “Generative Fill,” leave the prompt blank, and hit Generate. Pick the best of three AI-generated fills to analyse and apply.

Step 7: Finalise and Compare
Toggle the mask visibility (Shift+click
the mask thumbnail) to compare your result. Save as a PSD to keep layers intact, then export as a PNG for transparency. You’ve now masked beyond Select Subject with AI precision!

Pro Tips
- Smart Radius: In Select and Mask, toggle “Smart Radius” for varied edge thickness on complex subjects.
- Decontaminate Colours: Tick this in Output Settings to remove background colour fringes.
- Multiple Subjects: Repeat Steps 2-5 per subject, using “Mask All Objects” (
Layer > Mask All Objects
) for group shots.
Why Go Beyond Select Subject?
Select Subject is fast but lacks finesse for intricate details. Combining Object Selection, Select and Mask, Camera Raw AI, and Generative Fill gives you surgical precision – perfect for pro-level composites or retouching.