Photoshop has always provided different ways to selectively and creatively blur our photos and artwork, creating effects that mimic the real-world behavior of our cameras, or more illustrative and surreal blurring styles. Which tool you use depends upon your purpose: for this example we'll stick to realistic photo retouching as our objective, with the idea that blurs can be made to look like they were handled in-camera.
Photoshop CS6 makes this task easier with the addition of the Blur Gallery, a feature that provides three different and intuitive methods for blurring a photograph. For the image below, my objective was to soften all of the details behind the flower, since they are not important except in providing a setting or frame of reference for the shot.
To create our blur effect, we'll use a combination of layer masking and the new Field Blur filter. Note that you can use the Iris Blur instead (also one of the three Blur Gallery options), which includes a built-in masking option (of sorts), but you can impart more control over the shape of the blurred area (and the transition between sharp and blur pixels) by using a layer mask with the globally applied Field Blur option.
The Lens Blur filter is also capable of using layer masks to define a blurred area, and you will gain more control over the appearance of the blurred pixels than either Blur Gallery option, but it can be slower (even though it's been optimized in CS6 for faster performance), and it's decidedly more complex.
The unblurred shot.
First, it's always a good idea to apply your blur to a separate layer from the original Background layer. For the time being, this feature is not Smart Filter aware. Once you've duplicated the layer, you'll need to create the layer mask. There are as many ways to create a mask, as there are ways of creating selections.
For this example, we'll use the Lasso tool to draw a simple selection area first, creating an arc above the flower. The reason a subtle arc is better than a straight line is that it's more in keeping with how cameras blur. Before you create the selection, expand the Feather value to a fairly large amount so there is a nice transition between masked and unmasked pixels. I chose a value of 60px here, but you may want to experiment to find the best value.
Draw the selection area around the area you wish to protect, then when it's finished, invert the selection. You can do this by pressing Command+Shift+I (Mac) or Control+Shift+I (Win). By doing this you're ensuring the mask (which is another way of saying 'select everything but these pixels so they're not changed by the filter) is protecting your subject and not the area you intend to blur.
Press Command+Shift+I (Mac) or Control+Shift+I (Windows) to invert the selection.
From there, make sure your duplicate layer is still selected, then click the Create Layer Mask button at the bottom of the Layers panel (third icon from the left). Next, we want to blur only the background here, so select the new layer mask by clicking it once, then choose the Brush tool (shortcut: B) from the Toolbar, and make sure the foreground color is set to black. Now you can paint around the sides and bottom of the mask, to make sure those feathered areas are completely masked from the blur effect.
Create a Layer Mask.
The next step is to select the layer icon again (i.e. make sure the layer mask is not selected when using a filter), then choose Filter > Blur > Field Blur. This will open the Blur Gallery into the Field Blur Mode, where you'll get a live preview of the effect and a few simple options.
Applying a Field Blur.
In the Options Bar, choose 'High Quality' to get the best preview and results, then set the Field Blur sider to a value that creates the amount of blur you need. Usually a modest amount goes a long way; I rarely need to set this value above 25. When you think you're at a good spot, take a look at the boundary of the blurred area. Look for any spots you might want to expand the mask into (or unmask) after you've applied the setting you want. When you're done click the OK button in the Options Bar.
Now that you've applied the blur, you can go back and tweak the mask by painting on it with the Brush tool some more. To expand the mask boundary, use a large soft brush and paint (black), with the edge of the brush over the areas where you want the new transition point to be. Some experimentation may be required but you can very quickly refine the blur effect with just a few brush strokes.
Use a large soft Brush tool to tweak the mask.
To get more creative, choose a middle shade of grey and 'spot paint' those details that you'd like to have partial focus or slightly more emphasis than the blurred areas. You can also select the layer mask and in the Properties panel, click the Mask Edge button to open the mask into the Refine Edge dialog, where you can preview the mask full size on your document to make further refinements. For example, you can expand or contract the entire mask boundary (maintaining the shape) by a specific number of pixels.
The Refine Mask dialog.
As long as you leave the layer mask intact and do not use the Apply Mask command to blend the mask effect into the layer itself you can go back and modify the mask as many times as you need to, until the effect area looks just the way you like it. The final image is shown below.