Description of problem:
I was trying to reproduce another bug, but let's see. On 21 Alpha:

Boot with installed F21 (with a tiny /boot)
Go to custom part, all defaults so far
Remove the existing F21 install, create a new layout with the 'automatic' button, then set the / mount point (*not* the VG) to be encrypted
Complete custom part, enter a passphrase
Go back to Installation Destination, this type check the box on ID that says you want encryption
Pick custom part again
Try to remove the existing installation again

Version-Release number of selected component:
anaconda-21.48.6-1

The following was filed automatically by anaconda:
anaconda 21.48.6-1 exception report
Traceback (most recent call first):
  File "/usr/lib/python2.7/site-packages/blivet/devices.py", line 2584, in _removeLogVol
    raise ValueError("specified lv is not part of this vg")
  File "/usr/lib/python2.7/site-packages/blivet/devicetree.py", line 450, in _removeDevice
    dev.vg._removeLogVol(dev)
  File "/usr/lib/python2.7/site-packages/blivet/devicetree.py", line 496, in registerAction
    self._removeDevice(action.device)
  File "/usr/lib/python2.7/site-packages/blivet/__init__.py", line 1296, in destroyDevice
    self.devicetree.registerAction(action)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/spokes/custom.py", line 1703, in _destroy_device
    self._storage_playground.destroyDevice(device)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/spokes/custom.py", line 122, in decorated
    return func(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/gui/spokes/custom.py", line 1826, in on_remove_clicked
    self._destroy_device(dev)
ValueError: specified lv is not part of this vg

Additional info:
addons:         com_redhat_kdump
cmdline:        /usr/bin/python  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-S-21_A-x86_64 quiet
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         3.16.1-301.fc21.x86_64
product:        Fedora-Server
release:        Cannot get release name.
type:           anaconda
version:        21_Alpha