After booting the system to check:

[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Password is correct:

[root@fitlet2 ~]# cryptsetup luksOpen --test-passphrase --key-slot 0 /dev/sda3 && echo correct
Enter passphrase for /dev/sda3:
correct

Remove the password and try again:

[root@fitlet2 ~]# clevis luks unbind -d /dev/sda3 -s 1
The unbind operation will wipe a slot. This operation is unrecoverable.
Do you wish to erase LUKS slot 1 on /dev/sda3? [ynYN] y
Enter any remaining passphrase:

[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1 inactive empty
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Try to use the clevis commands again:

[root@fitlet2 ~]# UUID=$(lsblk | grep luks | sed 's/^.*luks-//' | cut -d ' ' -f1)
[root@fitlet2 ~]# DEV=$(blkid --uuid $UUID)
[root@fitlet2 ~]# echo $DEV
/dev/sda3
[root@fitlet2 ~]# clevis luks bind -f -k- -d $DEV tpm2 '{}' <<< fedora
[root@fitlet2 ~]# luksmeta show -d /dev/sda3
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

Reboot and it again fails to unlock. The same commands work on a standard F29/30 installation.

(In reply to Paul Whalen from comment #0)

[snip]

> 
> Works when using a standard Fedora 30 installation with the same package
> versions and kickstart snippets.
> 
> Clevis is included in the initramfs for IoT:
> 
> lsinitrd initramfs-5.0.0-0.rc8.git0.1.fc30.x86_64.img | grep clevis
> clevis

Are the tpm2-tools and libraries also included in the initramfs? i.e:

lsinitrd /boot/initramfs-$(uname -r).img | grep -E 'clevis|tpm2|tss2'

lsinitrd initramfs-5.0.0-300.fc30.x86_64.img | grep -E 'clevis|tpm2|tss2'

clevis
-rw-r--r--   1 root     root            1 Feb 14 14:03 etc/cmdline.d/99clevis.conf
-rwxr-xr-x   1 root     root         1563 Aug 13  2018 usr/bin/clevis
-rwxr-xr-x   1 root     root         1614 Aug 13  2018 usr/bin/clevis-decrypt
-rwxr-xr-x   1 root     root        33304 Jan 31 09:46 usr/bin/clevis-decrypt-sss
-rwxr-xr-x   1 root     root         2721 Aug 13  2018 usr/bin/clevis-decrypt-tang
-rwxr-xr-x   1 root     root         3794 Aug 13  2018 usr/bin/clevis-decrypt-tpm2
-rwxr-xr-x   1 root     root        95408 Feb  3 04:10 usr/bin/tpm2_createprimary
-rwxr-xr-x   1 root     root        92712 Feb  3 04:10 usr/bin/tpm2_load
-rwxr-xr-x   1 root     root        87968 Feb  3 04:10 usr/bin/tpm2_pcrlist
-rwxr-xr-x   1 root     root       108424 Feb  3 04:10 usr/bin/tpm2_unseal
-rwxr-xr-x   1 root     root       303880 Feb 14 14:03 usr/lib64/libtss2-mu.so.0.0.0
lrwxrwxrwx   1 root     root           19 Feb 14 14:03 usr/lib64/libtss2-mu.so.0 -> libtss2-mu.so.0.0.0
-rwxr-xr-x   1 root     root       403216 Feb 14 14:03 usr/lib64/libtss2-sys.so.0.0.0
lrwxrwxrwx   1 root     root           20 Feb 14 14:03 usr/lib64/libtss2-sys.so.0 -> libtss2-sys.so.0.0.0
-rwxr-xr-x   1 root     root        36416 Feb 14 14:03 usr/lib64/libtss2-tcti-device.so.0.0.0
lrwxrwxrwx   1 root     root           28 Feb 14 14:03 usr/lib64/libtss2-tcti-device.so.0 -> libtss2-tcti-device.so.0.0.0
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/online/60-clevis-hook.sh
-rwxr-xr-x   1 root     root           49 Jan 31 09:46 usr/lib/dracut/hooks/initqueue/settled/60-clevis-hook.sh
-rwxr-xr-x   1 root     root         2873 Aug 13  2018 usr/libexec/clevis-luks-askpass

With Fedora-IoT-ostree-x86_64-30-20190215.0.iso I get:

# echo foobar | clevis encrypt tpm2 '{}' | clevis decrypt
WARNING:tcti:src/tss2-tcti/tcti-device.c:254:tcti_device_receive() Got EOF instead of response. 
ERROR: 
Create Object Failed ! ErrorCode: 0xa0008

ERROR: Unable to run tpm2_create
Creating TPM2 object for jwk failed!

# tpm2_rc_decode 0xa0008
error layer
  hex: 0xa0000
  identifier: TSS2_TCTI_RC_LAYER
  description: Error from the TCTI
base error code
  identifier: TSS2_BASE_RC_NO_CONNECTION
  description: Fails to connect to next lower layer

# uname -r
5.0.0-0.rc6.git1.1.fc30.x86_64

Which is reported to be a regression in the kernel:

https://github.com/tpm2-software/tpm2-tools/issues/1356

Following is the thread in the linux-integrity mailing list where the problem and possible solutions are being discussed:

https://www.spinics.net/lists/linux-integrity/msg06386.html

Upstream patch that solves this bug:

https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1959715.html

kernel-tools-5.0.4-300.fc30 kernel-headers-5.0.4-300.fc30 kernel-5.0.4-300.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-4e7590b99c

kernel-tools-5.0.4-200.fc29 kernel-headers-5.0.4-200.fc29 kernel-5.0.4-200.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-0ba1e6642f

kernel-5.0.4-300.fc30, kernel-headers-5.0.4-300.fc30, kernel-tools-5.0.4-300.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-4e7590b99c

kernel-5.0.4-200.fc29, kernel-headers-5.0.4-200.fc29, kernel-tools-5.0.4-200.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-0ba1e6642f

kernel-5.0.4-200.fc29, kernel-headers-5.0.4-200.fc29, kernel-tools-5.0.4-200.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

kernel-5.0.4-300.fc30, kernel-headers-5.0.4-300.fc30, kernel-tools-5.0.4-300.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Using compose Fedora-IoT-30-20190327.0

[root@fitlet2 ~]# echo foo | clevis encrypt tpm2 '{}' | clevis decrypt
foo
[root@fitlet2 ~]# uname -r
5.0.0-300.fc30.x86_64

But the automatic decryption still fails on boot. If I add 'rd.break=initqueue' to the kernel args to get a shell prior to decryption, then 'exit' the system will decrypt the volume and boot to log in. It appears to be working to some degree but perhaps a timing issue?

This also happens with a F29 compose which includes kernel-4.20.16-200.fc29.

Created attachment 1555593 [details]
tpm2-luks1-ks.cfg

Created attachment 1555594 [details]
tpm2-luks2-ks.cfg

I tried to reproduce this issue on the same hardware than Paul used and couldn't reproduce the exact issue but had a different one that could be related.

Using the kickstart tpm2-luks1-ks.cfg file attached in Comment 14, the system was installed correctly but the LUKS volume was not unlocked automatically by clevis. Looking at the LUKSv1 and LUKSMeta headers, I noticed that a LUKSMeta slot was added for the bound pin but no a LUKS slot:

$ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e

$ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
Key Slot 0: ENABLED

But if I bind a pin to a LUKS volume after the installation, it did work correctly:

$ sudo luksmeta wipe -d /dev/sda3 -s 1

$ sudo clevis luks bind -d /dev/sda3 tpm2 '{}'

$ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e

$ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
Key Slot 0: ENABLED
Key Slot 1: ENABLED

And also the LUKS volume was automatically unlocked on boot.

It also worked correctly if I used LUKSv2 instead of LUKSv1 like in the tpm2-luks2-ks.cfg kickstart file attached in Comment 15.

So the problem seems to be only when storing the clevis pin on a LUKSv1 volume and when the clevis luks bind command is executed in the kickstart file.

I've been having a similar issue on Fedora Silverblue, as this seems to still be open I thought I'd post it here rather than open a new bug report.

It seems to be that the required initramfs modules are not loaded by the layer provided by what I assume would be the "clevis-dracut" package (if that's not the correct terminology let me know as I only installed Silverblue yesterday to play around with).

Enabling client side generation of the initramfs via "rpm-ostree initramfs --enable" resolves this.

(In reply to Javier Martinez Canillas from comment #16)
> I tried to reproduce this issue on the same hardware than Paul used and
> couldn't reproduce the exact issue but had a different one that could be
> related.
> 
> Using the kickstart tpm2-luks1-ks.cfg file attached in Comment 14, the
> system was installed correctly but the LUKS volume was not unlocked
> automatically by clevis. Looking at the LUKSv1 and LUKSMeta headers, I
> noticed that a LUKSMeta slot was added for the bound pin but no a LUKS slot:
> 
> $ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
> 0   active empty
> 1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
> 
> $ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
> Key Slot 0: ENABLED
> 
> But if I bind a pin to a LUKS volume after the installation, it did work
> correctly:
> 
> $ sudo luksmeta wipe -d /dev/sda3 -s 1
> 
> $ sudo clevis luks bind -d /dev/sda3 tpm2 '{}'
> 
> $ sudo luksmeta show -d /dev/sda3 | grep "^[0-9][ ]*active"
> 0   active empty
> 1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
> 
> $ sudo cryptsetup luksDump /dev/sda3 | grep ENABLED
> Key Slot 0: ENABLED
> Key Slot 1: ENABLED
> 
> And also the LUKS volume was automatically unlocked on boot.
> 
> It also worked correctly if I used LUKSv2 instead of LUKSv1 like in the
> tpm2-luks2-ks.cfg kickstart file attached in Comment 15.
> 
> So the problem seems to be only when storing the clevis pin on a LUKSv1
> volume and when the clevis luks bind command is executed in the kickstart
> file.

Javier: Can you still reproduce this issue?

(In reply to Sergio Correia from comment #19)
> (In reply to Javier Martinez Canillas from comment #16)

[snip]

> > 
> > So the problem seems to be only when storing the clevis pin on a LUKSv1
> > volume and when the clevis luks bind command is executed in the kickstart
> > file.
> 
> Javier: Can you still reproduce this issue?

I don't have access to the machine where I was able to reproduce the issue.

But you could ask Paul Whalen or Peter Robinson if they are still facing issues with clevis and TPM2 devices in the Fedora IoT spin.

This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

I have a similar issue happen earlier this year. I believe it can be the same (or at least similar) cause.

Description of my problem:
Given I have my home directory on luks volume with Network-Bound Disk Encryption configured (clevis + tang, tang is on RHEL-7) on my laptop. Earlier this year after updating a big couple of software in my system (for both my RHEL-7 and my laptop), I can no longer unlock my disk. 

I cannot really debug a lot since the tang server is in the office, while we are not allowed to go back to the office at the moment.
Today I got my laptop from office with the help of a colleague. I tried to downgrade either clevis or cryptsetup, and also booting into older kernels, system cannot unlock my home partition during boot even if I typed the right password.

From the log on the screen, I find something suspicious

May 06 18:13:50 systemd[1]: Created slice system-dbus\x2d:1.2\x2dorg.freedesktop.problems.slice.
May 06 18:13:50 systemd[1]: Started dbus-:1.2-org.freedesktop.problems.
May 06 18:13:50 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-:1.2-org.freedesktop.problems@0 comm="systemd" exe="/usr/l>
May 06 18:13:50 systemd-cryptsetup[5839]: Failed to activate with specified passphrase. (Passphrase incorrect?)
May 06 18:13:50 systemd-tty-ask-password-agent[5841]: Failed to query password: Connection refused
May 06 18:13:50 systemd-tty-ask-password-agent[5841]: Failed to show password: Connection refused
May 06 18:13:50 systemd[1]: Starting Clevis LUKS systemd-ask-password Responder...
May 06 18:13:50 abrt-notification[5965]: Process 487 (plymouthd) crashed in _nl_load_domain.cold()
May 06 18:13:50 clevis-luks-askpass[5966]: Error communicating with the server!
May 06 18:13:50 systemd-cryptsetup[5839]: Set cipher aes, mode xts-plain64, key size 256 bits for device /dev/fedora/home.
May 06 18:13:50 systemd[1]: clevis-luks-askpass.service: Succeeded.
May 06 18:13:50 systemd[1]: Started Clevis LUKS systemd-ask-password Responder.
May 06 18:13:50 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=clevis-luks-askpass comm="systemd" exe="/usr/lib/systemd/system>
May 06 18:13:50 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=clevis-luks-askpass comm="systemd" exe="/usr/lib/systemd/systemd>
May 06 18:13:53 systemd-cryptsetup[5839]: Failed to activate with specified passphrase. (Passphrase incorrect?)
May 06 18:13:53 systemd-cryptsetup[5839]: Too many attempts to activate; giving up.
May 06 18:13:53 systemd[1]: systemd-cryptsetup: Main process exited, code=exited, status=1/FAILURE
May 06 18:13:53 systemd[1]: systemd-cryptsetup: Failed with result 'exit-code'.
May 06 18:13:53 systemd[1]: Failed to start Cryptography Setup for home.
May 06 18:13:53 systemd[1]: Dependency failed for /dev/mapper/home.
May 06 18:13:53 systemd[1]: Dependency failed for File System Check on /dev/mapper/home.
May 06 18:13:53 systemd[1]: Dependency failed for /home.
May 06 18:13:53 systemd[1]: Dependency failed for Remote File Systems.

The log do shows failed to activate with specified password at least twice, however I only type the password once before it shows too many attempts.

Workaround:

I tried to boot without my /home and then totally remove clevis-dracut and clevis-systemd, dracut -f. Then I can unlock it with password after reboot.

Steps of my setup is mostly here https://bugzilla.redhat.com/show_bug.cgi?id=1574413#c0

Additional info:

$ sudo cryptsetup luksDump /dev/fedora/home | egrep -i "enable|version"
Version:        1
Key Slot 0: ENABLED
Key Slot 1: ENABLED
$ sudo luksmeta show -d /dev/fedora/home 
0   active empty
1   active cb6e8904-81ff-40da-a84a-07ab9ab5715e
2 inactive empty
3 inactive empty
4 inactive empty
5 inactive empty
6 inactive empty
7 inactive empty

$ rpm -q clevis dracut cryptsetup
clevis-12-1.fc31.x86_64
dracut-050-26.git20200316.fc31.x86_64
cryptsetup-2.3.0-1.fc31.x86_64
$ uname -r
5.6.8-200.fc31.x86_64

I'd like to help during my spare time if you need some more information and if it is not destructive. But again since the tang server is in the office, I cannot help test the tang related part.

Hi Zamir,

(In reply to Zamir SUN from comment #22)
> I have a similar issue happen earlier this year. I believe it can be the
> same (or at least similar) cause.
> 

[...]
 
> From the log on the screen, I find something suspicious
> 
> May 06 18:13:50 systemd[1]: Created slice
> system-dbus\x2d:1.2\x2dorg.freedesktop.problems.slice.
> May 06 18:13:50 systemd[1]: Started
> dbus-:1.2-org.freedesktop.problems.
> May 06 18:13:50 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295
> ses=4294967295 subj=system_u:system_r:init_t:s0
> msg='unit=dbus-:1.2-org.freedesktop.problems@0 comm="systemd" exe="/usr/l>
> May 06 18:13:50 systemd-cryptsetup[5839]: Failed to activate with specified
> passphrase. (Passphrase incorrect?)

This looks suspicious. Are you able to unlock the device with say clevis luks unlock -d <device>?

Paul: would you be able to check whether the issue reported here is still reproducible with clevis-14?

Hi Sergio, 

This issue seemed to be the specific hardware I was using, but I am happy to report it is now working in Fedora IoT. We also have an automated test in OpenQA with swtpm which is also working. Closing as fixed. Thanks!

I think I am running into this same issue in Red Hat 8.3 Edge.

The encrypted partitions I set up during kickstart installation are not automatically unlocked by the clevis-luks-askpass@ service. 
But if I execute the binding again, once the OS is deployed, then the service can automatically unlock them.

My kickstart is something like this

%post --erroronfail --log=/root/disk-encryption-configuration.log
set -euxo pipefail

lvcreate -y -l 20%VG -n opt system
printf "password" | cryptsetup -q luksFormat /dev/system/opt -d -
printf "password" | clevis luks bind -f -k- -d /dev/system/opt tpm2 '{}'
printf "password" | cryptsetup luksOpen /dev/system/opt c1 -d -
mkfs.ext4 /dev/mapper/c1
sleep 1
cryptsetup luksClose c1
echo "luks-system-opt /dev/system/opt none _netdev" >> /etc/crypttab
echo "/dev/mapper/luks-system-opt /var/opt ext4 defaults,_netdev  0 2" >> /etc/fstab
%end

When the service tries to unlock it, it complains about a TPM integrity policy.
And after I execute this line again, once the OS is deployed, 

printf "password" | clevis luks bind -f -k- -d /dev/system/opt tpm2 '{}'
Then the service can unlock it during boot. I guess it tries the first slot and fails, and then succeed with the second slot.
I am using rhel-8.3-beta-1-x86_64-boot.iso to install. 
Should this fix be included in that boot iso?

(In reply to nicolasoliver03 from comment #26)
> I think I am running into this same issue in Red Hat 8.3 Edge.
> 

[...] 

> When the service tries to unlock it, it complains about a TPM integrity
> policy.
> And after I execute this line again, once the OS is deployed, 

This seems to be a different issue. Would you mind opening a new BZ for tracking this?

Done, see bug 1877470

Thanks Sergio!