Why wasn't this done well before Final freeze? Changes are supposed to be 'code complete' by *Beta*.

For each Fedora release, we always target the final freeze to get all of our builds done.  This is necessary due to how the upstream release schedule aligns with Fedora's and because the upstream API does not finalize until the final release (making packaging earlier release candidates difficult).

We have been building and testing release candidates in COPR for the last two months, getting the final builds into Fedora 38 is just the last step of a longer process.  Right now we have most of the builds done and there are just 2 or 3 left to do.

The contingency deadline of the 'Final Freeze' was documented in the change proposal: https://fedoraproject.org/wiki/Changes/LLVM-16 and we re-iterated our Final Freeze goal to FESCO in a recent status update: https://pagure.io/fesco/issue/2958#comment-843760

We do our best to get the builds done on time, but occasionally we miss and need a freeze exception.  The last time this happened was for Fedora 35: https://bugzilla.redhat.com/show_bug.cgi?id=2072077

That time, the update had already been in updates-testing for a while. This time there is nothing in updates-testing.

It's just not an appropriate development process to be landing major changes after the final freeze. That's not how we build stuff. I appreciate that there is a schedule problem here, but the appropriate solution for that is not "let's land major new versions of stuff during Fedora final freeze". There needs to be a better solution, like llvm stabilizing its API during the pre-release phase so we can land a pre-release earlier in the cycle and only have to bump to the final release later.

Discussed during the 2023-04-03 blocker review meeting: [0]

The decision to delay the classification of this as a blocker bug was made as we do not have a clear vote on this (we are at +2 / -3 for a total of -1), so we will punt it. Note: this is effectively close to a rejection, as we are very unlikely to accept this later if we don't accept it now.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2023-04-03/f38-blocker-review.2023-04-03-16.01.txt

(In reply to Geoffrey Marr from comment #4)
> Discussed during the 2023-04-03 blocker review meeting: [0]
> 
> The decision to delay the classification of this as a blocker bug was made
> as we do not have a clear vote on this (we are at +2 / -3 for a total of
> -1), so we will punt it. Note: this is effectively close to a rejection, as
> we are very unlikely to accept this later if we don't accept it now.
> 
> [0]
> https://meetbot.fedoraproject.org/fedora-blocker-review/2023-04-03/f38-
> blocker-review.2023-04-03-16.01.txt

When is the next meeting?

(In reply to Adam Williamson from comment #3)
> That time, the update had already been in updates-testing for a while. This
> time there is nothing in updates-testing.
> 
> It's just not an appropriate development process to be landing major changes
> after the final freeze. That's not how we build stuff. I appreciate that
> there is a schedule problem here, but the appropriate solution for that is
> not "let's land major new versions of stuff during Fedora final freeze".

I'm not sure this comment is really fair.  Our solution involved landing the changes
before the final freeze as documented in the change proposal.  We weren't originally
planning to push the changes in after the freeze.  It just turned out to be more work
than expected so we missed the deadline.

> There needs to be a better solution, like llvm stabilizing its API during
> the pre-release phase so we can land a pre-release earlier in the cycle and
> only have to bump to the final release later.

We used to do it this way, but it turned out to be way too difficult to coordinate all
the rebuilds of the dependent packages.  That is why we switched to our current method
where we do all the builds and testing for the pre-releases in COPR and then do the final
builds in koji once we feel like everything is ready.

At the end of each release we look at our process and try to find ways to improve it,
so if anyone has suggestions for how to improve what we do, we are happy to listen.

> When is the next meeting?

Next Monday. Async voting is possible in the pagure ticket - https://pagure.io/fedora-qa/blocker-review/issue/1136 .

>> I appreciate that
>> there is a schedule problem here, but the appropriate solution for that is
>> not "let's land major new versions of stuff during Fedora final freeze".

> I'm not sure this comment is really fair.  Our solution involved landing the changes
> before the final freeze as documented in the change proposal.  We weren't originally
> planning to push the changes in after the freeze.  It just turned out to be more work
> than expected so we missed the deadline.

I understand the behind-the-scenes, but what it *boils down to* is trying to land a new major version during the final freeze. The way the Change process is supposed to work is that if you miss the deadline, the Change gets put off to the next release. The way the freeze policy works is we don't change stuff during freezes unless there's a really good reason to.

(In reply to Adam Williamson from comment #7)
> > When is the next meeting?
> 
> Next Monday. Async voting is possible in the pagure ticket -
> https://pagure.io/fedora-qa/blocker-review/issue/1136 .
> 

OK, I won't be able to attend the next meeting. Where is the best place for me to leave feedback, here or on the pagure  ticket?

Either works. Theoretically the idea is that discussion of the bug per se goes here and discussion of blocker/FE-ness goes in the ticket, but when the bug is purely a blocker/FE bureaucracy bug, obviously the distinction barely exists :)

FEDORA-2023-3a602914f6 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-3a602914f6

Please include iwyu-0.20-1.fc38 build to LLVM 16 Bodhi update. I built and tagged it to this side tag.

FEDORA-2023-3a602914f6 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-3a602914f6

Would it be possible to extend this FE to this Bodhi update too? https://bodhi.fedoraproject.org/updates/FEDORA-2023-07c0c553a1
This updates qt-creator, removing an unnecessary dependency that previous package versions had on LLVM packages.

(In reply to Tulio Magno Quites Machado Filho from comment #13)
> Would it be possible to extend this FE to this Bodhi update too?
> https://bodhi.fedoraproject.org/updates/FEDORA-2023-07c0c553a1
> This updates qt-creator, removing an unnecessary dependency that previous
> package versions had on LLVM packages.

I'd probably propose that as a separate FE, but it should be possible to mark that update as fixing this bug too, afaik.

FEDORA-2023-3a602914f6 has been pushed to the Fedora 38 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-3a602914f6

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

+5 in https://pagure.io/fedora-qa/blocker-review/issue/1136 , marking accepted.

FEDORA-2023-3a602914f6 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-3a602914f6

FEDORA-2023-3a602914f6 has been pushed to the Fedora 38 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-3a602914f6

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2023-3a602914f6 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.