Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1776529 - Enable EROFS support in the kernel
Summary: Enable EROFS support in the kernel
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: rawhide
Hardware: All
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-25 22:04 UTC by fedora.dm0
Modified: 2020-01-13 16:49 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-13 16:49:50 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Package EROFS userspace utilities (869 bytes, text/x-matlab)
2019-11-25 22:04 UTC, fedora.dm0
no flags Details

Description fedora.dm0 2019-11-25 22:04:33 UTC
Created attachment 1639626 [details]
Package EROFS userspace utilities

This is an enhancement request to enable EROFS support in the kernel:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/erofs.txt

The Fedora kernel package's master branch has "# CONFIG_EROFS_FS is not set" in the config.  It was moved out of staging with 5.4, and the userspace utilities have a 1.0 release now, so I am interested in trying it as an option for uncompressed immutable images.

The userspace utilities aren't packaged in Fedora, so I'll attach a spec for it if it helps.

Comment 1 fedora.dm0 2019-12-03 18:04:49 UTC
To be clear, I'd prefer at least the following options so ACLs and SELinux can be used.

CONFIG_EROFS_FS=m
CONFIG_EROFS_FS_XATTR=y
CONFIG_EROFS_FS_POSIX_ACL=y
CONFIG_EROFS_FS_SECURITY=y

Comment 2 fedora.dm0 2019-12-22 15:29:16 UTC
Since 5.4 was allowed to promote to Fedora 31 now, I'd also like to request the feature be enabled in the f31 branch.

Comment 3 fedora.dm0 2019-12-31 05:26:06 UTC
Same for f30 now.

Comment 4 Gao Xiang 2020-01-07 05:36:28 UTC
Hi,

Just noticed this, and I'd also like if some chance on Fedora as well...

EROFS has already been enabled as a kernel module in openSUSE and Debian,
and here is the Debian configration:

https://salsa.debian.org/kernel-team/linux/commit/33639568475c399acc9cd9d45e9b06a3a2df6e75

##
## file: fs/erofs/Kconfig
##
CONFIG_EROFS_FS=m
# CONFIG_EROFS_FS_DEBUG is not set
CONFIG_EROFS_FS_XATTR=y
CONFIG_EROFS_FS_POSIX_ACL=y
CONFIG_EROFS_FS_SECURITY=y
CONFIG_EROFS_FS_ZIP=y
CONFIG_EROFS_FS_CLUSTER_PAGE_LIMIT=1

It'd be appreciated folks could take some time evaluating this filesystem for
more scenarios, raise up useful feature request proposals and contribute it
if have extra time...

I will devote myself to this all the time.

Thanks,
Gao Xiang

Comment 5 Dave Young 2020-01-08 02:02:48 UTC
Hi Gao Xiang,

Would you like to maintain the Fedora userspace pkg?  I think this needs two bugs, one for userspace pkg, another for the kernel options.

Thanks
Dave

Comment 6 Dave Young 2020-01-08 02:04:17 UTC
Oh, fedora.dm0 wrote a spec file, probably he/she want to maintain it

Comment 7 Dave Young 2020-01-08 02:05:33 UTC
Since this is for the kernel, so need a separate bug for the utils pkg review, please refer to below bz:
https://bugzilla.redhat.com/show_bug.cgi?id=1782532

Comment 8 Dave Young 2020-01-08 02:07:27 UTC
(In reply to Gao Xiang from comment #4)
> Hi,
> 
> Just noticed this, and I'd also like if some chance on Fedora as well...
> 
> EROFS has already been enabled as a kernel module in openSUSE and Debian,
> and here is the Debian configration:
> 
> https://salsa.debian.org/kernel-team/linux/commit/
> 33639568475c399acc9cd9d45e9b06a3a2df6e75
> 
> ##
> ## file: fs/erofs/Kconfig
> ##
> CONFIG_EROFS_FS=m
> # CONFIG_EROFS_FS_DEBUG is not set
> CONFIG_EROFS_FS_XATTR=y
> CONFIG_EROFS_FS_POSIX_ACL=y
> CONFIG_EROFS_FS_SECURITY=y
> CONFIG_EROFS_FS_ZIP=y
> CONFIG_EROFS_FS_CLUSTER_PAGE_LIMIT=1
> 
> It'd be appreciated folks could take some time evaluating this filesystem for
> more scenarios, raise up useful feature request proposals and contribute it
> if have extra time...
> 

Hi, we maintains kexec/kdump component in Fedora, and currently we use squashfs for kdump initrd to save memory, probably we can try erofs see if it runs better.

Comment 9 fedora.dm0 2020-01-08 02:37:23 UTC
(In reply to Dave Young from comment #6)
> Oh, fedora.dm0 wrote a spec file, probably he/she want to maintain it

No thanks, I build the tools locally.  This is a request for the kernel so I can use the produced images on Fedora.  Someone can take the attached spec file if they want to maintain it for Fedora.

Comment 10 Gao Xiang 2020-01-08 03:16:06 UTC
(In reply to Dave Young from comment #5)
> Hi Gao Xiang,
> 
> Would you like to maintain the Fedora userspace pkg?  I think this needs two
> bugs, one for userspace pkg, another for the kernel options.
> 
> Thanks
> Dave

Hi Dave,

Thanks for your reply.

I'm not familiar with the whole .rpm stuffs but it seems openSUSE has accepted erofs-utils as well...

I'm currently using Debian and maintaining its .deb package.
If David has some interest in that, I'm happy that he could take some time on this package.

Hi David,
Is there some chance on this userspace package?
I will fix all the reported issues in upstreaming so I think it would be easy to maintaining this package for Fedora...

> Hi, we maintains kexec/kdump component in Fedora, and currently we use squashfs for kdump initrd to save memory, probably we can try erofs see if it runs better.

EROFS still isn't good at high compression ratio scenarios yet, I'm struggling in developping fixed-sized output XZ (LZMA2) algorithm.
So for now I think the dynamic performance could be better, but I'm not sure if the current compression ratio meets your original requirement.
But anyway, high CR algorithm supports are on the way...

Thanks,
Gao Xiang

Comment 11 Dave Young 2020-01-08 03:27:21 UTC
> EROFS still isn't good at high compression ratio scenarios yet, I'm
> struggling in developping fixed-sized output XZ (LZMA2) algorithm.
> So for now I think the dynamic performance could be better, but I'm not sure
> if the current compression ratio meets your original requirement.
> But anyway, high CR algorithm supports are on the way...

Thanks for the explanation, good to know about the plan :)

Comment 12 Gao Xiang 2020-01-08 03:33:00 UTC
(In reply to Dave Young from comment #11)
> > EROFS still isn't good at high compression ratio scenarios yet, I'm
> > struggling in developping fixed-sized output XZ (LZMA2) algorithm.
> > So for now I think the dynamic performance could be better, but I'm not sure
> > if the current compression ratio meets your original requirement.
> > But anyway, high CR algorithm supports are on the way...
> 
> Thanks for the explanation, good to know about the plan :)

I'm doing that in my spare time, mainly due to enriching community ecosystem
rather than our products (so rare company support).

LZMA fixed-sized output compression can be implemented in principle, I'm coding
now (range coder and LZ matchfinder is almost ready), and need debugging.
Hopefully can be done first half past of this year.

Thanks,
Gao Xiang

Comment 13 Kairui Song 2020-01-08 04:35:03 UTC
(In reply to Gao Xiang from comment #12)
> (In reply to Dave Young from comment #11)
> > > EROFS still isn't good at high compression ratio scenarios yet, I'm
> > > struggling in developping fixed-sized output XZ (LZMA2) algorithm.
> > > So for now I think the dynamic performance could be better, but I'm not sure
> > > if the current compression ratio meets your original requirement.
> > > But anyway, high CR algorithm supports are on the way...
> > 
> > Thanks for the explanation, good to know about the plan :)
> 
> I'm doing that in my spare time, mainly due to enriching community ecosystem
> rather than our products (so rare company support).
> 
> LZMA fixed-sized output compression can be implemented in principle, I'm
> coding
> now (range coder and LZ matchfinder is almost ready), and need debugging.
> Hopefully can be done first half past of this year.
> 
> Thanks,
> Gao Xiang

Hi, thanks for the work, I'll definitely try it out when that's ready.

Will it have a higher compression ratio and less memory usage than squashfs? kdump maybe benefit from it.

Comment 14 Gao Xiang 2020-01-08 05:31:43 UTC
(In reply to Kairui Song from comment #13)
> (In reply to Gao Xiang from comment #12)
> > (In reply to Dave Young from comment #11)
> > > > EROFS still isn't good at high compression ratio scenarios yet, I'm
> > > > struggling in developping fixed-sized output XZ (LZMA2) algorithm.
> > > > So for now I think the dynamic performance could be better, but I'm not sure
> > > > if the current compression ratio meets your original requirement.
> > > > But anyway, high CR algorithm supports are on the way...
> > > 
> > > Thanks for the explanation, good to know about the plan :)
> > 
> > I'm doing that in my spare time, mainly due to enriching community ecosystem
> > rather than our products (so rare company support).
> > 
> > LZMA fixed-sized output compression can be implemented in principle, I'm
> > coding
> > now (range coder and LZ matchfinder is almost ready), and need debugging.
> > Hopefully can be done first half past of this year.
> > 
> > Thanks,
> > Gao Xiang
> 
> Hi, thanks for the work, I'll definitely try it out when that's ready.
> 
> Will it have a higher compression ratio and less memory usage than squashfs?
> kdump maybe benefit from it.

I will send out XZ RFC solation after it is worth to be tried out.

My current estimate is
On lower compress cluster size it would have higher compression ratio than squashfs,
On larger compress cluster size compression ratio of EROFS is close to squashfs.

Due to in-place decompression (no extra memory allocation and memcpy), its dynamic
memory overhead is smaller than squashfs.

Thanks,
Gao Xiang

Comment 15 Justin M. Forbes 2020-01-09 17:11:05 UTC
I will enable this in rawhide for the next build, it will trickle down to F31 and F30 as the 5.5 rebases happen, which should give some time to get the userspace utilities built and sorted.

Comment 16 fedora.dm0 2020-01-09 17:27:39 UTC
I submitted the spec in #1789492.

Comment 17 fedora.dm0 2020-01-10 13:46:11 UTC
This can probably be closed since the kernel functionality is enabled in rawhide, unless it's supposed to be blocked by the userspace package.

Comment 18 fedora.dm0 2020-01-13 16:49:50 UTC
The userspace tools have been added, and the kernel support will promote from rawhide, so I'll close this.


Note You need to log in before you can comment on or make changes to this bug.