Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.
Bug 1432219 - dnf in rawhide uses much more RAM than dnf in Fedora 25
Summary: dnf in rawhide uses much more RAM than dnf in Fedora 25
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 27
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: TRACKER-bugs-affecting-libguestfs 1432213
TreeView+ depends on / blocked
 
Reported: 2017-03-14 19:45 UTC by Richard W.M. Jones
Modified: 2017-11-07 11:06 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-07 11:06:36 UTC
Type: Bug
Embargoed:

Attachments (Terms of Use)

Description Richard W.M. Jones 2017-03-14 19:45:16 UTC 
Description of problem:

... causing this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1432213

libguestfs runs dnf in an appliance with a mere 512 MB of RAM.  Previously that worked
fine, but in current Rawhide, everything fails with errors like:

[  204.467753] dnf invoked oom-killer: gfp_mask=0x24201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=0, order=0, oom_score_adj=0
[  204.470038] dnf cpuset=/ mems_allowed=0
[  204.470849] CPU: 0 PID: 619 Comm: dnf Tainted: G      D W       4.9.13-201.fc25.x86_64+debug #1
[  204.472509] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-1.fc25589] Mem-Info:
[  312.006153] active_anon:102189 inactive_anon:19 isolated_anon:0
[  312.006153]  active_file:35 inactive_file:37 isolated_file:0
[  312.006153]  unevictable:0 dirty:0 writeback:0 unstable:0
[  312.006153]  slab_reclaimable:5224 slab_unreclaimable:5739
[  312.006153]  mapped:48 shmem:29 pagetables:459 bounce:0
[  312.006153]  free:1117 free_pcp:174 free_cma:0
[  312.010844] Node 0 active_anon:408756kB inactive_anon:76kB active_file:140kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:192kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 116kB writeback_tmp:0kB unstable:0kB pages_scanned:34 all_unreclaimable? no
[  312.014950] Node 0 DMA free:1844kB min:92kB low:112kB high:132kB active_anon:13736kB inactive_anon:0kB active_file:12kB inactive_file:4kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB slab_reclaimable:48kB slab_unreclaimable:24kB kernel_stack:16kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[  312.019434] lowmem_reserve[]: 0 438 438 438 438
[  312.020250] Node 0 DMA32 free:2624kB min:2628kB low:3284kB high:3940kB active_anon:395020kB inactive_anon:76kB active_file:152kB inactive_file:120kB unevictable:0kB writepending:0kB present:495608kB managed:454312kB mlocked:0kB slab_reclaimable:20848kB slab_unreclaimable:22932kB kernel_stack:1136kB pagetables:1836kB bounce:0kB free_pcp:696kB local_pcp:696kB free_cma:0kB
[  312.024943] lowmem_reserve[]: 0 0 0 0 0
[  312.025628] Node 0 DMA: 3*4kB (U) 5*8kB (UM) 4*16kB (UM) 4*32kB (U) 1*64kB (U) 0*128kB 0*256kB 1*512kB (M) 1*1024kB (U) 0*2048kB 0*4096kB = 1844kB
[  312.027975] Node 0 DMA32: 84*4kB (UMEH) 21*8kB (UMEH) 6*16kB (UMEH) 2*32kB (M) 1*64kB (H) 1*128kB (M) 1*256kB (M) 1*512kB (H) 1*1024kB (H) 0*2048kB 0*4096kB = 2648kB
[  312.030588] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[  312.031834] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[  312.033067] 96 total pagecache pages
[  312.033603] 0 pages in swap cache
[  312.034102] Swap cache stats: add 0, delete 0, find 0/0
[  312.034868] Free swap  = 0kB
[  312.035296] Total swap = 0kB
[  312.035711] 127900 pages RAM
[  312.036151] 0 pages HighMem/MovableOnly
[  312.036716] 10345 pages reserved
[  312.037191] 0 pages cma reserved
[  312.037667] 0 pages hwpoisoned
[  312.038126] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
[  312.039361] [  400]     0   400    11585      188      24       3        0         -1000 systemd-udevd
[  312.040734] [  479]     0   479    22299     3077      42       3        0             0 dhclient
[  312.042018] [  497]     0   497    54191    21799      80       4        0             0 guestfsd
[  312.043315] [  500]     0   500    31598       86      27       3        0             0 lvmetad
[  312.044585] [  619]     0   619   189475    76957     274       3        0             0 dnf
[  312.045829] Out of memory: Kill process 619 (dnf) score 637 or sacrifice child
[  312.046867] Killed process 619 (dnf) total-vm:757900kB, anon-rss:307828kB, file-rss:0kB, shmem-rss:0kB
commandrvf: stdout=n stderr=n flags=0x0
commandrvf: umount /sysroot/sys
[  312.089002] oom_reaper: reaped process 619 (dnf), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

and:

Running transaction
  Installing  : gdk-pixbuf2-2.36.5-1.fc26.x86_64                          1/157 
error: Couldn't fork %post(gdk-pixbuf2-2.36.5-1.fc26.x86_64): Cannot allocate memory
  Installing  : libjpeg-turbo-1.5.1-0.fc26.x86_64                         2/157 
error: Couldn't fork %post(libjpeg-turbo-1.5.1-0.fc26.x86_64): Cannot allocate memory
  Installing  : libwayland-client-1.13.0-1.fc26.x86_64                    3/157 
error: Couldn't fork %post(libwayland-client-1.13.0-1.fc26.x86_64): Cannot allocate memory
  Installing  : atk-2.23.4-1.fc26.x86_64                                  4/157 
error: Couldn't fork %post(atk-2.23.4-1.fc26.x86_64): Cannot allocate memory
  Installing  : libwayland-server-1.13.0-1.fc26.x86_64                    5/157 

Version-Release number of selected component (if applicable):

Current Rawhide version of dnf.

How reproducible:

100%

Steps to Reproduce:
1. Run dnf in a machine with 512MB of RAM and no swap.
Comment 1 Richard W.M. Jones 2017-03-14 19:57:48 UTC 
FWIW increasing the RAM to 1 GB (from 512 MB) works.
Comment 2 Honza Silhan 2017-03-22 12:44:31 UTC 
We should try to reproduce it inside isolated cgroup with limited RAM.
Comment 3 Jan Kurik 2017-08-15 06:57:03 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.
Comment 4 Jaroslav Mracek 2017-11-07 10:34:26 UTC 
I have fixed several memory-leaks in dnf, libdnf. Please can you confirm that the issue is still reproducible with "dnf-0:2.7.5-1.fc26.noarch" and "libdnf-0.11.1"?
Comment 5 Richard W.M. Jones 2017-11-07 11:06:36 UTC 
A (long) one-line reproducer is:

$ virt-builder \
    --install fedora-repos-rawhide \
    --edit '/etc/yum.repos.d/fedora-rawhide.repo: s/^enabled=0/enabled=1/' \
    --edit '/etc/yum.repos.d/fedora-rawhide.repo: s/^gpgcheck=1/gpgcheck=0/' \
    --run-command 'dnf -y update dnf dnf-plugins-core libdnf' \
    --install bash,emacs,mlocate,net-tools,nfs-utils,ntp,sudo,telnet \
    fedora-26

To see what versions of dnf, libdnf were installed you have to grep
the /tmp/builder.log file inside the final image:

$ virt-cat fedora-26.img /tmp/builder.log | less

With Rawhide today I happened to get:

dnf 2.7.5-2.fc28
libdnf 0.11.1-2.fc28

These versions do appear to work in the restricted 500 MB environment
of the appliance, and so I am closing this bug as fixed in rawhide.
Note You need to log in before you can comment on or make changes to this bug.