Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 1304850

Summary: Beagle Bone Black: mmcqd/0: page allocation failure on dnf update
Product: [Fedora] Fedora Reporter: Roman Yepishev <rye>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 23CC: gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, mchehab, pbrobinson, pwhalen, rye
Target Milestone: ---Flags: rye: needinfo-
Target Release: ---   
Hardware: armv7hl   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-20 13:45:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 245418    

Description Roman Yepishev 2016-02-04 19:05:15 UTC
Description of problem:

After installation of Fedora-Server-armhfp-23-10-sda.raw.xz to the external MMC and boot from uboot provided by the image, running dnf update breaks the filesystem.


Version-Release number of selected component (if applicable):

Kernel 4.2.3-300.fc23.armv7hl (default in the image)


How reproducible:

Always


Steps to Reproduce:

1. Write Fedora-Server-armhfp-23-10-sda.raw.xz to an SD card
2. Boot BeagleBoneBlack while pressing S2 switch (to use uboot from SD card)
3. Perform initial configuration with display attached - languate, root password, etc.
4. Run dnf update


Actual results:

[ 1977.215963] mmcqd/0: page allocation failure: order:4, mode:0xc020
[ 1977.216205] CPU: 0 PID: 552 Comm: mmcqd/0 Not tainted 4.2.3-300.fc23.armv7hl #1
[ 1977.216397] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 1977.216620] [<c02184b0>] (unwind_backtrace) from [<c0213584>] (show_stack+0x18/0x1c)
[ 1977.216848] [<c0213584>] (show_stack) from [<c08d71d0>] (dump_stack+0x74/0x90)
[ 1977.217059] [<c08d71d0>] (dump_stack) from [<c03307a4>] (warn_alloc_failed+0xf0/0x11c)
[ 1977.217293] [<c03307a4>] (warn_alloc_failed) from [<c0333bb4>] (__alloc_pages_nodemask+0x7a8/0x850)
[ 1977.217533] [<c0333bb4>] (__alloc_pages_nodemask) from [<c0333ea0>] (alloc_kmem_pages+0x68/0xfc)
[ 1977.217777] [<c0333ea0>] (alloc_kmem_pages) from [<c034c02c>] (kmalloc_order+0x18/0x28)
[ 1977.217999] [<c034c02c>] (kmalloc_order) from [<c034c05c>] (kmalloc_order_trace+0x20/0x80)
[ 1977.218233] [<c034c05c>] (kmalloc_order_trace) from [<c05b6c30>] (edma_prep_slave_sg+0xc8/0x284)
[ 1977.218698] [<c05b6c30>] (edma_prep_slave_sg) from [<bf0da948>] (omap_hsmmc_request+0x368/0x470 [omap_hsmmc])
[ 1977.219158] [<bf0da948>] (omap_hsmmc_request [omap_hsmmc]) from [<bf074fe0>] (mmc_start_request+0x234/0x274 [mmc_core])
[ 1977.219591] [<bf074fe0>] (mmc_start_request [mmc_core]) from [<bf07609c>] (mmc_start_req+0x2a8/0x32c [mmc_core])
[ 1977.219961] [<bf07609c>] (mmc_start_req [mmc_core]) from [<bf13e2e0>] (mmc_blk_issue_rw_rq+0x414/0xa04 [mmc_block])
[ 1977.220271] [<bf13e2e0>] (mmc_blk_issue_rw_rq [mmc_block]) from [<bf13ecb8>] (mmc_blk_issue_rq+0x3e8/0x454 [mmc_block])
[ 1977.220572] [<bf13ecb8>] (mmc_blk_issue_rq [mmc_block]) from [<bf13f460>] (mmc_queue_thread+0xd4/0x168 [mmc_block])
[ 1977.220857] [<bf13f460>] (mmc_queue_thread [mmc_block]) from [<c02669c4>] (kthread+0xe0/0xf4)
[ 1977.221106] [<c02669c4>] (kthread) from [<c020fa08>] (ret_from_fork+0x14/0x2c)
[ 1977.221303] Mem-Info:
[ 1977.221400] active_anon:11857 inactive_anon:20158 isolated_anon:0
                active_file:33763 inactive_file:34101 isolated_file:0
                unevictable:0 dirty:2050 writeback:6891 unstable:0
                slab_reclaimable:7612 slab_unreclaimable:8511
                mapped:5559 shmem:40 pagetables:426 bounce:0
                free:2845 free_pcp:59 free_cma:1714
[ 1977.222246] DMA free:11380kB min:2840kB low:3548kB high:4260kB active_anon:47428kB inactive_anon:80632kB active_file:135052kB inactive_file:136404kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:523264kB managed:505504kB mlocked:0kB dirty:8200kB writeback:27564kB mapped:22236kB shmem:160kB slab_reclaimable:30448kB slab_unreclaimable:34044kB kernel_stack:872kB pagetables:1704kB unstable:0kB bounce:0kB free_pcp:236kB local_pcp:236kB free_cma:6856kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 1977.223302] lowmem_reserve[]: 0 0 0 0
[ 1977.223563] DMA: 589*4kB (UMC) 470*8kB (UEMC) 185*16kB (UEMC) 60*32kB (UMC) 4*64kB (C) 1*128kB (C) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 11380kB
[ 1977.224197] 68043 total pagecache pages
[ 1977.224311] 139 pages in swap cache
[ 1977.224413] Swap cache stats: add 1364, delete 1225, find 105/161
[ 1977.224568] Free swap  = 495356kB
[ 1977.224660] Total swap = 500236kB
[ 1977.224753] 130816 pages RAM
[ 1977.224838] 0 pages HighMem/MovableOnly
[ 1977.224940] 344 pages reserved
[ 1977.225026] 4096 pages cma reserved
[ 1977.225143] edma-dma-engine edma-dma-engine.0: edma_prep_slave_sg: Failed to allocate a descriptor
[ 1977.225377] omap_hsmmc 48060000.mmc: prep_slave_sg() failed
[ 1977.225528] omap_hsmmc 48060000.mmc: MMC start dma failure
[ 1977.227260] mmcblk0: unknown error -1 sending read/write command, card status 0x900
[ 1977.277269] mmc0: tried to reset card
[ 1977.284137] mmcqd/0: page allocation failure: order:4, mode:0xc020

Eventually the system becomes so unstable that even dmesg fails with Segmentation Fault.



Expected results:

Update completes successfully.



Additional info:

This seems to be related - https://bugs.launchpad.net/ubuntu/+source/linux-ti-omap4/+bug/746137

Setting vm.min_free_kbytes = 8192 via sysctl makes the error go away and update completes ~successfully (if kernel package is not excluded, then the system will not boot, but that seems to be a different issue I am still investigating).

Comment 1 Peter Robinson 2016-02-05 09:53:52 UTC
what sort of SD card do you have, we've found that some really don't cope well with the running of a standard filesystem/distro and I suspect it might well be a HW issue

Comment 2 Roman Yepishev 2016-02-05 14:55:11 UTC
Sure, I had the same idea. Originally failure occured on a brand new SD card (Samsung microSDHC UHS-I - class 10 card).

Then I reverted to the older Kingston 16GB SDC10 card (also class 10) - it went through a bunch of heavy I/O operations before with Ubuntu/Debian (updates, syncthing, dd write tests), but I've started getting page allocation failures once I let dnf do it's thing on Fedora 23. Both cards started to work fine once I adjusted vm.min_free_kbytes.

Comment 3 Josh Boyer 2016-05-27 13:33:20 UTC
Is this still an issue with 4.5.y?

Comment 4 Roman Yepishev 2018-02-19 17:53:36 UTC
Leaving this closed and clearning needinfo flag as I no longer have the device in question to check newer kernels.