Note: This is a public test instance of Red Hat Bugzilla. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback at bugzilla.redhat.com.

Bug 560642

Summary: abrtd dies with "/usr/lib/abrt/lib.so missing" msg
Product: [Fedora] Fedora Reporter: Ankur Sinha (FranciscoD) <sanjay.ankur>
Component: abrtAssignee: Jiri Moskovcak <jmoskovc>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 12CC: anton, beland, dfediuck, didierg-divers, dvlasenk, iprikryl, jeremy.butler36, jlaska, jmoskovc, kklic, mads, mnowak, npajkovs
Target Milestone: ---Keywords: CommonBugs
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: https://fedoraproject.org/wiki/Common_F12_bugs#abrt-alert-icon
Fixed In Version: abrt-1.0.7-1.fc12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-02-23 05:39:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 538274    
Attachments:
Description Flags
/etc/abrt/abrt.conf none

Description Ankur Sinha (FranciscoD) 2010-02-01 13:39:09 UTC
Description of problem:
failure to start abrt

[not a duplicate of 5547424]

Version-Release number of selected component (if applicable):
[Ankur1@localhost ~]$ rpm -qa  '*abrt*'
abrt-libs-1.0.4-1.fc12.x86_64
abrt-plugin-runapp-1.0.4-1.fc12.x86_64
abrt-1.0.4-1.fc12.x86_64
abrt-desktop-1.0.4-1.fc12.x86_64
abrt-addon-ccpp-1.0.4-1.fc12.x86_64
abrt-plugin-logger-1.0.4-1.fc12.x86_64
abrt-addon-python-1.0.4-1.fc12.x86_64
abrt-plugin-bugzilla-1.0.4-1.fc12.x86_64
abrt-gui-1.0.4-1.fc12.x86_64
abrt-addon-kerneloops-1.0.4-1.fc12.x86_64

[Ankur1@localhost ~]$ rpm -qaV  '*abrt*'
..?......  c /etc/abrt/plugins/Bugzilla.conf



[root@localhost abrt]# service abrtd restart
Stopping abrt daemon:                                      [FAILED]
Starting abrt daemon:                                      [  OK  ]
[root@localhost abrt]# service abrtd status
abrt dead but pid file exists


removing the /var/cache/abrt looks like it solved the problem.

I got this in my /var/log/messages | grep abrtd

Jan 31 20:45:48 localhost abrtd: Registered Analyzer plugin 'Python'
Jan 31 20:45:48 localhost abrtd: Registered Action plugin 'KerneloopsScanner'
Jan 31 20:45:48 localhost abrtd: Registered Reporter plugin 'Logger'
Jan 31 20:45:48 localhost abrtd: Registered Analyzer plugin 'Kerneloops'
Jan 31 20:45:48 localhost abrtd: Registered Reporter plugin
'KerneloopsReporter'
Jan 31 20:45:48 localhost abrtd: warning: /proc/sys/kernel/core_pattern already
contains |/usr/libexec/abrt-hook-ccpp /var/cache/abrt %p %s %u %c, did abrt
daemon crash recently?
Jan 31 20:45:48 localhost abrtd: Registered Analyzer plugin 'CCpp'
Jan 31 20:45:48 localhost abrtd: Registered Reporter plugin 'Bugzilla'
Jan 31 20:45:48 localhost abrtd: Registered Database plugin 'SQLite3'
Jan 31 20:45:49 localhost abrtd: can't load '/usr/lib64/abrt/lib.so':
/usr/lib64/abrt/lib.so: cannot open shared object file: No such file or
directory


There is no lib.so there.. i checked. 

regards,
Ankur

Comment 1 Ankur Sinha (FranciscoD) 2010-02-01 13:45:01 UTC
rpm -qaV '*abrt*' as root returned me nothing

Comment 2 Denys Vlasenko 2010-02-01 15:05:56 UTC
The line

Jan 31 20:45:49 localhost abrtd: can't load '/usr/lib64/abrt/lib.so':
/usr/lib64/abrt/lib.so: cannot open shared object file: No such file or
directory

suggests that here we are trying to load a plugin with pName == "":

    string libPath = ssprintf(PLUGINS_LIB_DIR"/"PLUGINS_LIB_PREFIX"%s."PLUGINS_LIB_EXTENSION, pName);
    CLoadedModule* module = new CLoadedModule(libPath.c_str());

Please show your /etc/abrt/abrt.conf.

Also please run

ls -l /usr/lib*/abrt

What this command prints?

Comment 3 Denys Vlasenko 2010-02-01 15:28:08 UTC
*** Bug 558767 has been marked as a duplicate of this bug. ***

Comment 4 Ankur Sinha (FranciscoD) 2010-02-01 17:36:45 UTC
Created attachment 388094 [details]
/etc/abrt/abrt.conf

Comment 5 Ankur Sinha (FranciscoD) 2010-02-01 17:39:46 UTC
[Ankur1@localhost resume]$ su -c 'ls -l /usr/lib*/abrt'
Password: 
total 240
-rw-r--r--. 1 root root 11682 2010-01-22 20:40 Bugzilla.GTKBuilder
-rw-r--r--. 1 root root  5215 2010-01-22 20:40 KerneloopsReporter.GTKBuilder
-rwxr-xr-x. 1 root root 40960 2010-01-22 20:40 libBugzilla.so
-rwxr-xr-x. 1 root root 45096 2010-01-22 20:40 libCCpp.so
-rwxr-xr-x. 1 root root 15640 2010-01-22 20:40 libKerneloopsReporter.so
-rwxr-xr-x. 1 root root 24328 2010-01-22 20:40 libKerneloopsScanner.so
-rwxr-xr-x. 1 root root 13400 2010-01-22 20:40 libKerneloops.so
-rwxr-xr-x. 1 root root 15624 2010-01-22 20:40 libLogger.so
-rwxr-xr-x. 1 root root  9464 2010-01-22 20:40 libPython.so
-rwxr-xr-x. 1 root root  9872 2010-01-22 20:40 libRunApp.so
-rwxr-xr-x. 1 root root 28192 2010-01-22 20:40 libSQLite3.so
-rw-r--r--. 1 root root  6183 2010-01-22 20:40 Logger.GTKBuilder

Comment 6 Denys Vlasenko 2010-02-03 12:29:11 UTC
Unfortunately, I don't see anything which hits me what the problem is.

Please do as root:

killall abrtd
abrtd -dvvv


wait a few secs, then ^C running abrtd and cut-n-paste abrtd's on-screen log
here.

Comment 7 Ankur Sinha (FranciscoD) 2010-02-03 13:14:24 UTC
(In reply to comment #6)
> Unfortunately, I don't see anything which hits me what the problem is.
> 
> Please do as root:
> 
> killall abrtd
> abrtd -dvvv
> 
> 
> wait a few secs, then ^C running abrtd and cut-n-paste abrtd's on-screen log
> here.    

hi,

[root@localhost ~]# killall abrtd
[root@localhost ~]# abrtd -dvvv
abrtd: Initializing XML-RPC library
abrtd: Creating glib main loop
abrtd: Initializing inotify
abrtd: Loading plugins from /usr/lib64/abrt
abrtd: Loaded RunApp.conf
abrtd: Plugin RunApp: 'Enabled' is not set, not loading it (yet)
abrtd: Loaded SQLite3.conf
abrtd: Plugin SQLite3: 'Enabled' is not set, not loading it (yet)
abrtd: Loaded Python.conf
abrtd: Loaded plugin Python v.0.0.1
abrtd: Registered Analyzer plugin 'Python'
abrtd: Loaded Kerneloops.conf
abrtd: Loaded plugin KerneloopsScanner v.0.0.1
abrtd: Scanning dmesg...
abrtd: Registered Action plugin 'KerneloopsScanner'
abrtd: Loaded Logger.conf
abrtd: Loaded plugin Logger v.0.0.1
abrtd: Registered Reporter plugin 'Logger'
abrtd: Loaded Kerneloops.conf
abrtd: Loaded plugin Kerneloops v.0.0.2
abrtd: Registered Analyzer plugin 'Kerneloops'
abrtd: Loaded Kerneloops.conf
abrtd: Loaded plugin KerneloopsReporter v.0.0.1
abrtd: Registered Reporter plugin 'KerneloopsReporter'
abrtd: Loaded CCpp.conf
abrtd: Loaded plugin CCpp v.0.0.1
abrtd: Registered Analyzer plugin 'CCpp'
abrtd: Loaded Bugzilla.conf
abrtd: Loaded plugin Bugzilla v.0.0.4
abrtd: Registered Reporter plugin 'Bugzilla'
abrtd: Loading settings
abrtd:  ParseListWithArgs(RunApp("test x\"`cat component`\" = x\"xorg-x11-server-Xorg\" && cp /var/log/Xorg.0.log ."))
abrtd:  adding (RunApp,test x"`cat component`" = x"xorg-x11-server-Xorg" && cp /var/log/Xorg.0.log .)
abrtd:  ParseListWithArgs(Bugzilla,Logger)
abrtd:  adding (Bugzilla,)
abrtd:  adding (Logger,)
abrtd: AnalyzerActionsAndReporters['CCpp']=...
abrtd:  ParseListWithArgs(KerneloopsReporter)
abrtd:  adding (KerneloopsReporter,)
abrtd: AnalyzerActionsAndReporters['Kerneloops']=...
abrtd:  ParseListWithArgs(Bugzilla,Logger)
abrtd:  adding (Bugzilla,)
abrtd:  adding (Logger,)
abrtd: AnalyzerActionsAndReporters['Python']=...
abrtd:  ParseListWithArgs(KerneloopsScanner)
abrtd:  adding (KerneloopsScanner,)
abrtd: Loading GPG key '/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora'
abrtd: Adding actions or reporters
abrtd: AddActionOrReporter('RunApp','test x"`cat component`" = x"xorg-x11-server-Xorg" && cp /var/log/Xorg.0.log .')
abrtd: Adding analyzers, actions or reporters
abrtd: Initializing dbus
abrtd: dbus_bus_get
abrtd: dbus_connection_set_watch_functions
abrtd: add_watch(watch:0xc95730, data)
abrtd:  dbus_watch_get_unix_fd():6
abrtd: toggled_watch(watch:0xc95730, data)
abrtd: add_watch(watch:0xc95780, data)
abrtd:  dbus_watch_get_unix_fd():6
abrtd: toggled_watch(watch:0xc95780, data)
abrtd:  adding watch to glib main loop. dbus_flags:1 glib_flags:1
abrtd: dbus_connection_set_timeout_functions
abrtd: dbus_connection_register_object_path
abrtd: dbus_bus_request_name
abrtd: add_timeout()
abrtd: remove_timeout()
abrtd: dbus init done
abrtd: Adding inotify watch to glib main loop
abrtd: Adding signal pipe watch to glib main loop
abrtd: Creating lock file
abrtd: Creating pid file
abrtd: Scanning for unsaved entries
abrtd: Locked '/var/cache/abrt/pyhook-1264961628-2604.lock'
abrtd: UnLocked '/var/cache/abrt/pyhook-1264961628-2604.lock'
abrtd: Loaded SQLite3.conf
abrtd: Loaded plugin SQLite3 v.0.0.2
abrtd: Registered Database plugin 'SQLite3'
abrtd: 4 rows returned by SQL:SELECT * FROM abrt_v2 WHERE UID = '500' OR UID = '-1';
abrtd: /var/cache/abrt/pyhook-1264961628-2604 is already saved in database
abrtd: Locked '/var/cache/abrt/ccpp-1265129386-7975.lock'
abrtd: UnLocked '/var/cache/abrt/ccpp-1265129386-7975.lock'
abrtd: 4 rows returned by SQL:SELECT * FROM abrt_v2 WHERE UID = '500' OR UID = '-1';
abrtd: /var/cache/abrt/ccpp-1265129386-7975 is already saved in database
abrtd: Locked '/var/cache/abrt/ccpp-1264963243-2788.lock'
abrtd: UnLocked '/var/cache/abrt/ccpp-1264963243-2788.lock'
abrtd: 1 rows returned by SQL:SELECT * FROM abrt_v2 WHERE UID = '501' OR UID = '-1';
abrtd: /var/cache/abrt/ccpp-1264963243-2788 is already saved in database
abrtd: Locked '/var/cache/abrt/ccpp-1265093673-2612.lock'
abrtd: UnLocked '/var/cache/abrt/ccpp-1265093673-2612.lock'
abrtd: 4 rows returned by SQL:SELECT * FROM abrt_v2 WHERE UID = '500' OR UID = '-1';
abrtd: /var/cache/abrt/ccpp-1265093673-2612 is already saved in database
abrtd: Locked '/var/cache/abrt/ccpp-1265050007-1902.lock'
abrtd: UnLocked '/var/cache/abrt/ccpp-1265050007-1902.lock'
abrtd: 4 rows returned by SQL:SELECT * FROM abrt_v2 WHERE UID = '500' OR UID = '-1';
abrtd: /var/cache/abrt/ccpp-1265050007-1902 is already saved in database
abrtd: Running...

^Cabrtd: Got signal 2 through signal pipe
abrtd: UnRegistered Reporter plugin Bugzilla
abrtd: UnRegistered Analyzer plugin CCpp
abrtd: UnRegistered Analyzer plugin Kerneloops
abrtd: UnRegistered Reporter plugin KerneloopsReporter
abrtd: UnRegistered Action plugin KerneloopsScanner
abrtd: UnRegistered Reporter plugin Logger
abrtd: UnRegistered Analyzer plugin Python
abrtd: UnRegistered Database plugin SQLite3
abrtd: Got signal 2, exiting


Ankur,

I checked the messages file and it hasn't given the lib.so error again yet.

Comment 8 Denys Vlasenko 2010-02-03 15:37:16 UTC
What I am after is a line like "abrtd: Loaded .conf" - that is, with empty plugin name! By looking at the log around it, I'd figure out how it happens. When you'll get this lib.so error again, collect abrtd log again and post it here please.

Comment 9 Karel Klíč 2010-02-03 17:26:27 UTC
>/usr/lib64/abrt/lib.so: cannot open shared object file: No such file or
directory

It can be reproduced by creating a new dump directory with empty "analyzer" file.

Comment 10 Denys Vlasenko 2010-02-04 15:29:47 UTC
Yes Karel, thanks!

I made a patch which prevents abrtd from exiting in this case. Then started abrtd, crashed /bin/sleep, stopped abrtd, changed analyzer to "", restrated abrtd, ran abrt-gui and it now presents me with mostly empty report dialog.
abrtd log:

Feb  4 16:25:01 abrtd: Locked '/var/cache/abrt/ccpp-1265297033-18308.lock'
Feb  4 16:25:01 abrtd: UnLocked '/var/cache/abrt/ccpp-1265297033-18308.lock'
Feb  4 16:25:01 abrtd:  run_analyser_CreateReport('')
Feb  4 16:25:01 abrtd: Loaded .conf
Feb  4 16:25:01 abrtd: Can't load '/usr/lib64/abrt/lib.so': /usr/lib64/abrt/lib.so: cannot open shared object file: No such file or directory
Feb  4 16:25:01 abrtd: Warning(':1.80'): Can't load '/usr/lib64/abrt/lib.so': /usr/lib64/abrt/lib.so: cannot open shared object file: No such file or directory
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: Plugin '' is not registered
Feb  4 16:25:01 abrtd: Warning(':1.80'): Plugin '' is not registered
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: Loaded .conf
Feb  4 16:25:01 abrtd: Can't load '/usr/lib64/abrt/lib.so': /usr/lib64/abrt/lib.so: cannot open shared object file: No such file or directory
Feb  4 16:25:01 abrtd: Warning(':1.80'): Can't load '/usr/lib64/abrt/lib.so': /usr/lib64/abrt/lib.so: cannot open shared object file: No such file or directory
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: Plugin '' is not registered
Feb  4 16:25:01 abrtd: Warning(':1.80'): Plugin '' is not registered
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: Error running ''
Feb  4 16:25:01 abrtd: Warning(':1.80'): Error running ''
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: CreateCrashReport() returns 10
Feb  4 16:25:01 abrtd: Particular analyzer plugin isn't loaded or there is an error within plugin(s)
Feb  4 16:25:01 abrtd: Warning(':1.80'): Particular analyzer plugin isn't loaded or there is an error within plugin(s)
Feb  4 16:25:01 abrtd: DBus message sent
Feb  4 16:25:01 abrtd: DBus message sent

IOW: it's better now, but GUI fails to properly react to analyzer fatal errors (in this case, bad analyzer name!). [Will apply the patch to git in a few mins...]

Comment 11 Denys Vlasenko 2010-02-04 15:40:48 UTC
Applied patch to git. Want to also fix "GUI doesn't react to total analyzer failure" problem, thus not closing this bug.

Comment 12 Denys Vlasenko 2010-02-04 16:15:51 UTC
*** Bug 552869 has been marked as a duplicate of this bug. ***

Comment 13 Denys Vlasenko 2010-02-04 19:21:22 UTC
*** Bug 554724 has been marked as a duplicate of this bug. ***

Comment 14 Denys Vlasenko 2010-02-09 03:37:10 UTC
Fixed in git: in cases like this, now abrtd will not die, but will complain in syslog

Comment 15 Christopher Beland 2010-02-12 22:24:14 UTC
Nominated for F13Beta, since this is listed on F12 Common Bugs, a fix is available, and ABRT failures can impact beta testing of other components.

Comment 16 Fedora Update System 2010-02-15 14:07:59 UTC
abrt-1.0.7-1.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/abrt-1.0.7-1.fc12

Comment 17 Fedora Update System 2010-02-18 22:31:47 UTC
abrt-1.0.7-1.fc12 has been pushed to the Fedora 12 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update abrt'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F12/FEDORA-2010-1598

Comment 18 Fedora Update System 2010-02-23 05:37:48 UTC
abrt-1.0.7-1.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 James Laska 2010-03-04 23:25:33 UTC
Removing CommonBugs keyword.  I'm not able to determine which existing entry matches the description of this bug (https://fedoraproject.org/wiki/Common_F12_bugs)

Comment 20 James Laska 2010-03-04 23:29:08 UTC
(In reply to comment #19)
> Removing CommonBugs keyword.  I'm not able to determine which existing entry
> matches the description of this bug
> (https://fedoraproject.org/wiki/Common_F12_bugs)    

Correction, I've linked to the existing ABRT entry on the Common_F12_Bugs page.  However, the description doesn't seem to match my understanding of this bug.