本文是笔者最近遇到的一个故障的处理过程，解决方案是Rebuild RPM 的DB，后面内容其实是REDHAT官方的solutions,不过我遇到的现象和解决方案都与官方有点出入，故一直帖出来：
[root@migrationLin6 ~]# rpm -qa error: rpmdbNextIterator: skipping h# 514 Header V3 RSA/SHA256 Signature, key ID fd431d51: BAD error: rpmdbNextIterator: skipping h# 259 Header V3 RSA/SHA256 Signature, key ID fd431d51: BAD error: rpmdbNextIterator: skipping h# 4 Header V3 RSA/SHA256 Signature, key ID fd431d51: BAD
没想到，按照官方步骤做了 rpm --rebuilddb以后，YUM又现现如下错误
[root@migrationLin6 rpm]# yum clean all Traceback (most recent call last): File "/usr/bin/yum", line 29, in <module> yummain.user_main(sys.argv[1:], exit_code=True) File "/usr/share/yum-cli/yummain.py", line 294, in user_main errcode = main(args) File "/usr/share/yum-cli/yummain.py", line 115, in main base.getOptionsConfig(args) File "/usr/share/yum-cli/cli.py", line 228, in getOptionsConfig self.conf File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 910, in <lambda> conf = property(fget=lambda self: self._getConfig(), File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 299, in _getConf ig startupconf = config.readStartupConfig(fn, root) File "/usr/lib/python2.6/site-packages/yum/config.py", line 892, in readStartu pConfig startupconf.releasever = _getsysver(startupconf.installroot, startupconf.dis troverpkg) File "/usr/lib/python2.6/site-packages/yum/config.py", line 1043, in _getsysve r hdr = idx.next() StopIteration
rm -f /var/lib/rpm/* rpm -vv -rebuilddb yum update -y
How to rebuild RPM database on a Red Hat Enterprise Linux system?
- Red Hat Enterprise Linux (RHEL) - All Versions - All architectures
- RPM is not working correctly, updates cannot be applied.
- How to rebuild the RPM database?
- Why does rpm command segfault when executed?
yum updatefails with rpmdb errors.
- Damaged header instance error
rpmcommand terminates with "cannot open Packages database in /var/lib/rpm"
- Why does rpm or yum terminate with
rpmdb: Lock table is out of available locker entries?
- Attempted to do an
rpm -qon a couple of packages and got errors like this:
rpmdb: /var/lib/rpm/Packages: unexpected file type or format error: cannot open Packages index using db3 - Invalid argument (22)
- On doing rpm query or install, it is failing with following error:-
error: rpmdbNextIterator: skipping h# 1601 Header V4 RSA/SHA1 signature: BAD, key ID 2874ffbe
- Why "rpm -qa" command is not showing installed packages?
rpmdatabase got corrupted
- Depending on the particular scenario, there are several different paths that correct this situation
Important: Before attempting to rebuild the rpm database, first refer to the following known issue for Red Hat Enterprise Linux 6 systems:
1) Stale lock file cleanup - files needed for updating are being held open.
Note: There are two methods (1A & 1B) to correct this:
1A) Reboot system (preferred)
The best course of action is to simply reboot the machine, as the system cleans up stale locks during the sysinit portion of boot. The cleanup occurs at this stage as the system is assured there is no process that would have a lock on the RPM database.
# grep rpm /etc/rc.d/rc.sysinit rm -f /var/lib/rpm/__db* &> /dev/null
1B) Reboot not feasible (more risky than reboot)
Delete all files in /var/spool/up2date
# cd /var/spool/up2date # rm * # rm .*
Ensure there are no processes with the RPM database files open. One way to do this is using
# ps -aux | grep -e rpm -e yum -e up2date # lsof | grep /var/lib/rpm
If, and only if, there is no RPM database activity going on it is safe to delete the lock files.
WARNING: If the system is in multi-user mode it is possible that a process may open the RPM database at any time. Therefore, the recommended method for clearing stale locks is the reboot which cleans up the locks prior to entering multi-user mode.
# rm -f /var/lib/rpm/__db*
2) DB corruption recovery process - The RPM database has become corrupt and recovery is desired.
As always it best to start by taking a backup before attempting to repair.
# cd /var/lib # tar -zcvf /var/preserve/rpmdb-$(date +%Y-%m-%d_%H-%M-%S).tar.gz rpm
tarbackup can be used if the attempt to recover the RPM database has issues.
Verify integrity of the
# cd /var/lib/rpm # rm -f __db* # to avoid stale locks # /usr/lib/rpm/rpmdb_verify Packages
If the last command shown in method 2 returns without any error, then we can be sure about the integrity of the rpm database. You can also issue the following command to be sure of the success of the rpmdb_verify command.
$ /usr/lib/rpm/rpmdb_verify Packages $ echo $? 0
A return value of 0 from
echo $?means that the last command has been executed successfully.
If the above verify reports success, go to step 6.
If the above verify reports any errors, a dump and load of the database is required.
# mv Packages Packages.orig # /usr/lib/rpm/rpmdb_dump Packages.orig | /usr/lib/rpm/rpmdb_load Packages
Verify the newly loaded Packages file, as well as
# /usr/lib/rpm/rpmdb_verify Packages # rpm -qa >/dev/null # Validating there are no errors # rpm -qa # Validating list of installed RPMs
If no errors and everything looks good, rebuild the RPM indexes
# rpm -vv --rebuilddb
Once again verify RPM database:
# cd /var/lib/rpm # /usr/lib/rpm/rpmdb_verify Packages
Note: If the above recovery steps fail or throw errors, restoring
/var/lib/rpm directory from a recent system backup may be your best option. Remember to not overwrite the
tar backup as it may still be valuable.
Note: If you have another machine having the exactly same installed package set, you can rebuild the rpm db with Packages file on the machine.
Note: If you have a rpm database in other directory except for
/var/lib/rpm and it's corrupted, you will need to do the same things to the rpm database, not one in
/var/lib/rpm. The rpm command has
--root option to specify the root directory for the rpm database. Especially, please take care of the
rpm -vv --rebuilddb command to rebuild the rpm database. For example, if you are using
/opt/var/lib/rpm for a directory including rpm database, run this command
rpm -vv --rebuilddb --root /opt.
3.) Other resources:
- To restore rpm database file from
/var/log/rpmpkgs, refer to: How to recover rpm database file /var/lib/rpm/Packages using /var/log/rpmpkgs ?
- Comprehensive notes on RPM database recovery can be found at: http://people.redhat.com/berrange/notes/rpmrecovery.html.
- rpm database corruption
yum updatefails with the following:
[root@localhost ~]# yum update rpmdb: PANIC: fatal region error detected; run recovery error: db3 error(-30974) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db3 - (-30974) error: cannot open Packages database in /var/lib/rpm CRITICAL:yum.main: Error: rpmdb open failed [root@localhost ~]# yum clean all rpmdb: PANIC: fatal region error detected; run recovery error: db3 error(-30974) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db3 - (-30974) error: cannot open Packages database in /var/lib/rpm CRITICAL:yum.main: Error: rpmdb open failed
rpmshows damaged header instances:
$ rpm -Va error: rpmdb: damaged header instance #<header number> retrived, skipping error: rpmdb: damaged header instance #<header number> retrived, skipping (above message repeated ad infinitum)
rpmcommands terminate with the error below:
rpmdb: unable to join the environment error: db3 error(11) from dbenv->open: Resource temporarily unavailble error: cannot open Packages index using db3 - Resource temporarily unavailable (11) error: cannot open Packages database in /var/lib/rpm
yumcommands return the following:
$ rpm -qa ... rpmdb: Lock table is out of available locker entries error: db4 error(22) from db->close: Invalid argument