Linking Fails With "ld: 0706-010 The binder was killed by a signal: Segmentation fault" On AIX 6.1 And AIX 7.1 [ID 1264074.1]

In this Document
Symptoms
Cause
Solution
References

---

This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.

Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.2.0.3 - Release: 10.2 to 11.2
Enterprise Manager Base Platform - Version: 10.2.0.2 to 10.2.0.5 [Release: 10.2 to 10.2]
IBM AIX on POWER Systems (64-bit)
.
The problem below appears to affect Oracle installations on AIX servers running:

AIX 6.1 TL2 SP10 (6100-02-10-1036) and higher
AIX 6.1 TL3 SP7 (6100-03-07-1036) and higher
AIX 6.1 TL4 SP7 (6100-04-07-1036) and higher
AIX 6.1 TL5 SP3 (6100-05-03-1036) and higher
AIX 6.1 TL6 (all SP levels)
AIX 6.1 TL7 (all SP levels)
AIX 7.1 (no TL installed)

The problem does not appear to affect:

AIX 6.1 TL2 SP9 or lower
AIX 6.1 TL3 SP6 or lower
AIX 6.1 TL4 SP6 or lower
AIX 6.1 TL5 SP2 or lower

However, a few customers have reported this issue on lower SP levels.

Symptoms

While installing Oracle10g Release 2 (10.2), Oracle11g Release 1 (11.1) or Oracle11g Release 2 (11.2) on an AIX server running AIX 6.1 or AIX 7.1, the following error may occur:

INFO:
ld: 0706-010 The binder was killed by a signal: Segmentation fault
Check for binder messages or use local problem reporting procedures.

INFO: make: 1254-004 The error code from the last command is 254.

Stop.

INFO: make: 1254-004 The error code from the last command is 2.

Stop.

INFO: End output from spawned process.
INFO: ----------------------------------
INFO: Exception thrown from action: make
Exception Name: MakefileException
Exception String: Error in invoking target 'agent nmb nmo nmhs' of makefile '/u01/app/oracle/product/11.2.0/sysman/lib/ins_emagent.mk'.

More specifically, the "ld: 0706-010" error may occur when running the following command:

% make -f $ORACLE_HOME/sysman/lib/ins_emagent.mk relink_sharedobj SHAREDOBJ=libnmemso

Using 'truss' to trace this command generates output similar to this:

:
15925416: 20512937: kopen("/usr/lib/glink64.o", O_RDONLY) = 31
15925416: 20512937: fstatx(31, 0x0FFFFFFFFFFFCBF0, 176, 0) = 0
15925416: 20512937: shmat(31, 0x0000000000000000, 6144) = 0x0700000300000000
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011161E140
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011162E160
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011163E180
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011164E1A0
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011165E1C0
15925416: 20512937: __libc_sbrk(0x0000000000010020) = 0x000000011166E1E0
15925416: 20512937: _getpid() = 15925416
15925416: 20512937: lseek(0x0000000000000003, 0x0000000000000000, 0x0000000000000001) = 0x0000000000002252
:
15925416: 20512937: kread(0x0000000000000003, 0x00000001102C14B8, 0x0000000000001000) = 0x0000000000001000
15925416: 0x00000000: "\n"
15925416: 20512937: shmat(4, 0x07F0000000000000, 2048) = 0x07F0000000000000
15925416: Received signal #11, SIGSEGV [default]
15925416: *** process killed ***
:

The following error may also occur:

ld: 0711-711 ERROR: Input file $ORACLE_HOME/lib//libnmemso.so is empty. The file is being ignored.

NOTE

To verify which AIX version is running, use the command:

% oslevel -s

Cause

Oracle Support opened Bug:10259393 for this issue but it appears to be a known issue on the following AIX versions:

AIX 6.1 TL2 SP10 (6100-02-10-1036) and higher
AIX 6.1 TL3 SP7 (6100-03-07-1036) and higher
AIX 6.1 TL4 SP7 (6100-04-07-1036) and higher
AIX 6.1 TL5 SP3 (6100-05-03-1036) and higher
AIX 6.1 TL6 (all SP levels)
AIX 6.1 TL7 (all SP levels)
AIX 7.1 (no TL installed)

However, a few customers have reported this issue on lower SP levels.

IBM have investigated the error and released fixes

Solution

For AIX 6.1 TL6 SP1, IBM have released fix IZ88711:

ftp://public.dhe.ibm.com/aix/efixes/iz88711/

NOTE: Support have received feedback that the issue persists even after applying fix IZ88711 and that IBM has come up with IZ88711sp2 on top of IZ88711 to fix this issue. This has not been verified by Support and customers should check with IBM.

Fixes for other AIX 6.1 and AIX 7.1 TL's are also available:

IZ89304 for AIX 6.1 TL3
IZ89302 for AIX 6.1 TL4
IZ89300 for AIX 6.1 TL5
IZ88711 or IZ89514 for AIX 6.1 TL6 - check with IBM
IZ89165 for AIX 7.1 TL0 SP2

Please contact IBM to identify the correct fix for your environment.

No reboot is normally required after applying the patch.

After installing the AIX fix, run the following command:

% make -f $ORACLE_HOME/sysman/lib/ins_emagent.mk relink_sharedobj SHAREDOBJ=libnmemso

to link the libnmemso.so library

Alternative solution

If these errors occur on AIX 6.1 TL4 SP6/SP7, you may alternatively be hitting the AIX defect below:

IZ44217 CRASH CAUSED BY STALE FILE DESCRIPTOR IN FORKED CHILD

References

BUG:10383618 - AGENT INSTALL ON AIX 6.1:ERROR IN INVOKING TARGET 'AGENT NMB NMO NMHS EMSUBAGENT
NOTE:1300175.1 - Ld: 0711-711 Error: Input File $Oracle_home/Lib//Libnmemso.So Is Empty. The File Is Being Ignored.
NOTE:1298284.1 - Grid Control Agent Install on AIX 6.1 fails with ERROR IN INVOKING TARGET 'AGENT NMB NMO NMHS EMSUBAGENT'