How to Disable ASLR and Why you Should do it
Update: Oct 28, 2013 – the below applies only to DB2 before DB2 10.1 Fixpack 3. IBM says that ASLR can safely be used with DB2 on 10.1 Fixpack 3 and later. See the technote at http://www-01.ibm.com/support/docview.wss?uid=swg21365583. FixPack 3 was released on September 27th. Also, please see comment from a reader at the bottom about the security implications of disabling ASLR.
This post is specific to Linux. The commands shared have been tested on RedHat 6.2.
What is ASLR
Address Space Layout Randomization is a security feature that randomizes where in a process’s address space key data is stored. This enhances security by making it harder for a hacker to find these key memory areas. I believe it is enabled by default on recent RedHat distributions. You can find more information on the concept here: http://en.wikipedia.org/wiki/Address_space_layout_randomization
Why ASLR Should be Disabled on all DB2 Database Servers
Quite simply, enabling ASLR can cause odd behavoir for a number of DB2 processes other than the core db2sysc. IBM details it in this technote: https://www-304.ibm.com/support/docview.wss?uid=swg21365583. DB2 maps a shared memory object across processes, and apparently it can’t always handle the addressing if ASLR is enabled.
The error message from the db2 diagnostic log that I’ve most frequently seen is one of these two:
2012-01-12-188.8.131.523671-300 E304269938E671 LEVEL: Error (OS) PID : 12732 TID : 46912562375120PROC : db2bp INSTANCE: db2inst1 NODE : 000 APPID : *LOCAL.db2inst1.120117015847 FUNCTION: DB2 UDB, oper system services, sqlowqueInternal, probe:40 MESSAGE : ZRC=0x870F003E=-2029060034=SQLO_QUE_BAD_HANDLE "Bad Queue Handle" DIA8555C An invalid message queue handle was encountered. CALLED : OS, -, msgsnd OSERR: EINVAL (22) DATA #1 : system V message queue identifier., PD_TYPE_SYSV_QUEUE_ID, 4 bytes 0x02F7000C DATA #2 : Pointer, 8 bytes 0x00000000007d7890 DATA #3 : unsigned integer, 8 bytes 3841
2011-08-17-184.108.40.2065199-300 I657879649E972 LEVEL: Error PID : 15948 TID : 47622807768096PROC : db2egcf INSTANCE: db2inst1 NODE : 000 FUNCTION: <0>, <0>, <0>, probe:100 MESSAGE : ZRC=0x850F0005=-2062614523=SQLO_NOSEG "No Storage Available for allocation" DIA8305C Memory allocation failure occurred. DATA #1 : String, 22 bytes sqlocshr.-2062614523.. CALLSTCK:  0x00002B500232AA9D pdOSSeLoggingCallback + 0x91  0x00002B50076EE5FB /opt/IBM/db2/v9.5/lib64/libdb2osse.so.1 + 0x1B25FB  0x00002B50076EE3FC ossLog + 0xA6  0x0000000000402BBE _Z21db2haAttachToDatabasePcP9SQLE_KRCBPP16sqeLocalDatabase + 0x22A  0x0000000000402C58 _Z17db2haGetHADRStatePcP9HDR_STATES_ + 0x64  0x0000000000402225 main + 0x1DD  0x0000003ECC01D994 __libc_start_main + 0xF4  0x0000000000401FAA pdOSSeLoggingCallback + 0x7A  0x0000000000000000 ?unknown + 0x0  0x0000000000000000 ?unknown + 0x0
In the second, you’ll note that the process – db2egcf – is a monitoring process for HADR. Since one of the possible failure scenarios with ASLR relates to unexpected HADR failovers, this fits right in with the tech note linked above.
How to Disable it
First, you must have root for this. Or use an SA as your Waldo if you are not able to get root yourself. There are two pieces to this – disabling it for your server currently and ensuring that change is persistent across reboots.
This is quite simple – a single command:
# sysctl -w kernel.randomize_va_space=0
You do not have to restart db2 or the server for the change to take effect.
Saving it for all future reboots
To make your change persistent, you have to add a line to the /etc/sysctl.conf file (on RedHat). This can easily be accomplished using:
# echo "" >> /etc/sysctl.conf # echo '# Disabling ASLR per IBM recommendation for DB2 servers:' >> /etc/sysctl.conf # echo '# ASLR may cause intermittent failures in shmat() for any db2 processes other than db2sysc.' >> /etc/sysctl.conf # echo 'kernel.randomize_va_space = 0' >> /etc/sysctl.conf
Obviously, I’ve included comment lines here. They help to keep it clear why something is defined – my hope is that if some System Administrator sees that at some point in the future, they would research or contact a DBA before changing the setting