How to Disable ASLR and Why you Should do it

Update: Oct 28, 2013 – the below applies only to DB2 before DB2 10.1 Fixpack 3. IBM says that ASLR can safely be used with DB2 on 10.1 Fixpack 3 and later. See the technote at http://www-01.ibm.com/support/docview.wss?uid=swg21365583. FixPack 3 was released on September 27th. Also, please see comment from a reader at the bottom about the security implications of disabling ASLR.

This post is specific to Linux. The commands shared have been tested on RedHat 6.2.

What is ASLR

Address Space Layout Randomization is a security feature that randomizes where in a process’s address space key data is stored. This enhances security by making it harder for a hacker to find these key memory areas. I believe it is enabled by default on recent RedHat distributions. You can find more information on the concept here: http://en.wikipedia.org/wiki/Address_space_layout_randomization

Why ASLR Should be Disabled on all DB2 Database Servers

Quite simply, enabling ASLR can cause odd behavoir for a number of DB2 processes other than the core db2sysc. IBM details it in this technote: https://www-304.ibm.com/support/docview.wss?uid=swg21365583. DB2 maps a shared memory object across processes, and apparently it can’t always handle the addressing if ASLR is enabled.

The error message from the db2 diagnostic log that I’ve most frequently seen is one of these two:

2012-01-12-15.38.24.853671-300 E304269938E671      LEVEL: Error (OS)
PID     : 12732                TID  : 46912562375120PROC : db2bp
INSTANCE: db2inst1             NODE : 000
APPID   : *LOCAL.db2inst1.120117015847
FUNCTION: DB2 UDB, oper system services, sqlowqueInternal, probe:40
MESSAGE : ZRC=0x870F003E=-2029060034=SQLO_QUE_BAD_HANDLE "Bad Queue Handle"
          DIA8555C An invalid message queue handle was encountered.
CALLED  : OS, -, msgsnd                           OSERR: EINVAL (22)
DATA #1 : system V message queue identifier., PD_TYPE_SYSV_QUEUE_ID, 4 bytes
0x02F7000C
DATA #2 : Pointer, 8 bytes
0x00000000007d7890
DATA #3 : unsigned integer, 8 bytes
3841
2011-08-17-12.14.51.565199-300 I657879649E972      LEVEL: Error
PID     : 15948                TID  : 47622807768096PROC : db2egcf
INSTANCE: db2inst1             NODE : 000
FUNCTION: <0>, <0>, <0>, probe:100
MESSAGE : ZRC=0x850F0005=-2062614523=SQLO_NOSEG
          "No Storage Available for allocation"
          DIA8305C Memory allocation failure occurred.
DATA #1 : String, 22 bytes
sqlocshr.-2062614523..
CALLSTCK:
  [0] 0x00002B500232AA9D pdOSSeLoggingCallback + 0x91
  [1] 0x00002B50076EE5FB /opt/IBM/db2/v9.5/lib64/libdb2osse.so.1 + 0x1B25FB
  [2] 0x00002B50076EE3FC ossLog + 0xA6
  [3] 0x0000000000402BBE _Z21db2haAttachToDatabasePcP9SQLE_KRCBPP16sqeLocalDatabase + 0x22A
  [4] 0x0000000000402C58 _Z17db2haGetHADRStatePcP9HDR_STATES_ + 0x64
  [5] 0x0000000000402225 main + 0x1DD
  [6] 0x0000003ECC01D994 __libc_start_main + 0xF4
  [7] 0x0000000000401FAA pdOSSeLoggingCallback + 0x7A
  [8] 0x0000000000000000 ?unknown + 0x0
  [9] 0x0000000000000000 ?unknown + 0x0

In the second, you’ll note that the process – db2egcf – is a monitoring process for HADR. Since one of the possible failure scenarios with ASLR relates to unexpected HADR failovers, this fits right in with the tech note linked above.

How to Disable it

First, you must have root for this. Or use an SA as your Waldo if you are not able to get root yourself. There are two pieces to this – disabling it for your server currently and ensuring that change is persistent across reboots.

Runtime

This is quite simple – a single command:

# sysctl -w kernel.randomize_va_space=0

You do not have to restart db2 or the server for the change to take effect.

Saving it for all future reboots

To make your change persistent, you have to add a line to the /etc/sysctl.conf file (on RedHat). This can easily be accomplished using:

# echo "" >> /etc/sysctl.conf
# echo '# Disabling ASLR per IBM recommendation for DB2 servers:' >> /etc/sysctl.conf
# echo '#   ASLR may cause intermittent failures in shmat() for any db2 processes other than db2sysc.' >> /etc/sysctl.conf
# echo 'kernel.randomize_va_space = 0' >> /etc/sysctl.conf

Obviously, I’ve included comment lines here. They help to keep it clear why something is defined – my hope is that if some System Administrator sees that at some point in the future, they would research or contact a DBA before changing the setting

6 comments… add one

  • The comment is key – you want future admins to know why something was done and the potential impact of changing it. Nice article.

    Reply
  • I have seen the DIA8555C message appear in db2diag.log after the completion of a RESTORE.
    The RESTORE is in a shell script and includes WITH PROMPTING and is followed by the required ROLLFOWARD. The script stops dead after the successful RESTORE without indicating any errors . Is it possible that ASLR is the cause of this?
    PS All commands run succesfully from command prompt. DB2 V9.7.2 Linux Redhat

    Reply
  • ASLR has a function and quite an important one. It’s not the end-all security solution to end all hacking attempts ever, but it’s an important part of making it harder for hackers to gain access to systems.

    If you get problems like this, please update to the latest versions of your Linux distributions and DB2. If the problem persists, file a support call at IBM, describing them that they probably suffer from a bug in their software that makes it impossible to work reliably with ASLR enabled. Since most Government facilities require ASLR to be enabled, IBM can’t afford this to happen and they should be more than willing to fix this bug.

    Don’t fight symptoms, but fix buggy software. Don’t let DBAs make security or SA decisions, but let them escalate this to a place where they can properly solve the problem, not mask the cause.

    Reply
    • DB2 10.1 FixPack 3 corrects this issue. I’ll update the blog entry to reflect that.

      Reply

Leave a Comment