su Forking and the Incorrect Trapping of SIGINT (CTRL-C)
by Seth Miller
This issue taunted me from the first day at my current employer. It took about a month before I figuratively threw my hands in the air and proclaimed that I would delay what I was working on to find the cure.
The symptoms of the issue were very difficult to search for since it was somewhat hard to describe the behavior. Basically, I would log into the server as myself. Then I would su to oracle with sudo. Once I was at the command prompt as the oracle user (or any other non-privileged user), I would either want to clear what I had on the command line or cancel a tail using a ctrl-c. However, when I typed ctrl-c, I was brought back to the user I logged into the server with.
What I found out later when investigating the problem was that the process I tried to cancel out of (tail for example) was still running in another pseudo-terminal (pts). The issue turned out to be an unintended consequence of a security fix to su.
What really threw me off was that this issue was reproducible on all of the servers except for one, which led me to believe it was a pts or shell setting of some kind.
I had access to the root account, so I tried suing to root first, then suing to oracle which worked just fine. In addition to the obscurity caused by the inconsistency between servers, the lack of reproducibility when not using sudo seemed to point to sudo as the culprit.
I probed some people that had been running into the issue for awhile to see what had been done. The issue had in fact been brought to the Unix team who could not reproduce it. They were just suing without using the full sudo command that I was limited to using.
The full sudo command I was using to switch to the oracle user was:
sudo /bin/su - oracle -c /usr/bin/sudosh
The last part of the command is a monitoring tool to replay the terminal output of a session. The behavior is that the user is switched to oracle and sudosh returns control of the terminal back to the screen in a similar way that screen does.
There did not need to be anything running to pass a SIGINT to the terminal. Once the signal was passed, the prompt was returned to the original user. In addition, the terminal settings were completely different from what they were before. There is now no echo of characters returned to the terminal and line feed settings seem to be corrupted.
[oracle ~]$ (control c pressed here) [milles ~]$ [milles ~]$ [milles ~]$ [milles ~]$ [milles ~]$ [milles ~]$ [milles ~]$ Password: su: incorrect password [milles ~]$
I found out that the terminal settings are being cleared. Resetting them (stty sane) gets back to a normal terminal but now I have to sudo again and start over on what I was doing.
Here are the stty settings in a broken and fixed (normal) state:
Broken: speed 38400 baud; line = 0; intr = <undef>; quit = <undef>; erase = <undef>; kill = <undef>; eof = ^A; start = <undef>; stop = <undef>; susp = <undef>; rprnt = <undef>; werase = <undef>; lnext = <undef>; flush = <undef>; min = 0; time = 0; -icrnl ixany -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke Normal: [milles ~]$ stty –a speed 38400 baud; rows 45; columns 120; line = 0; intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0; -parenb -parodd cs8 -hupcl -cstopb cread -clocal -crtscts -cdtrdsr -ignbrk brkint -ignpar -parmrk -inpck istrip -inlcr -igncr icrnl ixon -ixoff -iuclc -ixany imaxbel -iutf8 opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0 isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke
I tried trapping SIGINT and several other signals with various methods including the trap command. The signals never made it to the terminal which led me to believe that the terminal I was working in did not have the functionality of a fully operation terminal. In other words, another terminal was trapping my signals before they could reach my working terminal.
Looking at the processes confirmed that there was indeed two terminals running, one a child of the other. The child terminal was hosting sudosh which my command prompt was running out of.
[milles ~]$ sudo /bin/su - oracle -c /usr/bin/sudosh [oracle ~]$ ps -ef | grep sudo | grep -v grep root 16258 2670 0 21:03 pts/2 00:00:00 /bin/su - oracle -c /usr/bin/sudosh oracle 16279 16258 0 21:03 pts/1 00:00:00 /usr/bin/sudosh
However, in the server that was not reproducing the issue and functioning correctly, sudosh was running as a different process, but in the same terminal.
[milles ~]$ sudo /bin/su - oracle -c /usr/bin/sudosh [oracle ~]$ ps -ef | grep sudo | grep -v grep root 4615 5676 0 15:00 pts/1 00:00:00 /bin/su - oracle -c /usr/bin/sudosh oracle 4616 4615 0 15:00 pts/1 00:00:00 /usr/bin/sudosh
The biggest difference between these two systems is age, but the second biggest difference is version of the operating system. The latter example above was running RHAS 4 and the former running OEL 5.
su is part of an RPM called coreutils. I checked the difference in version of coreutils between the two servers as well:
[milles@RHAS4 ~]$ rpm -qi coreutils Name : coreutils Relocations: (not relocatable) Version : <strong>5.2.1</strong> Vendor: Red Hat, Inc. Release : 31.8.el4 Build Date: Thu 29 May 2008 08:09:45 AM CDT Install Date: Wed 03 Feb 2010 03:22:53 PM CST Build Host: hs20-bc1-5.build.redhat.com Group : System Environment/Base Source RPM: coreutils-5.2.1-31.8.el4.src.rpm Size : 7655912 License: GPL Signature : DSA/SHA1, Tue 10 Jun 2008 01:22:58 PM CDT, Key ID 219180cddb42a60e Packager : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla> URL : ftp://alpha.gnu.org/gnu/coreutils/ Summary : The GNU core utilities: a set of tools commonly used in shell scripts Description : These are the GNU core utilities. This package is the combination of the old GNU fileutils, sh-utils, and textutils packages.
[milles@OEL5 ~]$ rpm -qi coreutils Name : coreutils Relocations: (not relocatable) Version : <strong>5.97</strong> Vendor: Oracle USA Release : 23.el5_4.2 Build Date: Wed 24 Feb 2010 12:35:24 PM GMT Install Date: Fri 10 Jun 2011 06:42:00 PM GMT Build Host: ca-build9.us.oracle.com Group : System Environment/Base Source RPM: coreutils-5.97-23.el5_4.2.src.rpm Size : 9035080 License: GPLv2+ Signature : DSA/SHA1, Wed 24 Feb 2010 12:35:52 PM GMT, Key ID 66ced3de1e5e0159 URL : http://www.gnu.org/software/coreutils/ Summary : The GNU core utilities: a set of tools commonly used in shell scripts Description : These are the GNU core utilities. This package is the combination of the old GNU fileutils, sh-utils, and textutils packages.
Through the painful and long process of troubleshooting the issue, I found that there was a bug found somewhere between these two versions that identified a security vulnerability. The change to fork any programs started using the –c flag was put in version 5.93-2. The developer later identified that there was a valid reason to keep the ability to run programs within the same terminal despite the security flaw so they included the – -session-command flag to give the program the former functionality.
sudo runs every command it executes as the root user. The purpose of the new fix was to prevent users from running a program like sudosh that allows a user to be returned to a prompt and be able to kill that command and drop down into the parent root process, therefore giving them complete root access. With the fix in place, the user could drop out of the program executed, but they would be dropped into the forked non-privileged terminal.
This terminal forking also affects other built-ins and functionality such as shopt. When running the shell inside the child terminal, the automatic resize of the shell window if the terminal program (i.e. Putty) changes the window size was not functioning and the window size variables had to be changed manually.
The final fix was to run the sudosh program with the flag – -session-command instead of -c. This essentially reverts the functionality of su to its former vulnerable self but gives the signal trapping and other standard functionality back to the terminal in which it belongs.
Using this workaround to regain functionality is not ideal since it takes away a layer of the security onion. However, using sudosh is not the most secure way of monitoring session activity either. The time saved using the workaround is tremendous. Although I couldn’t find examples of others running into this issue, they must be out there. I hope this helps someone.
Please leave comments if you see any inaccuracies.