dino

Securing the TMP Partition and Tracking Hacks

by

Are your temp partitions putting out behind your back? Anyone who’s ever administered a Linux server would know the risk of leaving the /tmp directory unsecured, moreso on a webserver that is shared among multiple websites.

The tmp directory is world-writeable and used by a majority of services on a machine — including the storage of PHP and MySQL session files. One issue that I’ve seen on older servers is that one customer’s poorly-coded website would get exploited and end up downloading a file into the /tmp directory, then have that hack file executed on the server as a nobody-owned process. Hack processes are the easiest to find, as they always have a little something extra.

This should go without saying, but hack processes can run from almost anywhere, not just /tmp. I’m mainly bringing up the tmp directory because it’s the most targeted location for hack files if it isn’t secured properly.

Identifying Hack Processes

To start out, log into your server as root as issue the following command to see all the nobody-owned processes running on the machine. This is assuming that Apache on your webserver runs as ‘nobody’:

root@localhost[~] ps -efw |grep nobody |more

A majority of the output will reflect legitimate Apache processes:

nobody 8748 25841 0 21:35 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL
nobody 8785 25841 0 21:35 ? 00:00:01 /usr/local/apache/bin/httpd -DSSL
nobody 8988 25841 0 21:36 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL

You’ll see that they all have the same parent process ID of ‘25841′ and all of the commands look about the same. However, you may see something that looks like this:

nobody 15707 26407 0 11:18 ? 00:00:00 [sh <defunct>]
nobody 15717 1 0 11:18 ? 00:00:04 /usr/bin/perl
nobody 13016 1 0 14:14 ? 00:00:00 /usr/bin/perl

nobody 8988 1 0 21:36 ? 00:00:00 /usr/local/apache/bin/httpd -DSSL -d3

All of these are hack processes. A few things to look for:

– Perl and shell (sh) processes on most systems run as the user that is executing them, not ‘nobody’. If this is true on your system, a nobody-owned perl process is probably a hack

– The parent ID of the process is different than all the others of the same service — these can be easily faked

– For Apache, an extra little switch on the end of the command (../httpd -DSSL -d3) – ‘d3′ is usually the name of the hack file that is being executed through Apache (not always).

Now I know you probably have the urge to kill -9 it, but don’t. Killing a hack process before determining what it is or where it came from is only going to let it happen again.

Tracking the process:

Once you know which processes should not be running, you need to know where they came from. The *easiest* way to track a process is through the lsof command. If you take the process ID (not the parent) and run an lsof it will tell you everything you want to know:

lsof -p 8748

From here I can see that this process is running from a folder in a user’s account. All I need to do now is kill it off, then remove the hack files from that user’s directory and let them know to update their shit.

TMP Directory Hacks

I’ve seen my fair share, but many hack processes will be spawned from the temp partitions of the server (like /tmp, /tmpDIR, /dev/shm), as these are usually set to 777 to allow programs to store their temporary files. When you first look in your TMP directory you’re going to see a lot of stuff — this is normal, but you’ll want to look for nobody-owned files that stand out. Just a hint, some hacks will try to trick you with their names. I have seen many that are named “…” or “httpd” to make them look legitimate, but I know that httpd (the service that runs Apache) should not be in the tmp directory…and that’s what makes it stand out.

Securing the tmp Partition

I’m not going to go too far into this here, because there’s already a very good website with this information available.

If you are on a cPanel server, you should run /scripts/securetmp in addition to following the instructions above.

Installing IonCube loader with Zend Optimizer – cPanel

by

This is a common request we get for Ioncube to be installed. It’s generally not an issue, but when you factor in other optimization plugins like Zend and eAccelerator, a common misconception is that the three don’t get along. It’s very easy to install Ioncube into a PHP installation that already has Zend and eAccelerator.

This tutorial is specific to cPanel, assuming that you are using php 5.2.x with Zend 3.x.x.

If you need help installing eAccelerator, you can see this tutorial. For help with installing Zend, you can go here. The versions in both these tutorials are outdated, so you’ll probably want to apply the instructions to the newest versions available.
Go to http://www.ioncube.com/loader_download.php and pick your download. This example assumes that you are using php 5.2.

cd /usr/src
wget http://downloads2.ioncube.com/loader_downloads/ioncube_loaders_lin_x86.tar.gz
tar -xvzf ioncube_loaders_lin_x86.tar.gz
cd ioncube

Copy the loader config to the user’s public_html or another location where you can access it from a browser.

cp ioncube-loader-helper.php /home/username/public_html

Now in your browser go to the loader file that you just copied. This file will tell you exactly which extension you need to use. Choose the ‘php.ini Installation Instructions’ link, and you should see something like this after the php config output:

zend_extension = /<path>/ioncube_loader_lin_5.2.so

Move the ioncube directory to a more permanent location:

mv /usr/src/ioncube /usr/local
chown -Rf root:root /usr/local/ioncube

Edit the php.ini and add look for this section (may not be exact):

[Zend]
zend_extension_manager.optimizer=/usr/local/Zend/lib/Optimizer-3.0.1
zend_extension_manager.optimizer_ts=/usr/local/Zend/lib/Optimizer_TS-3.0.1
zend_optimizer.version=3.0.1
zend_extension=/usr/local/Zend/lib/ZendExtensionManager.so
zend_extension_ts=/usr/local/Zend/lib/ZendExtensionManager_TS.so

Above this section, add this line:

zend_extension=/usr/local/ioncube/ioncube_loader_lin_5.2.so

Of course, make sure that the .so file is the one that the loader helper told you to use! After that is added, STOP and then START Apache to make sure that it’s loading. You should now see IonCube in your phpinfo file.

If you’re using eAccelerator, you shouldn’t need to change the location of the plugin loader in your php.ini.

Note that if Apache doesn’t start, it’s probably because of the order in which you have Zend and ioncube loading.  The lines for Ioncube should be above those for Zend optimizer.

Lastly, you should test your IonCube installation to make sure that it can decode its own files. In the original ‘ioncube’ directory that you moved, there’s a test ‘ioncube-encoded-file.php’ file that you can load through a browser to make sure that it works.

eAccelerator on PHP 5

by

Here is what I did to get eAccelerator 0.9.5 running on PHP 5 on my V-Dedicated server. I did the exact steps when I had PHP 4 running the only difference was that with PHP 4 I used the 0.9.4 install of eAccelerator instead of 0.9.5. Everything was done via SSH access.

1. cd /usr/local/src/
2. wget http://bart.eaccelerator.net/source/0.9.5/eaccelerator-0.9.5.tar.bz2
3. bunzip eaccelerator-0.9.5.tar.bz2
3. tar -xvf eaccelerator-0.9.5.tar
4. cd eaccelerator-0.9.5
5. Run the following command inside that directory /usr/local/bin/phpize. You should get output like the following

Configuring for:
PHP Api Version: 20041225
Zend Module Api No: 20050922
Zend Extension Api No: 220051025

6. Run the next commands to get it all configured to work with your system and compiled/installed

./configure –enable-eaccelerator=shared –with-php-config=/usr/local/bin/php-config
make
make install

7. After running the make install the system will tell you know where it installed the module to. Take note of that directory, it should be something much like the following

Installing shared extensions: /usr/local/lib/php/extensions/no-debug-non-zts -20050922/

you will need this information for the php.ini else the module will not load.

8. Now you need to create the eaccelerator cache directory and give it the correct permissions.

mkdir /tmp/eaccelerator
chmod 0777 /tmp/eaccelerator

9. This is reallt the last main step, Next you need to add the following into your php.ini so that the module loads.

;;;;;;;;;;;;;;;;
; eAccelerator ;
;;;;;;;;;;;;;;;;
extension=”/usr/local/lib/php/extensions/no-debug-non-zts-20050922/eaccelerator.so”
eaccelerator.shm_size=”16″
eaccelerator.cache_dir=”/tmp/eaccelerator”
eaccelerator.enable=”1″
eaccelerator.optimizer=”1″
eaccelerator.check_mtime=”1″
eaccelerator.debug=”0″
eaccelerator.filter=””
eaccelerator.shm_max=”0″
eaccelerator.shm_ttl=”0″
eaccelerator.shm_prune_period=”0″
eaccelerator.shm_only=”0″
eaccelerator.compress=”1″
eaccelerator.compress_level=”9″

Of course make sure to change out extension=”/usr/local/lib/php/extensions/no-debug-non-zts-20050922/eaccelerator.so” with the exact location of your module as told to you after the make install command.

10. Restart Apapche and it should be all set. You can load up your phpinfo page and you should see that the eAccelerator is loaded on the system

How to Upgrade PHP – cPanel

by

Whether you compiled manually or with EasyApache, running a PHP upgrade from a previous version is super easy but also one of the most common questions I get. There are 3 likely assumptions about your current environment (specific to Linux servers, sorry Windows users):

1. You are running a cPanel server and have PHP compiled by EasyApache

2. You compiled PHP manually from source

3. You used Yum, Aptitude, or another package manager

EasyApache

Perhaps the easiest (but least efficient time-wise), simply log into WebHost Manager > Apache Update and select “Previously Saved Config” option and “Start Customizing Based on Profile“. The next screen should take you to the Apache version, which you can keep the same or upgrade as well. Then you’ll be taken to select either PHP 4 or PHP 5, proceeded by the actual version you wish to run and then the options for both Apache and PHP on the next two steps (Advanced Configuration). You usually do not need to change the options if your target is just a simple upgrade within the same version family, but if you changed the Apache version, updated cPanel recently, or are upgrading to a different PHP version family (like 4.4.7 to 5.2.x), then you’ll want to double-check the Apache/PHP build options to make sure they are what they need to be before selecting “Save and Build.” If you are running older versions of EasyApache (usually with cPanel 10 or early versions of cPanel 11 STABLE) then all you have to do is select ‘Load Previous Config‘, pick your PHP version, and ‘Build‘.

Source

If you’ve compiled PHP from source (./configure && make && make install method), you can use your previous configure arguments to compile against a different version. Refer to your server’s phpinfo file (if you don’t have one just create a PHP script with <?php phpinfo(); ?>) and copy the entire ./configure statement into notepad and remove the single quotes. Once you’ve done this, download the source tarball of the new PHP version, untar, and then enter the installation directory. From there, all you have to do is paste the configure statement from notepad. For instance, for PHP 5.2.5:

wget http://www.php.net/get/php-5.2.5.tar.gz/from/this/mirror

tar -xvzf php-5.2.5.tar.gz

cd php 5.2.5

./configure –options-from-your-phpinfo

After the configuration is complete, the script may indicate at the end of its output that you’ve specified configuration options that do not exist. This usually will not affect your build, but you’ll want to review them and consult the PHP documentation on the correct syntax or alternatives to the invalid build options, as these can change depending on which version of PHP you are installing. Doing a ./configure –help will also display the valid options you can use.

Once you have a good configuration, you can go ahead with the make and make install to install the new version of PHP. If you have PHP integrated with Apache (usually you would unless you compiled as CGI) then the installation should have already updated the PHP binary for Apache and module loader in httpd.conf. However, you may need to manually comment out old module loaders if there are conflicts. You’re looking for something like this:

LoadModule php5_module        modules/libphp5.so

Package Installation

Some people have PHP installed via package manager, like Yum or Aptitude. Since the package software usually handles all aspects of the configuration and installation (outside of modules you may have installed via PEAR or Pecl), then you can use its update function to take care of the upgrade as well. Most have a specific option for upgrades, for instance Yum uses:

yum update php

For more information, see NixCraft’s article on PHP installations with Yum.

Preventing DDOS Attacks with Mod_Evasive

by

Denial of Service attacks are among the oldest yet most common form of attacking a server. Most system administrators have had to deal with DOS attacks taking down a server, router, or other networking device and know how difficult they can be to prevent.

Mod_evasive is an Apache module that limits the number of Apache connections to the server at once, and blocks an offending IP for a specified amount of time. This tutorial will show you how to install mod_evasive on your system.

1. Install:

wget http://www.zdziarski.com/projects/mod_evasive/mod_evasive_1.10.1.tar.gz
tar -xvzf mod_evasive_1.10.1.tar.gz
cd mod_evasive_1.10.1
/usr/local/apache/bin/apxs -cia mod_evasive.c

2. Configure:

Once the module is compiled, add these lines to httpd.conf and stop Apache completely before starting it up again:

<IfModule mod_evasive.c>
DOSHashTableSize 3097
DOSPageCount 6
DOSSiteCount 50
DOSPageInterval 2
DOSSiteInterval 2
DOSBlockingPeriod 600
</IfModule>

Below is an explanation of these settings:
DOSHashTableSize – Size of the hash table. The greater this setting, the more memory is required – faster

DOSPageCount – Max number of requests for the same page within the ‘DOSPageInterval’ interval

DOSSiteCount – Max number of requests for a given site, uses the ‘DOSSiteInterval’ interval.

DOSPageInterval – Interval for the ‘DOSPageCount’ threshold in second intervals.

DOSSiteInterval– Interval for the ‘DOSSiteCount’ threshold in second intervals.

DOSBlockingPeriod – Blocking period in seconds if any of the thresholds are met. The user will recieve a 403 (Forbidden) when blocked, and the timer will be reset each time the site gets hit when the user is still blocked.

Fixing Common Perl Issues – cPanel

by

Sometimes you’ll notice that a certain function of cPanel or WHM was working fine, now all of a sudden you’re getting results that you’ve never seen before. Believe it or not, many problems with cPanel functionality related to processes running off of scripts is because of a perl corruption. To resolve this, usually the first step you should take is running a cPanel update, as this will correct most issues. If the problem persists, you may need to recompile perl from it’s core.

Before we start, here are a few of the most common indicators that there is a perl issue on your server:

  • Accounts are not created properly, like missing files
  • Frontpage extensions refuse to be installed
  • Stats won’t run, result in cryptic perl errors
  • Error messages that look like this:
(internal death) Sat Jun 3 21:01:55 2006 [32719] error: List::Util object version 1.14 does not match
 bootstrap parameter 1.18 at /usr/lib/perl5/5.8.7/i686-linux/XSLoader.pm line 92.
==> Starting cpbandwd (bandwidth monitoring for IMAP/POP)
Can't locate Class/Std.pm in @INC (@INC contains: /usr/local/cpanel /usr/lib/perl5/5.8.7/i686-linux
/usr/lib/perl5/5.8.7
BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.7/Unix/PID.pm line 8.
Compilation failed in require at /usr/local/cpanel/bin/cpbandwd line 15.
BEGIN failed--compilation aborted at /usr/local/cpanel/bin/cpbandwd line 15.

In some cases, the error is specific about a perl module missing. If this is the case, look for the perlmod format in the error message. In the above example you can see ‘List::Util’ being referenced, so you can try to reinstall that either manually or from WHM:

/scripts/perlinstaller –force List::Util

If reinstalling the perl module itself does not solve the problem, you can recompile perl. First, do a perl -v from command line to see what version you are running. Then follow these steps:

wget http://layer1.cpanel.net/perl587installer.tar.gz
tar xvzf perl587installer.tar.gz
cd perl587installer
./install

/usr/local/cpanel/bin/checkperlmodules

If you are running 5.8.8, use the installer from:

http://layer1.cpanel.net/perl588installer.tar.gz

The recompile may take as long as 20 minutes to complete.

Downgrading MySQL to 4.x – cPanel

by

If you’ve upgraded MySQL via WebHost manager, the downgrade process is slightly more complex than the upgrade. This tutorial mainly covers a downgrade from 4.1 to 4.0, but also applies to downgrading from 5.x to 4.x.

*Back up all databases before proceeding, as all existing databases will be deleted!

1. Uninstall existing MySQL packages:

rpm -qa | grep -i mysql-

This should return a list of several rpm’s installed. You will need to remove them all with the rpm -e command .

2. Delete the active MySQL directory:

rm -Rfv /var/lib/mysql

3. Get cPanel Ready

Edit /var/cpanel/cpanel.config and change the version to the one that you are downgrading to (4.0, 4.1, etc)

mysql-version=4.0

Make sure that MySQL updates aren’t being skipped:

rm /etc/mysqldisable
rm /etc/mysqlupdisable

4. Install MySQL:

/scripts/mysqlup –force

5. Update the perl module:

/scripts/perlinstaller –force Bundle::DBD::mysql

Finally, recompile Apache and PHP via Web Host Manager.

Changing the SSH Port

by

Sometimes you don’t want SSH running on standard port 22. Here are the steps to change it:

It’s VERY important that you leave port 22 open while you are testing the new port, otherwise you may lock yourself out of the server!
SSH into the server normally…

cp /etc/ssh/sshd_config /etc/ssh/sshd_config.root

In the /etc/ssh/sshd_config.root file that you just made, specify a new port number with the Port line:

Port 9670
#Protocol 2,1
#ListenAddress 0.0.0.0
#ListenAddress ::

Also, make sure that PermitRootLogin is set to yes or commented out.
Now open /etc/init.d/sshd and locate this section:

# Some functions to make the below more readable
KEYGEN=/usr/bin/ssh-keygen

Directly below that, add this line:

OPTIONS=”-f /etc/ssh/sshd_config.root”

Now locate this line:

initlog -c “$SSHD $OPTIONS” && success || failure

Directly ABOVE that add this line:

initlog -c “$SSHD” && success || failure

Now you need to restart sshd (service sshd restart). Before shutting down port 22, make sure that whatever port you created for ssh is either opened in the firewall or has the appropriate IPs added. Make sure to test this in a separate ssh window to make sure you can log in!

Once you’ve verified that you can log in through the new port, you have a couple options for port 22.

  • Close it altogether or just open it for specific IPs as usual
  • Disable root logins through port 22 (preferred)

To disable root logins, open the original /etc/ssh/sshd_config file and uncomment out PermitRootLogin and set to No. This will essentially allow someone to log in as the user, but not root. You didn’t really change the SSH port, you’re just running a copy of it on another port and making port 22 useless.

If you want to disable port 22 altogether, you can just directly edit the /etc/ssh/sshd_config file and restart SSHD, but you only have one chance to get it right.

Once the port is changed you would ssh into the server with the command ssh -p 9670 root@server with 9670 being the port you chose for ssh.

Closing Open Nameservers – DNS recursion

by

Open nameservers allow anyone in the world to perform queries on them, which can often lead to DOS attacks and slower performance. Some system administrators prefer to have their nameservers restricted to only trust parties. To do this kind of setup, you will need to configure your named configuration:
On command line:

pico /etc/named.conf

Look for this line at the top:

include “/etc/rndc.key”;

Now add this right below it:

acl “trusted” {
205.134.252.71;66.117.3.128;127.0.0.1;
};

The IPs should be those of the nameservers…you can add other trusted IPs as well.

Now in the options section right below that, add these lines:

allow-recursion { trusted; };
allow-notify { trusted; };
allow-transfer { trusted; };

So your options section will look like this:

options {
directory “/var/named”;
dump-file “/var/named/data/cache_dump.db”;
statistics-file “/var/named/data/named_stats.txt”;
/*
* If there is a firewall between you and nameservers you want
* to talk to, you might need to uncomment the query-source
* directive below. Previous versions of BIND always asked
* questions using port 53, but BIND 8.1 uses an unprivileged
* port by default.
*/
// query-source address * port 53;
allow-recursion { trusted; };
allow-notify { trusted; };
allow-transfer { trusted; };
};

Restart named

/etc/init.d/named restart

Now you can check a service like DNSreport to make sure the changes took.

Disabling SMTP Tweak – cPanel

by

The SMTP tweak will prevent users from bypassing the mail server to send mail (this is a common practice used by spammers). It will only allow the MTA (mail transport agent), mailman, and root to connect to remote SMTP servers.  In order words, users will need to use ‘localhost’ as their SMTP server for local mailing scripts to sent mail (source).  SMTP Tweak can be enabled or disabled in the ‘Security Center’ of WHM, or from command line using:

/scripts/smtpmailgidonly on|off