Installation


Upgrades


DTC-Xen Installation


DTC-Xen / Dom0 Howtos

DTC-Xen / DomU Howtos

FAQ


DTC Howtos


Manuals


Features


Roadmap


Devel docs


Wiki - i18n


Wiki - Meta


SBOXAndDTC

1. What is SBOX?

SBOX is a cgi wrapper. Instead of running your (php, python, perl, ruby) scripts directly with your web server, SBOX is executed, and then SBOX execute your scripts with some restrictions.

2. Why using SBOX?

If you run some software, like a forum, a blog, a wiki, or whatever, sometimes they may have some defects. In such case, if you have a security problems with the web application you have installed, you really want this issue to affect only your website. It shouldn't affect your whole server, and the problem should be contained in the miss-behaving subdomain. That's what SBOX helps doing.

3. When is SBOX started?

DTC uses the AddHandler directive of Apache for certain types of file extensions. Here's a typical content of a VirtualHost directive that DTC generates:

   php_admin_flag engine off
   ScriptAlias /cgi-bin /usr/lib/cgi-bin
   php_admin_flag engine off 
   AddHandler php-cgi-wrapper .php
   Action php-cgi-wrapper /cgi-bin/sbox
   AddHandler python-cgi-wrapper .py
   Action python-cgi-wrapper /cgi-bin/sbox
   AddHandler ruby-cgi-wrapper .rb
   Action ruby-cgi-wrapper /cgi-bin/sbox
   AddHandler ruby-cgi-wrapper .pl
   Action ruby-cgi-wrapper /cgi-bin/sbox
   Options +ExecCGI

This means that when apache "sees" a file with .php, instead of running it with mod_php directly, it executes SBOX. Then SBOX in its turn sees what type of script is to be ran, and starts the corresponding interpreter so that the script is executed correctly.

But that's not it. Instead of just running the scripts, SBOX will perform a lot of actions to improve security, by limiting what the script will be able to do. See below.

4. Limiting what a script can do

There's all sorts of ways a script could misbehave, potentially harming your server. For example, a script could: - use all CPU - run all the time - use all memory - write huge files and fill up your hard drive - read/write files that it's not supposed to (like the ones of another customer on the same server)

With SBOX, all these are simply impossible.

FYI, this wiki (which is pmWiki), and the DTC forums (running a mildly old phpBB) are both running using SBOX, and this without a single issue. I have yet to find any PHP software that have issues running SBOX. When you find some problems, they are always fixable.

5. Putting scripts in a chroot jail

The most interesting feature of SBOX is probably the fact that it can do a chroot before executing a script. For example, if your web page runs in:

   /var/www/sites/username/example.com/subdomains/www/html/index.php

then sbox will perform a chroot in:

  /var/www/sites/username/example.com/subdomains/www

before executing. Meaning that your index.php script will "think" that it's running in:

   /html/index.php

instead. As a result, it's also going to be impossible for your index.php to read anything that doesn't belong to it, outside of this chroot jail.

But then, if your web app is in this chroot, it will not have access to all PHP modules and libraries. So it needs to have a local copy of a full operating system template in order to run. That's where the AUFS comes into play.

6. AUFS: union file system to save space

If you had 600 websites, then you wouldn't like to have 600 copies of a full system for each individual 600 VirtualHost. Which is why DTC uses AUFS: it will create a shadow copy of an operating system, shared among all sites.

Once you have installed DTC, you should do:

   cd /usr/share/dtc/admin
   ./create_sbox_bootstrap_copy
   ./update_sbox_bootstrap_copy

This will initialize a full operating system image in /var/lib/dtc/sbox_copy, using the shell tool "debootstrap". It will then customize it to match the SBOX needs (for example, it will tweak a few directives in the php.ini config file).

Then, once the cron.php of DTC will run, it will mount:

   /var/lib/dtc/sbox_copy

as read only source for AUFS

   /var/www/sites/username/example.com/subdomains/www

as read-write source for AUFS, and finally

   /var/www/sites/username/example.com/subdomains.aufs/www

as destination mount point.

This means that if you modify something in the subdomains.aufs/www folder, it will in fact be written in subdomains/www. This includes for example subdomains.aufs/www/etc/php5/cgi/php.ini. A copy will be stored in subdomains/www/etc/php5/cgi/php.ini so that the sbox_copy folder will not have to be touched.

So that's another cool feature for PHP users: each individual site may have a customize php.ini with tweaked directives.

7. What does it mean for my site design?

First of all, since your site is running in a chroot, you will not have access to the files of your main operating system. One of which is quite important:

   /var/run/mysqld/mysqld.sock

This is the socket file used to connect to MySQL. You will not have access to it. As a consequence, your web scripts will be forced to use network connectivity to MySQL instead of using the socket file.

The way to tell MySQL client is simply to use "127.0.0.1" instead of just "localhost". Unfortunately, many scripting language forces you to specify what is the address of your MySQL server, instead of leaving this as default from the MySQL client library.

PHP is one of these misbehaving scripting language. So, for example, if you have a phpBB forum, you will have to edit config.php, and change the dbhost variable to use 127.0.0.1 instead of localhost.

Also, each script has to be made executable, as another security measure. As an administrator wishing to switch a site to the sbox_aufs mode, you can do:

   find . -iname '*.php' -exec chmod +x {} \;
   find . -iname '*.php' -exec sed -i "s/localhost/127.0.0.1/" {} \;

In other words, what you can do to convert all your sites at once is:

   for i in /var/www/sites/*/*/subdomains/*/html ; do
      cd $i
      find . -iname '*.php' -exec chmod +x {} \;
   done
   for i in /var/www/sites/*/*/subdomains/*/html ; do
      cd $i/.. ; rm -rf bin dev etc lib lib64 libexec sbin usr var
   done
   cd /var/www/sites
   rm */*/bin */*/dev */*/etc */*/lib */*/lib64 */*/sbin */*/tmp */*/usr */*/var
   rm */bin */dev */etc */lib */lib64 */sbin */tmp */usr */var

Then manually checking for the replacement of localhost by 127.0.0.1. If you run a lot of Joomla sites, you can do:

   for i in /var/www/sites/*/*/subdomains/*/html/configuration.php ; do
      sed -i "s/var \$host = 'localhost';/var \$host = '127.0.0.1';/" $i
   fi

For wordpress:

   for i in /var/www/sites/*/*/subdomains/*/html/wp-config.php ; do
      sed -i "s/define('DB_HOST', 'localhost');/define('DB_HOST', '127.0.0.1');/" $i
   done

For Sugar CMS:

   for i in /var/www/sites/*/*/subdomains/*/html/config.inc.php ; do
      sed -i "s/\$dbconfig\['db_server'\] = 'localhost';/\$dbconfig\['db_server'\] = '127.0.0.1';/" $i
   done

8. Cleaning the old folders DTC was copying

Unfortunately, before SBOX was fully designed and ready for production, DTC was copying some chroot templates. For example, in the below path:

   /var/www/sites/username/example.com/subdomains/www

you may still find:

   bin
   dev
   etc
   lib
   lib64
   libexec
   sbin
   tmp
   usr
   var

These have to be cleaned, because they might hold outdated libraries, and it may prevent your PHP cgi script interpreter to work. So simply do:

   rm -f lib64;  rm -rf bin dev etc lib libexec sbin usr var

note that it's fine to leave the tmp folder. Be careful with lib64, which is a symlink to /lib -- DO NOT include a trailing "/" slash (eg. "./rm -rf lib64/"), or it will remove all files under /lib!

9. How does it performs?

Chances are, you wont notice any difference. SBOX is quite fast, and is very small. Some might then tell that apache has to fork a PHP / Python / Ruby interpreter each time a script has to be launched. Yes, that's truth, but if you have enough memory, chances are that your interpreter will STAY in the RAM, thanks to the efficient caching that Linux does.

Also, remember that SBOX is invoked only for scripting. On a normal site, for each script, you will have about one or 2 dozens of non-script files to serve (like images, css templates, and the like). These are served directly by Apache in the normal way, without the chroot mechanism.

10. The bad news

AUFS isn't very stable. Sometimes, when you mount and unmount often, it just crashes. It's a well known issue, and that's the reason why it's not in the Linux mainline kernels, but just as a module shipped with Debian. Luckily, if you don't do too many mounts and unmounts, it should be all right.

So just be aware that you may have to sometimes reboot your server, but it shouldn't happen so often. Also, we're looking into alternatives, like overlayfs, which might be better implementations.

One of the issue you might have is AUFS that would appear not to work, with some folders missing. Here's how to fix.

  • 1/ Unmount subdomains.aufs/www
  • 2/ Go in subdomains/www
  • 3/ There, you'll see lots of .wh..SOMETHING folders, with SOMETHING being usr, var, etc. Delete all these with: rm -rf .wh.*
  • 4/ Finally remount the folder with: cd /usr/share/dtc/admin && ./remount_aufs

11. Using sbox_aufs or sbox_copy?

Another very cool stuff with SBOX, is that since you are running your scripts in a chroot, you can run any version of the interpreter that you like, without putting your whole system at risk. Said otherwise: each site can run the PHP version that you like, and that, on the same server. That means that if you have a pretty old software that still runs only with PHP 5.2, and wont support PHP 5.3, then you can still do that on a Squeeze, or even Wheezy server. Simply, only your site will run on a chroot with an outdated operating system.

Yes, that's bad, yes, it's still a security issue, yes, it will take about 500 MB of files and a lots of inodes only for a single site, but it's still better to do that than to hold an upgrade to Debian stable, just because you have a stupid web software for which upstream didn't do the work of updating his scripts.

In such case, you'd be choosing to use the sbox_copy mode of DTC, and not the aufs, so that your VirtualHost can use it's own version of everything, including the script interpreter (again, this isn't only limited to PHP, this can be truth as well for Python, for example, if you need an old version of it).

12. Tweaking SBOX parameters

In the file /etc/sbox/sbox.conf, you will see lots of directives. They all have comments, so we wont run through each of them. Most likely, you will only need to tweak the memory ones to fit your needs. Remember that SBOX has precedence on things like the php.ini max_memory directives, and that your users will be left with very little clue of what's happening (only a "premature script end" or something similar in the error.log).

Also, if you have some site specific parameters to make, there's a convenient /etc/sbox/vhosts.d folder, where you can do that. If your site is www.example.com, then simply drop a copy of sbox.conf with that name:

   cp /etc/sbox/sbox.conf /etc/sbox/vhosts.d/www.example.com

and then change things in this file. SBOX will pick-up that file if it sees it there.

13. Cleaning your subdomains

If for one reason or another, your subdomains are full of operating system folders (for example if you did chown -R dtc.dtcgrp /var/www/sites, which you shouldn't...), then you might need to clean-up your subdomains. The following command will then help:

   for i in bin boot dev etc home lib lib64 media mnt opt proc root sbin selinux srv sys tmp usr var ; do
      rm -rf /var/www/sites/*/*/subdomains/*/$i
   done

14. Upgrading to progress-linux Squeeze backports kernel

Since the AUFS v2 module has often problems (eg: lockups, etc.), it might be a good idea to upgrade to AUFS v3, available with the backport kernel. But aufs v3 isn't available from backports.debian.org. So it's a good idea to use the backported kernel available from progress-linux. Progress-linux is maintained by Debian Developers, and is of a very good quality. The rules for this distribution is that only things who can't be added to Debian will be installed in the progress repositories. Anyway, here's how to do that:

   echo "deb https://cdn.archive.progress-linux.org/packages/(approve sites) artax-backports main" >>/etc/apt/sources.list
   wget https://cdn.archive.progress-linux.org/packages/project/pgp/archive-key-artax.asc(approve sites)
   apt-key add archive-key-artax.asc
   apt-get update
   apt-get install -t artax-backports  linux-image-3.2.0-4artax1-amd64 linux-headers-3.2.0-4artax1-amd64 linux-support-3.2.0-4artax1 aufs-tools

Then reboot and enjoy running AUFS v3.

15. Converting your DB to correct path problems

Since you're now moving to a chroot, you may want to replace the path that look like this:

   /var/www/site/USERNAME/example.com/subdomains/www/html/whatever

to something like that:

   /html/whatever

The way to do it, is to first dump all of our database, do a search and replace, then import back. You can do this with these simple shell commands:

   mkdir dbs
   cd dbs
   echo "#!/bin/sh
   set -e
   set -x" >dump.sql
   cat /var/lib/dtc/etc/net_backup.sh | grep mysqldump | grep -v dtcdb.sql | grep -v mysqldb.sql >dump.sql
   ./dump.sql
   for in i *.sql ; do sed -i 's#/var/www/sites/.*/.*/subdomains/.*/html#/html#g' $i ; done
   for in i *.sql ; do mysql --defaults-file=/etc/mysql/debian.cnf <$i ; done

It is of course advised to make a backup before doing this.

Editing this page means accepting its license.

Page last modified on May 21, 2017, at 08:37 AM EST