So long Debian servers, welcome Ubuntu

I’ve been assessing the possibility of switching GNU/Linux Matters‘ servers (which are all powered by Debian) to Ubuntu, and I have finally decided to go for it.

They were running Debian for three reasons:

  1. Stability, something Debian is well-known for.
  2. A large user base, which brings two big advantages: A huge amount of packaged applications and a good community support.
  3. I’m only familiar with Debian-based distros :)

And they weren’t running Ubuntu because, to be honest, I didn’t find Ubuntu reliable enough to power a server, mainly because of the cutting-edge applications it includes by default; it was just great for my personal computer. I guess this is mostly due to I’ve been using Kubuntu since Breeze, and it was a highly unreliable system in the early versions – IMHO things begun to take shape in Feisty and now Hardy just rocks.

I’ve started to switch our servers to Ubuntu because I think it’s the best choice, at least for us, because we still have the three advantages of using Debian (it’s based on it after all), plus:

  1. Packaged applications are up-to-date, so I don’t have to compile and maintain software which have a old version in Debian repositories (we often need the latest stable version). Yes, I can use Debian Testing, but this is not the only drawback.
  2. Uncomplicated Firewall. An extremely easy-to-setup, basic firewall. It’s just a front-end to iptables-restore, so you can still add/adjust any rule according to your needs.
  3. Many other tiny (and not so tiny) benefits that together make a big difference.

The migration should take some months because it’s not a top-priority at present. There are many more things that should be done first.

Put simply, it’s not that Ubuntu is much better than Debian on the server, it just happens to make your job easier if you’re the administrator.

Installing Pootle on Debian Etch, The Easiest Way

Pootle is a user-friendly web application for online translation of contents. It’s used by organizations like Creative Commons, OpenOffice.org and GNU/Linux Matters.

I’m going to show you how to install it the easiest way, from my experience with the GLM Translation Service under Debian 4.0 (this guide might also work under Ubuntu, though). I’ll use the sudo utility, so I assume that you’ve already installed and configured sudo accordingly for your user.

Installing dependencies

Some of the Pootle’s dependencies are available in Debian repositories, so we may install them by running:
sudo aptitude install python-dev python-pysqlite2 python-celementtree subversion bzip2 g++ curl

We’re now going to install the other software packages required by Pootle. First, select the directory where you want to download and compile your software, for example:
cd /usr/src

To get the source code of the packages and extract them, run the following commands:
curl http://www.kid-templating.org/dist/0.9.6/kid-0.9.6.tar.gz | tar xzv
curl http://dfn.dl.sourceforge.net/sourceforge/translate/translate-toolkit-1.1.1.tar.bz2 | tar xjv
curl http://garr.dl.sourceforge.net/sourceforge/translate/Pootle-1.1.0.tar.bz2 | tar xjv
curl http://jtoolkit.sourceforge.net/snapshots/jToolkit-0.7.8/jToolkit-0.7.8.tar.gz | tar xzv

Installing the applications:
sudo python translate-toolkit-1.1.1/setup.py install
sudo python kid-0.9.6/setup.py install
sudo Pootle-1.1.0/pootlesetup.py install
sudo jToolkit-0.7.8/jToolkitSetup.py install

Do not install PyLucene

Pootle is supposed to work better with PyLucene, but if you (like me) hate headaches, you won’t install PyLucene on Debian Etch. If you try to install it, you’d get this error:
install: cannot stat `/usr/lib/security/libgcj.security': No such file or directory
make: *** [install] Error 1

There’s nothing you could install that would create that file; don’t you even try to create it as an empty file: You’d be able to install PyLucene, but Pootle won’t work and you’d get this error when you’ll try to run it:
GC Warning: Repeated allocation of very large block (appr. size 65536):
May lead to memory leak and poor performance.

If it’s too late, and you’ve already installed PyLucene with libgcj.security as an empty file (as suggested in some mailing-lists), you may fix it by running the following commands:
sudo mv /usr/lib/python2.4/site-packages/_PyLucene.so /usr/lib/python2.4/site-packages/_PyLucene.so.old
sudo mv /usr/lib/python2.4/site-packages/PyLucene.py /usr/lib/python2.4/site-packages/PyLucene.py.old
sudo mv /usr/lib/python2.4/site-packages/PyLucene.pyc /usr/lib/python2.4/site-packages/PyLucene.pyc.old

Don’t you have a mail server?

If your mail server is not in the same host as Pootle, you’ll need to install one. Just run the command below to install Postfix and then select “Internet website” when asked about what you’ll use postfix for:
sudo apt-get install postfix

Configuring Pootle

To make it easy for you to configure and play with Pootle, I suggest you create softlinks to Pootle’s configuration files in a folder like /etc/pootle:
sudo mkdir /etc/pootle
sudo ln /usr/lib/python2.4/site-packages/Pootle/pootle.prefs -s /etc/pootle/pootle.conf
sudo ln /usr/lib/python2.4/site-packages/Pootle/users.prefs -s /etc/pootle/users.conf
sudo ln /usr/lib/python2.4/site-packages/Pootle/html -s /etc/pootle/html
sudo ln /usr/lib/python2.4/site-packages/Pootle/templates -s /etc/pootle/templates

This way, any file you might ever need to edit will be in /etc/pootle.

Now let’s edit the main configuration file in Pootle: sudo nano /etc/pootle/pootle.conf

It’s well-documented, so I won’t talk a lot about it. I just suggest you only set the following parameters accordingly:

  • description: Describe your website powered by Pootle, with a text aimed at (potential) translators.
  • fromaddress.
  • supportaddress: The email address for translators to make questions. This might be a mailing list.
  • defaultrights: I suggest you set it to defaultrights = "view, suggest, archive, pocompile, translate, commit"
  • podirectory: It’s the path to your translation files. For this HOWTO I’ll use /var/translations.

Your pootle user

I suggest you create a user for running pootle:
sudo adduser --disabled-password --disabled-login pootle

Setting up service scripts for Pootle

Create the file /etc/init.d/pootle as root with the following contents:
#!/bin/bash
# /etc/init.d/pootle: start and stop the Pootle Server
# Pootle runs as user -pootle- via sudo
# This script is based on the one published here http://translate.sourceforge.net/wiki/pootle/service_scripts
PREFIX=/usr/local/pootle
test -x /usr/bin/PootleServer || exit 0
. /lib/lsb/init-functions
case "$1" in
start)
log_begin_msg "Starting Pootle Server..."
/usr/bin/sudo -u pootle $PREFIX/start_pootle &
log_end_msg 0
;;
stop)
log_begin_msg "Stopping Pootle Server..."
$PREFIX/stop_pootle >/dev/null 2>&1
log_end_msg 0
;;
restart)
log_begin_msg "Stopping Pootle Server..."
$PREFIX/stop_pootle >/dev/null 2>&1
log_end_msg 0
log_begin_msg "Starting Pootle Server..."
sudo -u pootle $PREFIX/start_pootle &
log_end_msg 0
;;
status)
$PREFIX/status_pootle
;;
*)
log_success_msg "Usage: /etc/init.d/pootle {start|stop|restart|status}"
exit 1
esac
exit 0

Now let’s create, as root, the auxiliary scripts required by the script above:

/usr/local/pootle/start_pootle

#!/bin/bash
# $PREFIX/start_pootle
# Start script in port 8080, separates errors from normal logs
# based on a script published here: http://translate.sourceforge.net/wiki/pootle/service_scripts
export HOME="/home/pootle"
/usr/bin/PootleServer -p 8080 $@ >> /var/log/pootle/`date "+%F"` 2>> /var/log/pootle/`date "+%F"`.err

/usr/local/pootle/stop_pootle


#!/bin/bash
# Stop script $PREFIX/stop_pootle
# based on a script published here: http://translate.sourceforge.net/wiki/pootle/service_scripts
echo "Killing Pootle"
pidpootle=`ps -ef |grep PootleServer |grep python | awk '{print $2}'`
pidpootle2=`ps -ef |grep start_pootle |grep bin | awk '{print $2}'`
kill $pidpootle2 >/dev/null 2>&1
kill $pidpootle >/dev/null 2>&1

/usr/local/pootle/status_pootle

#!/bin/bash
# Pootle Status $PREFIX/status_pootle
# based on a script published here: http://translate.sourceforge.net/wiki/pootle/service_scripts
pidpootle=`ps -ef |grep PootleServer |grep python | awk '{print $2}'`
if [ "$pidpootle" != "" ]
then
echo "Pootle Server running in pid ($pidpootle)"
else
echo "Pootle Server is not running"
fi

Make them work!

Now we have to make these scripts useful:
sudo chmod +x /etc/init.d/pootle
sudo chmod +x /usr/local/pootle/s*
sudo update-rc.d pootle

Configuring Pootle to use your repository

Pootle supports several VCSs, but I’ll only teach you how to setup a Subversion working copy that Pootle may update and commit. If you don’t use a version control system, I recommend you do so. If you’re sure you don’t need it, you may skip this section. If you’re using another VCS, you’ll need to go to the Pootle’s website to learn how to configure it with Pootle.

Setup your working copy

Let’s create the working copy at, say, /var/translations. To do that in GNU/Linux Matters, we would run:
sudo svn co https://svn.gnulinuxmatters.org:81/i18n /var/translations
You should replace the URL by yours.
And don’t forget to set the pootle user as the owner:
sudo chown pootle -R /var/translations

Create basic cronjobs

We should run from time to time two scripts:

  • One to commit the changes made by translators; ideally, these changes would be committed by translators themselves, but from my experience you should not count on that (most of them forget about it or just don’t know they can do that).
  • Another to update your PO templates from the repository.

So we’re going to ask our pootle user to do this by creating cronjobs. First, become pootle:
sudo su - pootle
Then run crontab -e and add the following two lines:
*/15 * * * * /usr/bin/svn update /var/translations >> /dev/null 2>> /home/pootle/update.err
0 0,8,16 * * * /usr/bin/svn commit -m "Uncommitted translations in the last 8 hours" /var/translations >> /dev/null 2>> /home/pootle/commit.err

Pootle is running on port 8080

For security reasons, if you followed this HOWTO you’ll be running as the pootle user, not as root. And due to a UNIX misfeature, only root may open ports below 1024. So, unless you love appending the port number to your URLs, and if port 80 is available on your system, you’d proxy Pootle through a webserver.

The proxy trick is recommended, but if you cannot do that or just don’t want, you can use iptables to redirect traffic from port 8080 to 80. My suggestion would be to use the next iptables ruleset:
# Generated by iptables-save v1.3.6 on Thu Jan 24 16:40:59 2008
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080
COMMIT
# Completed on Thu Jan 24 16:40:59 2008
# Generated by iptables-save v1.3.6 on Thu Jan 24 16:40:59 2008
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [495:60715]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Thu Jan 24 16:40:59 2008

For more information about this basic iptables-based firewall, you may read my Setting up your first server HOWTO.

Criticism? Suggestions? Complaints?

Leave a comment!

The only screencasting software that doesn’t crash in Ubuntu Hardy

I’ve spent a lot of time trying to record some screencasts for GLM, but every screencasting software I tried crashed under Ubuntu Hardy (and Gutsy too):

  • XVidCap: The workaround is supposed to be disabling sound, but didn’t work.
  • RecordMyDesktop (plus its two front-ends): Crashes with no error message. From time to time I was able to record videos.
  • Istanbul: Crashes if you select a window to be recorded.

So, the only solution is to install a newer version of Istanbul for Debian:
http://ftp.fr.debian.org/debian/pool/main/i/istanbul/istanbul_0.2.2-4_i386.deb
or, if you have a 64-bit box, http://ftp.fr.debian.org/debian/pool/main/i/istanbul/istanbul_0.2.2-4_amd64.deb

After making the appropriate bug reports, I can finally start recording the screencasts with Istanbul!

PS: Forget about the above. That Debian package doesn’t work either. I’m installing KDE4 to check whether I can record the screencast with it; I think I saw such an option when I tried it out.

PPS: The KDE4 built-in screencasting component uses a weird format (cps?) which cannot be opened with VLC, Mplayer nor Kaffeine (but there’s a dirty workaround). Anyways, it records the whole screen and I just want to record a single window.

PPPS: I’m not alone.

How to install setlock on Debian?

That’s the only question Google and IRC don’t answer – but after this post Google will.

About two hours wasted to find out the freaking stupid solution:

aptitude install daemontools-installer
build-daemontools

And hit ENTER for every question.

Setting up your first server

If you’re new to server management and use the terminal on GNU/Linux from time to time, this guide will hopefully come in handy for you to get started with your own server.

Unless you have good reasons to use another GNU/Linux distribution, I recommend you to install Debian. It has a lot of ready-to-install applications, is very stable and it’s perhaps the distribution with more tutorials around.

Please notice that this is a very basic tutorial and has only been tested on Debian.

Connect to your server

First of all, log in as root:
ssh root@123.123.123.123 # where 123.123.123.123 is your server's IP address

Some hosting providers disable ssh root access, so you will need to replace root by your user name. If this is the case, after you log in you should become root:

su -

Update your system

aptitude update
aptitude upgrade
aptitude dist-upgrade

Add your user

If your hosting provider disables root access, then you should skip this step.

adduser emacs

Replace emacs by VI VI VI if you don’t believe in Saint IGNUcius.

Sudo setup

sudo is a very useful utility, and I recommend you to use it.

First, let’s install it:

aptitude install sudo

Then, we add your user to the list of sudoers, by running visudo and then adding the following line at the end of the file emacs ALL=(ALL) ALL.

Now you become yourself:

su emacs -

Shared key ssh authentication

At this point you should use shared key ssh authentication, but for that there’s a great tutorial at ammonlauritzen.com.

Configuring the SSH daemon

Open /etc/ssh/sshd_config with your favorite text editor, say:

sudo nano /etc/ssh/sshd_config

And make sure the following lines are set this way, if not, add or modify them accordingly:
PermitRootLogin no
PasswordAuthentication no
X11Forwarding no
UsePAM no
AllowUsers emacs # separate two or more usernames by spaces

Finally, apply your modifications:
sudo /etc/init.d/ssh reload

Don’t log out yet, we need to check that you will be able to access your server via ssh (this is, that you didn’t break anything on the /etc/ssh/sshd_config file). To check if everything is OK, try to log in:
ssh emacs@123.123.123.123

If you’re able to access, then it’s well configured and you may close the second session. If not, then you should check your modifications and try again.

Setting up a basic firewall

We are going to setup a very basic firewall with the powerful netfilter/iptables. For this step you need to be root:
sudo -s

First, store the current iptables rules, in case something goes wrong with ours:
iptables-save > /etc/iptables.conf.old

Now, create the file /etc/iptables.conf and add the following contents:
# boring stuff for someone new to server administration
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [495:60715]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state INVALID -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# this is the port used by the SSH daemon
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
COMMIT

Please pay attention to this line:
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

You should use a line similar for every open port that you want to be accessible from the Internet. This is, if you have a webserver, you should copy that line but replace “22” by “80” (or any other port):
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT

This is how you “enable” ports.

Then we load the configuration (and don’t log out until we test it!):
iptables-restore < /etc/iptables.conf

Testing the rules

To test the rules, open another terminal and try to access your server:

ssh emacs@123.123.123.123

If you could access, then the rules should be OK. If not, reload the original rules until you find help:
iptables-restore < /etc/iptables.conf.old

Loading the rules when the server stars

If the rules we defined work, then our the firewall should be loaded when the server starts:

Create the file /etc/network/if-pre-up.d/iptables with the following contents:
#!/bin/bash
/sbin/iptables-restore < /etc/iptables.conf

Then make it executable:
chmod +x /etc/network/if-pre-up.d/iptables

We can now go back to our normal user:
exit

Your server is ready!

At this point, you are ready to start installing applications on your brand-new server!

What’s next?