TODO items
This is a list of (sub)projects which we think might make paphosting a little
bit more appealing/useful. They are listed in random order, in as much or as
little details we thought necessary to relay the idea.
- RRD Statistics - take the config/papfe.hosts file and
hit those haproxy machines at their /stats;csv output, figure out
which backends and frontends there are defined, and create RRD archives from
them. Ideally do this every 5 minutes (as is 'normal' for RRDs) and distribute
the RRDs to the other (thttpd) machines, or better yet, run the collector on
all the other (thttpd) machines. Then, we can create a CGI for thttpd which
can display graphs of things like hits/sec or bits/sec, and even a ~real time
view of which front- and backends are up.
- Geo DNS - we can have multiple haproxy frontends on multiple
continents, but then putting them all into one set of responses will make
clients jump around continents a lot. It'd be great if we could give out
the american haproxy addresses to american IPs, and european ones to european
IPs - powerdns does this. We could use
it. For now though, we should just stick to frontends at Coloclue, BIT and
True (and possibly IP-Man) - this is where we have the IP space to build out
SSL VIPs.
Items we've done:
- NGINX - Engine X, an HTTP and SSL (de)multiplexer. We now host
SSL VIPs only at the frontends, and speak regular HTTP at the backends (for example, with SixXS). This will allow us to build out SSL capacity in a
distributed fashion (ie. our frontends handle things like gzipping content
and SSL encrypting it), as well as build out more capacity by allowing the
frontends to serve static content themselves, and not bothering Apache
with that type of work. Done per Jan 2010.
- Auto Copy - we currently copy sites that are not within the RCS
out to peers (note: don't call them slaves, they are supposed to be
identical) automatically. For example, if people upload or edit their site on
bfib, we have a script that rsync's the /www directory every
hour (this would be sufficient for the be-apache and be-thttpd
backends, as users primarily hit bfib until it goes away).
Done per Dec 2009.
- Monitoring - high availability yes, but monitoring? How's about
YES! nagios looks after all of our physical machines, and their
services (even nginx SSL VIPs), and alerts us when things are down or
unavailable otherwise. We also have http-regtest.py which can read
a suite of tests to do (IPv4, IPv6, SSL, normal HTTP) and validates that
changes to our infrastructure do not compromise the user experience in an
obvious or bad way.
Done per Jan 2010.
- Apache Setup - documented the file with apache setup instructions for Ubuntu and OpenBSD. This was a pretty
involved step because we'd like to make sure we have a relatively feature-rich
set on those machines. Ideally, we would not have the entire httpd config in
RCS, but create an Include that allows us to root our websites, and otherwise
a well-known logs and siteroot location.
Done for Ubuntu/Debian per Jan 2010, pending Apache2+PHP5 completeness for
OpenBSD.