Today I woke up to several alerts from Linode informing me that one of my VPS nodes was exceeding the Disk I/O threshold that I had set. Curiously this VPS is used as a HTTP web proxy and whilst it gets about 300-400 visitors per day (mainly china) this morning I was seeing over 800 visitors in Google Analytics.

Attempting to ssh to the server failed with timeouts although the PHP web application was still responding to requests over HTTP fine. I suspect sshd was failing to reverse-lookup my IP address in any reasonable amount of time, or perhaps IP Tables – (Note to self: Look in to why that happened). Thankfully Linode provide out of band / console access via SSH and AJax so all was not lost.

Looking at the Network rrdgraph it shows that the server was approaching 7Mbit/s of HTTP traffic and almost 50GB had been consumed today alone. Whilst the server seemed to handle the load without problem (minus ssh access) consuming 50GB+ per day would quickly max out my monthly data transfer allowance with Linode – this wasn’t acceptable. I modified the firewall to accept HTTP/HTTPS traffic from my IP only in order to investigate and the load suddenly stopped and SSH was alive again.

Initially I had suspected that some sort of automated bot was using ehproxy.info to do automated scans and attacks but a closer inspection of the traffic showed an even number of distributed IPs (all from China – as Google Analytics confirms) all clicking various porn sites. I guess everyone in China was feeling horny this afternoon!

Further analysis of the access.log shows that the server (Linode XenU VPS with 720MB of ram) was handling 62 hits sec (2428863/39600) and lighttpd was dealing with the load no problem. Pretty good considering this is a pure PHP application utilising php-cgi.

For the record, the top five IP addresses were:

Hits : IP address

Our client in Redwood City, CA is in need of a seasoned system administrator and operations engineer to manage, maintain and support multiple environments supporting several different architectures and environments. The successful candidate will have strong exposure to large web site deployments with multiple tiers include web front-ends, middleware components, backend services, data storage systems and 2nd/3rd tier components like caching and search layers.

Requirements:

• 7 years Linux admin experience
• 5 years RHEL admin experience
• 5 years RPM management and admin experience
• 3 years working with MySQL
• Sound understanding of web services and related technologies (HTTP, REST, XML, JSON, etc)
• Experience with Linux web servers (Apache2, nginx, lighttpd, etc)
• Experience with monitoring systems and components (nagios, cacti, etc)
• Exposure to scripting languages such as Perl, Python, etc.

Really nice-to-haves but not required:

• Experience with Erlang
• Experience with Ruby on Rails
• Exposure to Jabber/XMPP
• Exposure to distributed / grid computing
• ejabberd administration and configuration
• Experience with second and third tier services and components like memcached (or like systems), search applications (lucene, xapian, sphinx, etc), etc.

Realistically, these are also nice to haves but not required:

• Exposure to functional programming languages.
• Exposure to the Facebook Platform, Open Social and or other web services and systems
• Comfortable working in an Agile work environment.
• Plays games and occasionally keeps tabs on the gaming world/industry.

If interested, please send us your resume along with the rate per hour, contact number, and availability for a phone interview to raj@mindsource.com.

A rather simple but often asked question was put forward to me today: How can I see the maximum amount of file descriptors my running process can open? (without killing the process!)

Typically one would say ‘check ulimit -n’ but lets say that a thread-driven or event-driven application like varnish or lighttpd is configured with an arbitrary amount of open file descriptors and you want to verify that they have taken effect before the application crashes.

A simple way to check this (atleast on Linux 2.6.26-1 or later) is to run:

svr1:~# awk '/Max open files/{ print $4}' /proc/$(pgrep -n lighttpd)/limits

1024

As you can see the above command returned the value of max open files for the running process. This means you can be sure that your lighttpd or varnish application will not suddenly die after being starved of file descriptors!

I have included the entire output of the limits table for the lighttpd process for completeness:

svr1:~# cat /proc/$(pgrep -n lighttpd)/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            ms
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             5824                 5824                 processes
Max open files            1024                 1024                 files
Max locked memory         32768                32768                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       5824                 5824                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

First of all, there are several wiki posts at pylonshq.com already, below are some of them:

• http://pylonsbook.com/en/1.0/deployment.html
• http://wiki.pylonshq.com/display/pylonscookbook/Running+Pylons+with+NGINX
• http://wiki.pylonshq.com/display/pylonscookbook/Running+Pylons+with+SCGI+and+Lighttpd

I believe there’s not a single best solution in choosing any of these strategies. This post is intended as sharing my experience in some of these. I’m definitely not set on particular one and will change my mind once I gained better understanding.

My testing methodology

I’m currently testing multiple deployment configurations on the same app, thus giving me the opportunity to blog about it. I use Apache Benchmark (on client side), top and ps afx on server side. My machine is 1 Linode 1440 instance.

My AB setup are:

ab -n 500 -c 50 -k http://rootapp.com/

ab -n 1200 -c 50 -k http://rootapp.com/

ab -n 1500 -c 50 -k http://rootapp.com/

ab -n 800 -c 800 -k http://rootapp.com/

The results varied insignificantly with some subtle interesting differences. I’ll explain those below.

Note: I am testing these configuration on dynamic AJAX-y web application, thus reporting hard numbers is not very useful.

CherryPy vs Paste HTTP Server behind NGINX

One thing  that I noticed immediately is that CherryPy has better performance than Paste’s HTTP server. On both, having multiple processes does not help much on overall performance, but significantly reduces number of failed requests. When run under multiple processes, CherryPy consistently have the least number of failed requests.

For my setup having 7-8 processes is the sweet spot. When I have more than that, top is telling me that the latter processes are under utilized.

Setting up CherryPy on your production.ini is painless:

use = egg:PasteScript#cherrypy
numthreads = 20
request_queue_size = 512
host = %(http_host)s
port = %(http_port)s

By just comparing the two, CherryPy is easily the winner.

Lighttpd and SCGI

This gist is basic configuration to get SCGI up and running on lighttpd while the following is setup for your production.ini:

use = egg:Flup#scgi_thread
host = %(http_host)s
port = %(http_port)s

given the same AB configuration as CherryPy and Paste counterpart, lighttpd and SCGI consistently capable of handling 30 requests/seconds. About 8-10 requests/seconds more than CherryPy. Even though this setup is better, I noticed that memory consumption continues to go up after 2 weeks. I haven’t spend much time in investigating why. The reason I didn’t choose this path is more because I simply like NGINX better.

If only SCGI module on NGINX isn’t so experimental.

NGINX and FastCGI

This gist is basic configuration to get FastCGI running on NGINX while the following is setup for your production.ini:

use = egg:Flup#fcgi_thread
host = %(http_host)s
port = %(http_port)s

With this configuration, I consistently get about 25 requests/seconds. It’s a bit behind lighttpd and SCGI configuration. Interestingly, when run under ab -n 1500 -c 50 -k, this configuration hangs NGINX requiring it to be restarted. It only happen once though.

Again, when load balanced properly (depending on your app), any one of these configurations would work well. Hopefully this post can help others to get up to speed in Pylons deployment.

hello friends i am thinking about a web server which is better in terms of performance, configuration and adminstration.

i have read some articles like bellow, you also can check:

1) apache vs nginx vs lighttpd? Which is simpler to configure and administer?
http://stackoverflow.com/questions/475386/apache-vs-nginx-vs-lighttpd-which-is-simpler-to-configure-and-administer

2) Lighttpd vs Apache HTTP Server
http://durgaprasad.wordpress.com/2006/09/28/lighttpd-vs-apache-http-server/

3) Lighttpd vs nginx
http://www.wikivs.com/wiki/Lighttpd_vs_nginx

4) Apache vs TrustLeap G-WAN vs Rock
http://trustleap.com/

5) nGinx vs Lighttpd – web server of choice
http://blog.grik.net/2007/08/nginx-vs-lighttpd-web-server-of-choice.html

6) Nginx vs. Lighttpd for a small VPS
http://hostingfu.com/article/nginx-vs-lighttpd-for-a-small-vps

7) Nginx, my new favorite front end for mongrel cluster
http://brainspl.at/articles/2006/08/23/nginx-my-new-favorite-front-end-for-mongrel-cluster

8.) The Other free Web Server out there!

http://www.cherokee-project.com/

Writing a 2c web app is a lot harder than it looks.

Objective

Write a mockup prototype for a web-based application. Tools: anything you like, as long as the job is done quickly and can easily be modified to accomodate rapidly-evolving requirements.  In fact, this application is meant as a living demonstration of the future full-fledged stuff re-programmed later with something industrial-grade. Think of it as specifications that compile and can actually be run.

As the local Python expert I thought I would demonstrate how it can quickly get the job done with a minimal amount of efforts. Little did I know…

Choosing the web server

We are dealing with something that will essentially be web-based, choosing the appropriate web server seems the first thing to do.

I have worked with two web servers in the past: Apache and lighttpd. Apache is notoriously difficult to configure, the config file is full of traps and possible inconsistencies. There are complete books and tutorials on the Net about how to configure your own server and believe me they are all worth consulting. Apache is a really good server but if you have never used it you’d better plan a couple of weeks in advance to learn how to use it correctly.

lighttpd (pronounced “lighty”) is a really fast and lightweight server, much easier to configure. Once you have it installed you can literally have it run within minutes. Unfortunately it does not support https client-side certificates (yet) and that feature is needed for what I want to do. One guy recently submitted a related patch but unfortunately I could not get it to work against every version of lighttpd I could find. Exit lighttpd, welcome Apache!

First attempt: Python CGI script

Python comes with a cgi module that is great for writing demo scripts but quickly becomes a serious pain whenever you want to implement anything more complicated like logins, sessions or database-related stuff. This is really bare-bone but maybe a little too much. After spending half a day re-coding a session mechanism I finally gave up and moved on to the next stage.

Second attempt: Apache + mod_python

mod_python is great! A Publisher algorithm browses through your Python files and publishes on the web server anything that looks like a string or a callable. Imagine a static server responding to these URLs:

http://[server]/hello
http://[server]/world

If you have a Python module containing two top-level strings named ‘hello’ and ‘world’, they will be published by mod_python and displayed verbatim. More interesting of course is to use callables (functions or instances) for a dynamic site.

Took me a complete week to finish the site with mod_python but development was a breeze. I spent more time in my application business than with the tools themselves, which is the main reason for using tools like Python.

And then I reached a stopping point: I need to authorize web clients to upload XML files to the server in an unusual MIME type. Unfortunately mod_python offers no support to do this and even worse: it silently absorbs the uploaded files and does not even bother warning your application that it missed a client request. Going through mod_python forums I could find that somebody else already mentioned this to the developers but the feature was rejected because if you want to do serious web stuff you should move to WSGI.

At that point I could have gone back to CGI for the file uploading stuff but I did not want to live with a schizophrenic code being half CGI half mod_python. Besides, I do not even want to know how much time I would have needed to make this work in the Apache configuration file. Time to leave mod_python behind and move on.

Third attempt: Python WSGI

Now I have to wade through this infamous WSGI stuff and see if it is really worth all the buzz. To make things short: WSGI is a pure Python standard that specifies how a Web framework should behave at its lowest level. The intention is to make it easier to port a WSGI-compliant application from one framework or web server to another without having to re-code anything.

I read the full specifications for WSGI and I have to admit I did not really understand the motivations behind this design. But oh well, I trust the guys to have done a good job at factorizing web frameworks. The WSGI standard itself is really low-level.  There is no way you can develop a web site just armed with it, it is only meant for middleware providers. So let’s hunt for WSGI middleware!

WSGI stage 1: Django, Pylons

Django and Pylons are full-fledged frameworks that come up with all bells and whistles. Nothing bad with these but they do suffer from the same issues, namely:

• They offer about a zillion features I do not care about
• They cover almost everything I need, but not quite

Which means that I will probably end up deploying lots of packages I will never use and will have to code additional functionalities into their framework just to cover my own needs.

Both packages come with half a million dependencies on various additional packages, and every package means more maintenance.  I spent a couple of days on each to try and explore and came to the conclusion that it must be really great to use them as a basis for a larger project but I would not want to do it.

Mental note: I need to train myself on these frameworks, it might come handy some day.

WSGI stage 2: Bottle

Bottle is a lightweight WSGI environment all contained within a single file. Can’t beat that in terms of the fewest dependencies!  It offers a very simple syntax to route your URLs to your objects and makes for clean code like:

@route('/admin')
def administration():
return '.... html page here ....'

@route('/')
def index():
return '.... html page here ....'

Nice package, but I would largely prefer having the framework pick routable objects directly from my Python modules, like mod_python’s Publisher does. There were some other features missing from it and Bottle does not seem to be maintained any more, so I reluctantly decided not to use it.

Just out of curiosity I also tried to run Bottle within lighttpd, loosing another evening in the process. lighttpd does not support WSGI, you have to install yet another middleware layer (python-flup) and run the server in FCGI mode. After a whole evening of messing around I still could not get any Hello World out of my setup and ended up tracking an obscure bug in the way lighttpd spawns sub-processes. I do not have the courage to get into that in depth.

My conclusion on lighttpd: great for serving static files, still a long way to go before it can compete with Apache. I have no doubt the lighttpd guys will eventually get there though.

WSGI stage 3: Colubrid, Werkzeug

Colubrid offers exactly the kind of thing I need: a Publisher algorithm that goes through your objects and publishes them at predictable URLs. It took me no more than an hour to transform my mod_python application to Colubrid and see it run. Documentation for this project is pretty sparse though, and it is unfortunately not maintained any more. The authors refer to Werkzeug as the tool of choice now.

Enters Werkzeug: described as a library of WSGI helpers, it tends to suffer from the same overweight issues as Django or Pylons. A lot of dependencies on other libraries and a model that is really hard to understand. I spent a couple of hours going through the tutorial and could not make sense out of it. It is probably very powerful but seems inadequate for my brain.

So Colubrid it will be. It is unmaintained but the library does not force other dependencies and even if it has little documentation I can at least understand it. If I ever face issues I will modify it to suit my needs without fear of seeing my own patches overwritten by a new version. I found a couple of bugs but no showstopper for the moment.

Wrapping it up

I learned quite a lot in the process. Python is sufficiently high-level to expect development to be quick and to the point.  And in a way, that goal is pretty much achieved. Getting a dynamic web application is just a matter of coding your business logic into classes and then hooking them into a View and a database.

On the other hand, the sheer number of dependencies for most frameworks is a definitive showstopper for production. Many of these libraries are still relatively young and lack the polish needed to adapt from the package developer’s needs to your own.

Another thing I learned is that as time goes on, Python frameworks tend to become more and more complicated, to the point that there is little left for people like me who want to just have something that handles the HTTP protocol and lets you hook in the tools you need one at a time.

Oh well… Give me enough time and I might just end up writing my own.

Looks like you are writing a framework

The image hosting at techpowerup.org is running on our mail server which is an old, old machine still hosted at ThePlanet. I’m reluctant to move it because properly setting up a mail server takes a long time and could break a lot of things – wouldn’t want to miss a “you won the lottery” notification.

So I had this box sitting around sorting emails all day. Why not put something useful on it…

Back in 2006 all free image hosting services sucked big time (don’t they still today?), so why not setup some free image hosting.

It’s all really simple:

• Give user some kind of upload page with file browser,
• upload image to Apache sitting on www.techpowerup.org,
• use PHP and ImageMagick to process it,
• put it somewhere on the HDD.
• Give the user a link to img.techpowerup.com which is a different IP on which Lighttpd is running that serves the images from the HDD.

Why Lighttpd ? Apache sucks to serve static content. When using mod_php every single apache worker consumes around 20 MB of RAM. Due to the prefork design you can run a maximum of 255 concurrent workers. So either you run out of memory ( 255 * 20 = 5.1 GB) or out of workers, but you are never able to handle more than 255 concurrent requests at a time.

Lighttpd has been designed and optimized to serve static content fast and with very limited memory use.

I’ve seen Lighttpd on our servers with several thousand connections using only a few hundred MB of memory and almost no CPU (note: disk seeks will start killing you at this point).

Another good Lighttpd alternative is nginx.

Main two reasons why we use Lighttpd and not nginx is that nginx doesn’t have a dynamic plugin architecture, all “plugins” have to be compiled in at compile time. This means constant nginx recompiles.
Second point is that Lighttpd has a nicer status module than nginx.

Right now Lighttpd development cycles are extremely slow and I doubt we will ever see a major new release version.

Now, jailed has its own automated scripts to generate its own proper fully-contained jails (as far as “jails” on GNU/Linux go, that is; they’re chroots, actually).

The following scripts are leftovers from me trying to do the same, before finding out that the utilities are already there. They have been lying around for quite some time. I’m just copying them here so I can delete them from the home dir.

=====

jailed-lighttpd-install
—–

#!/bin/sh
# Create a jail for lighttpd

jail=/jail/lighttpd

mkdir -p $jail
cd $jail

mkdir tmp
chmod 1777 tmp

mkdir -p        var/log/lighttpd var/run/lighttpd home/http
chown http:http var/log/lighttpd var/run/lighttpd home/http
chmod a+rx      var/log/lighttpd var/run/lighttpd home/http

mkdir etc
cp -avr /etc/php etc/
cp -v /etc/hosts /etc/nsswitch.conf /etc/resolv.conf /etc/services /etc/localtime etc/
cp -v /etc/group /etc/host.conf /etc/passwd /etc/protocols etc/

mkdir -p usr/bin
cp /usr/bin/php /usr/bin/php-cgi usr/bin/

# read l2chroot before use
./l2chroot /usr/bin/php
./l2chroot /usr/bin/php-cgi

# just in case some permissions were forgotten...
chmod a+rx $jail

=====

jailed-lighttpd.rc
—–

#!/bin/bash

# general config
. /etc/rc.conf
. /etc/rc.d/functions

jailroot=/jail/lighttpd
function jailcmd () {
su http -c "$*"
}

PID=`pidof -o %PPID /usr/sbin/lighttpd`

case "$1" in
start)
stat_busy "Starting jailed lighttpd Daemon"
[ -z "$PID" ] && jk_chrootlaunch -j $jailroot -x /usr/sbin/lighttpd -- -f /etc/lighttpd/lighttpd.conf 2>&1
if [ $? -gt 0 ]; then
stat_fail
else
add_daemon lighttpd
stat_done
fi
;;
stop)
stat_busy "Stopping jailed lighttpd Daemon"
[ ! -z "$PID" ] && jailcmd kill $PID &>/dev/null
if [ $? -gt 0 ]; then
stat_fail
else
rm_daemon lighttpd
rm -f $jailroot/var/run/lighttpd/lighttpd.pid
stat_done
fi
;;
restart)
$0 stop
sleep 1
$0 start
;;
*)
echo "usage: $0 {start|stop|restart}"
esac

=====

l2chroot
—–

#!/bin/bash
# Use this script to copy shared (libs) files to Apache/Lighttpd chrooted
# jail server.
# ----------------------------------------------------------------------------
# Written by nixCraft <http://www.cyberciti.biz/tips/>
# (c) 2006 nixCraft under GNU GPL v2.0+
# + Added ld-linux support
# + Added error checking support
# ------------------------------------------------------------------------------
# See url for usage:
# http://www.cyberciti.biz/tips/howto-setup-lighttpd-php-mysql-chrooted-jail.html
# -------------------------------------------------------------------------------
# Set CHROOT directory name
BASE="/jail/lighttpd"

if [ $# -eq 0 ]; then
echo "Syntax : $0 /path/to/executable"
echo "Example: $0 /usr/bin/php5-cgi"
exit 1
fi

[ ! $BASE ] && mkdir -p $BASE || :

# iggy ld-linux* file as it is not shared one
FILES="$(ldd $1 | awk '{ print $3 }' |egrep -v ^'\(')"

echo "Copying shared files/libs to $BASE..."
for i in $FILES
do
d="$(dirname $i)"
[ ! -d $BASE$d ] && mkdir -p $BASE$d || :
/bin/cp $i $BASE$d
done

# copy /lib/ld-linux* or /lib64/ld-linux* to $BASE/$sldlsubdir
# get ld-linux full file location
sldl="$(ldd $1 | grep 'ld-linux' | awk '{ print $1}')"
# now get sub-dir
sldlsubdir="$(dirname $sldl)"

if [ ! -f $BASE$sldl ];
then
echo "Copying $sldl $BASE$sldlsubdir..."
/bin/cp $sldl $BASE$sldlsubdir
else
:
fi

=====

There. Sorry for the tabbing, it went AWOL.

Voilà, j’ai (enfin) reçu ma Dedibox, c’est parti pour un petit tour d’horizon !

————————————————————————————————————-

Matériel, Dedibox XL :

Core2Duo @ 2.0GHz
3 Go DDR2
500 Go DD
100 Mbit/s

Installation :

Debian 5.0 (64bits)
swap de 2 Go
/home de 100 Go
/var de 300 Go
le reste pour le /

Première prise de contrôle :

Avec putty, toute première chose à faire “sécuriser” le serveur. Pour ce faire j’ai décidé de changer le port de SSH et d’interdire le login en tant que root.
Une fois connecté à votre serveur, tapez :

su

puis votre mot de passe root, ensuite :

nano /etc/ssh/sshd_config

dans ce fichier, changer le port par un numéro de votre choix (au-dessus de 1000 par exemple), et à la ligne PermitRootLogin remplacez “yes” par “no”.

Et enfin :

/etc/init.d/ssh restart

Déconnectez-vous, modifiez le port dans putty, puis essayer de vous loger en root, et / ou avec l’ancien port (22). Si tout c’est bien passé, vous ne devriez plus pouvoir vous connecter en root, ni sur le port 22.

Voilà, pour la base de la base niveau sécurisation.

Mise à jour :

En root, tapez les commandes suivantes (dans l’ordre indiqué) :

apt-get update

apt-get upgrade

Installation de Lighty :

Pour ce faire, suivez le guide, ici !

I have a VPS with 384MB RAM available as standard, and the option to burst up to 1GB.

This is enough to run some fairly high traffic sites if you’re careful with the setup, but with the default setups and fancy control panels, you’ll exceed the memory limit quite quickly.

I use Citadel as the email client, but I’ve disabled Webcit.  I also removed Spam Assassin, and just use the Realtime Blackhole List to prevent spam.  Those simple changes have shaved over 100MB off my normal usage, but there’s still a lot to go.

top shows that memory usage is leaking up into the burst limit over time – even with some basic optimizations of httpd.conf and MySQL.  I’m considering trying lighttpd instead of Apache2 – I really don’t NEED Apache2, and it seems to be quiet a hog.

The other possible optimization would be to use something lightweight in place of named – which is currently sat at 180m memory usage!

If anyone has any favourite lightweight alternatives to named, I’d love to hear about them.

“HowtoForge provides user-friendly Linux tutorials about almost every topic.”

howtoforge contiene tantissimi tutorial davvero per tutti i gusti.

Prima di googlare su come si configura l’ssh, come si abilita virtualbox 3 o su tutto quello che vi passa per la testa, vi consiglio di fare sempre un salto in questo paradiso di guide online che copre qualsiasi argomento d’interesse ( del mondo linux ovviamente ):

Howtos, Linux, CentOS, Debian, Fedora, Kernel, Mandriva, PCLinuxOS, SuSE, Ubuntu, Apache, Backup, Control Panels, DNS, Desktop, Email, Anti-Spam/Virus, Postfix, Procmail, Sendmail, FTP, High-Availability, Lighttpd, Monitoring, MySQL, Programming, C/C++, PHP, Samba, Security, Anti-Spam/Virus, Virtualization, Other, FreeBSD, Commercial, Mini-Howtos.

Link www.howtoforge.com

Rails es un completo entorno para desarrollar aplicaciones web con base de datos de acuerdo con la estructura Model-View-Control. Desde el Ajax en la vista, a la petición y respuesta en el controlador, hasta el modelo, Rails te da un entorno de desarrollo de Ruby. Para probarlo, solo necesitas una base de datos y un servidor web.

Todo el mundo desde startups a organizaciones sin ánimo de lucro pasando por compañías están usando Rails. Rails es todo acerca de la infraestructura así que se acopla perfectamente en cualquier tipo de aplicacion web Es software para colaboración, comunidad, e-commercio, cms, estatísticas, control, como quieras llamarlo. Algunos ejemplos:

Yang diambil dari “layartembak” tapi kalau kata id.wikipedia.org sih “Cuplikan Layar“

Gara-gara ini jadi lupa kalau ternyata jatah hidup dah minus 1 tahun.

$HTTP["url"] =~ "/\.svn/" {
  url.access-deny = ( "" )
}

[lighttpd]
comment = LigHTTPd HTTP server
paths = /usr/sbin/lighttpd, /usr/sbin/lighttpd-angel, /usr/lib/lighttpd, /etc/lighttpd
emptydirs = /tmp, /srv/http, /var/log/lighttpd, /var/run/lighttpd
devices = /dev/urandom, /dev/null
users = http
groups = http
includesections = uidbasics, netbasics, logbasics

[php]
comment = PHP 5
paths = /usr/bin/php, /usr/bin/php-cgi, /usr/bin/cgi-fcgi, /usr/bin/spawn-fcgi, /etc/php
includesections = netbasics, logbasics

Permissions need to be changed in your jaildir.

jailed-lighttpd:

(include file lost, sorry)

Apache is almost considered a standard when it comes to a linux web server.  It’s performance is pretty good especially comparing it to it’s windows counterpart, IIS.  However, it’s not the fastest…. In setting up a web farm using haproxy, I discovered that the Apache performance was disappointing when scaling it over hardware and pushing the load to the farm. I spend hours and hours tweaking and tuning apache to get the maximum performance.  Well, hopefully this document can help you save time avoiding that and rather just going with lighttpd.

For info on lighthttpd http://www.lighttpd.net/ see here.  Large deployments like youtube, wikipedia and others use this for their http server.  It’s ideal for PHP configurations and of course static webpages.  It does have limitation due to it’s small footprint, so it’s not perfect for everything.

This configuration will work with Debian Lenny or Ubuntu 8.0.4 LTS.

First, install the server (stop apache services if running or change to different port)

apt-get install lighttpd
apt-get install php5-cgi

Second, enable php

nano /etc/php5/cgi/php.ini
cgi.fix_path_info = 1

Third, enable php in lighthttpd

nano /etc/lighthttpd/lighthttpd.conf
nano /etc/lighthttpd/lighthttpd.conf
server.modules              = (
“mod_access”,
“mod_alias”,
“mod_accesslog”,
“mod_compress”,
“mod_fastcgi
)

add “mod_fastcgi”

##tweaks (depends on memory and CPU) so run some tests – max-proc and min-proc specify how many php processes to spawn, make sure you adequately assign according to memory, default is 2
server.max-connections = 250
server.max-fds = 625

fastcgi.server = ( “.php” => ((
“bin-path” => “/usr/bin/php5-cgi”,
“socket” => “/tmp/php.socket”,
“max-procs” => 4,
“min-procs” => 4,
“bin-environment” => (
“PHP_FCGI_CHILDREN” => “16″,
“PHP_FCGI_MAX_REQUESTS” => “10000″
),
“bin-copy-environment” => (
“PATH”, “SHELL”, “USER”
),
“broken-scriptfilename” => “enable”
)))

reset lighthttpd
/etc/init.d/lightttpd

DONE-now enjoy! default web root is /var/www

Extra-Need a SSL Cert? Here’s the config:

1. generate ssl cert using openssl (see other docs on the net)
2. mkdir /sslcerts     cd /sslcerts
3. cat your.key your.crt > lms.pem
4. nano /etc/lighthttpd/lighthttpd.conf
$SERVER["socket"] == “10.10.13.161:443″ {
ssl.engine = “enable”
ssl.pemfile = “/sslcerts/.your.pem”
ssl.ca-file = “/sslcerts/your.crt”
server.name = “lms”
server.document-root = “/var/websites/lms/www”
server.errorlog = “/var/log/lighttpd/lmsserror.log”
accesslog.filename = “/var/log/lighttpd/lmsaccess.log”
}

So-what are the benchmark results? using apachebench – 51 requests per second vs 21. Convinced?

apache2-

Server Software:        Apache/2.2.9
Server Hostname:
Server Port:            443
SSL/TLS Protocol:       TLSv1/SSLv3,DHE-RSA-AES256-SHA,1024,256

Document Path:          /
Document Length:        50825 bytes

Concurrency Level:      250
Time taken for tests:   33.324 seconds
Complete requests:      700
Failed requests:        0
Write errors:           0
Total transferred:      36709423 bytes
HTML transferred:       36135271 bytes
Requests per second:    21.01 [#/sec] (mean)
Time per request:       11901.542 [ms] (mean)
Time per request:       47.606 [ms] (mean, across all concurrent requests)
Transfer rate:          1075.76 [Kbytes/sec] received

lighthttpd
Server Software:        lighttpd/1.4.19
Server Hostname:
Server Port:            443
SSL/TLS Protocol:       TLSv1/SSLv3,AES256-SHA,1024,256

Document Path:          /
Document Length:        50825 bytes

Concurrency Level:      250
Time taken for tests:   13.606 seconds
Complete requests:      700
Failed requests:        0
Write errors:           0
Total transferred:      36342698 bytes
HTML transferred:       35862831 bytes
Requests per second:    51.45 [#/sec] (mean)
Time per request:       4859.177 [ms] (mean)
Time per request:       19.437 [ms] (mean, across all concurrent requests)
Transfer rate:          2608.53 [Kbytes/sec] received

LAMP环境下，WordPress可以设置简洁优雅的永久链接，但是对于Lighttpd和Windows下的IIS，问题就来了：永久链接里要加上“/index.php”才能避免“404 Not Found”。这里简单谈一下Lighttpd的设置。

首先启用mod_rewrite，编辑/etc/lighttpd/lighttpd.conf，在“server.modules”中去掉“mod_rewrite”的注释。

然后加入以下设置：

$HTTP["host"] =~ “localhost$” {
var.wpdir = “/blog/”
url.rewrite-once = (
“^” + wpdir + “(wp-.+).*/?” => “$0″,
“^” + wpdir + “(sitemap.xml)” => “$0″,
“^” + wpdir + “(xmlrpc.php)” => “$0″,
“^” + wpdir + “keyword/([A-Za-z_0-9-])/?$” => wpdir + “index.php?keyword=$1″,
“^” + wpdir + “(.+)/?$” => wpdir + “index.php/$1″
)
}

最后重启Lighttpd，就可以随意设置喜欢的永久链接了。

WeTwoo on Air

With several months delayed, we are launched a first beta version of WeTwoo.com (We-Two-o) which for the moment has few features available which are day to day improving …  May be you are asking, What hell is wetwoo?. WeTwoo is a facebook application which attempt to be a place to know information about  you friends that have some type of relationship. The current functionality are:

• Know the status of relationship of all your friends. For this, you can see a pie-chat which show the distribution of relationship of your friends.
• Show a List, in one place, the friends that have some type of relationship that is showed on facebook.
• Make comments about the relation of your friends. Click on the name of you friend on the “Friends in Couple” page and the comment will appear on the wall of your friend.
• Make comments about the application and these will be show on the wall of your friends.
• Invite to friend to join wetwoo. In this way, the list of friends listed on “Friends in couple” will grow because only is show the couple that are accepted the application.
• Available on English and Spanish (Español)

Other features that we hope add:

• Save and help to remember the anniversary of couple to friends.
• Save the comments that are done for some couples.
• Suggest possibles couples as “Couple-friends”.

The application is developed using Ubuntu, Python, Django, Apache, Lighttpd, and PostgreSQL with the help of some libraries as MatPlotLib for graphs and PyFacebook which allow access to Facebook API.
I hope write more on the next months about technical issues to share this process with you.

Before end, I would like to invite to join to the site and make any comments or suggestion about it and help to spread with your contacts.

Best regards,

Álvaro

to fix this

#apt-get install libfam0


Loopia har idag lanserat något som de kallar för Autobahn. Det är en uppsättning servrar, med Nginx, dedikerade för statiskt material såsom vanliga HTML-sidor och bilder. Nginx är, enkelt förklarat, en väldigt nedstrippad webbserver (som Apache), i stil med lighttpd.

Vid en första anblick är detta väldigt positivt. Nu får man, som kund på Loopia, möjlighet att snabba upp åtkomsten till statiska filer för sin hemsida. Och det hanteras genom kontrollpanelen, på samma sätt som UNIX- och Windows-konton på Loopia.

Eller, för att citera Jimmie Eriksson – “enklare och mer kraftfullt än så blir det inte”.

Men så började jag fundera…

Hur enkelt är det egentligen att använda separata utrymmen (läs: olika domäner/subdomäner och FTP-konton) för att hantera materialet för en hemsida? Och hur fungerar det med ett publiceringssystem, såsom WordPress eller Joomla? Vad händer om en plattform inte fungerar? Fler plattformar tenderar att leda till fler problem.

Personligen skulle jag prioritera andra saker (läs: dieselaggregat) på Loopia och min uppfattning om “enkelt” är snarare en lösning där kunderna laddar upp sina filer till ett enda utrymme, för att därefter (innan webbservrarna) styra om trafiken till olika webbservrar helt “osynligt” för kunderna (beroende på vad det är för material).

Men jag förstår varför Loopia gör detta (heja Daniel!) och jag tycker det är skönt nördigt likaså. Jag hoppas att mina farhågor inte besannas och att ledningen på Loopia och Mamut verkligen förstår vad detta innebär. Om inte annat så kan det sporra ytterligare aktörer till liknande förbättringar (och kanske ännu enklare lösningar).

Actualmente o apache lidera o ranking de utilização dos Web Servers. No entanto existem outros Web Servers que não tendo o grau de popularidade do apache conseguem ter um desempenho muito interessante.

Por esse motivo procedi a um teste de desempenho a 4 Web Servers, sendo eles: Apache httpd, Nginx, LiteSpeed e Lighttpd.
Para este teste fora usadas duas máquinas. Uma fazendo de servidor e outra simulando os clientes, usando uma ferramenta de medição (JMeter).

Todos os servidores foram testados usando uma páginas estática (HTML) e uma página dinâmica (PHP) com uma repetição de 5 vezes em cada teste para impedir possíveis erros circunstanciais.

O primeiro teste mostra Throughput (respostas por segundo) dos diversos servidores quando sujeitos a 100 clientes simultâneamente com cada um a pedir 500 páginas e com a opção KeepAlive activa.

Fig.1 – Throughput (Requests/Sec)

Podemos também ver o estado da memória e do CPU do servidor durante a execução de cada teste.

Fig.2 – ServerLoad

Com estes testes, verificamos que existem outras boas alternativas ao Apache. Para uso de conteúdo estático o nginx teve um desempenho muito bom, enquanto que para conteúdo dinâmico, o apache continuar a ser a melhor opção. Mas temos que ter em mente que a maioria dos sites, tem na sua maioria conteúdo estático (imagens, css, html, js, etc.).
Relativamente a carga de CPU, o nginx também é o que menos CPU consome, o que se vem reforçar a liderança deste benchmarking.
O único ponto a apontar ao Nginx, foi o facto de a sua instalação não ser simples, e requereu alguma pesquisa principalmente para colocar o modulo php a funcionar.

*Aggiornata la pagina del progetto*

Sin da quando utilizzo rtorrent mi sono agitato alla ricerca di una gui che rispettasse le grandi potenzialità di questo client torrent e che fosse alla portata di cervello mano della mia compagna. Son passato per nTorrent, client in java del quale sono anche diventato traduttore, per poi stufarmi del fatto che il client dovesse risiedere in ogni postazione dalla quale interrogare la mia seedbox. Mi sono infine stabilizzato sui client php (wTorrent e rtGUI) con lighttpd come webserver.

Fino a questo pomeriggio avevo installato sulle mie due seedbox, un router Asus Wl500gp con harddisk usb e un nas Buffalo linkstation Live HS-DH500GL, rispettivamente Transmission 1.51 con la sua recente -manco tanto- webgui Clutch e appunto rTorrent 8.4 + rtgui e wtorrent in contemporanea.

Cosa è successo?

Transmission mi ha rotto le balle con i suoi errori non richiesti ed ho deciso di percorrere la via rtorrent + gui anche sull’asus.

Troverete la guida dopo il break.

Per colpa di marte che è entrato in urano senza il suo permesso, sull’asus nTorrent funziona alla grande ma rtgui e wtorrent non vogliono saperne di funzionare.

In mio soccorso arriva questo client:

http://code.google.com/p/rutorrent/

Direttamente dalla pagina di google code, vi elenco le features:

• Lightweight server side, so it can be installed on old and low-end servers and even on some SOHO routers
• Extensible – there are several plugins and everybody can create his own one
• Nice look

Nelle prossime righe vedremo come configurare rtorrent per funzionare con la totalità dei suoi client.

Premessa:

rtorrent usa xmlrpc per interfacciarsi con i suoi client, questo schema preso dal sito di wtorrent e applicabile anche agli altri client, rende l’idea.

Ci occorreranno lighttpd e i suoi moduli scgi e fcgi, php5, xmlrpc-c, rtorrent, le sue dipendenze e naturalmente ruTorrent o quale sia il suo nome. La guida che segue è, per quanto riguarda la parte sull’installazione dei pacchetti, orientata a un sistema optware, cioè un qualsiasi coso sul quale gira linux e che installa tutte le applicazioni nella path /opt utilizzando ipkg come gestore di pacchetti. I file di configurazione invece sono uguali per ogni distro/architettura/gestoredipacchetti, percui ho inserito le diverse path ai file. Per le modifiche ai file ho usato nano.

Vi rimando a questo sito per maggiori informazioni sugli optware.

http://www.nslu2-linux.org/wiki/Optware/Packages?from=Unslung.Packages

Iniziamo

Installare lighttpd e php5 e nano (se non fosse gia presente).

ipkg update

ipkg install lighttpd php php-fcgi php-curl php-xmlrpc libcurl nano

Ora mofidichiamo il file di configurazione di lighttpd con nano

per i sistemi optware

per tutti gli altri

decommentiamo i seguenti moduli:

e aggiungendo questo in fondo al file di configurazione.

scgi.server = (
“/RPC2″ => # RT_DIR
( “127.0.0.1″ =>
(
“host” => “127.0.0.1″, # Ip where rtorrent is listening
“port” => 5000, # Port specified in .rtorrent.rc
“check-local” => “disable”
)
)
)fastcgi.server = ( “.php” =>
( “localhost” =>
(
“socket” => “/tmp/php-fastcgi.socket”,
“bin-path” => “/opt/bin/php-fcgi”,
“bin-environment” => (
“PHP_FCGI_CHILDREN” => “4″,
“PHP_FCGI_MAX_REQUESTS” => “4000″
)
)
)
)

Configurazione rtorrent

a questo punto editiamo il file di configurazione di rtorrent

Nel caso di sistemi optware è

per tutti gli altri

o

inseriamo quanto segue cancellando l’intero contenuto del file precedente e avendo cura di sostituire le path delle directory con la vostra.

scgi_port = 127.0.0.1:5000
session = /tmp/harddisk/torrents # nel mio caso è questa, voi sostituite con la vostra.
port_range = 10000-15000
directory = /tmp/harddisk/downloads # nel mio caso è questa, voi sostituite con la vostra.
check_hash = no
encryption = prefer_plaintext
max_uploads = 5
download_rate = 0
upload_rate = 20
dht = auto
dht_port = 6881
safe_sync = yes
# controlla se in una directory ci sono nuovi torrents,
# e li avvia automaticamente.
schedule = watch_directory,5,5,load_start=/tmp/harddisk/torrents/*.torrent
# ferma i torrent quando lo spazio su disco scende a 300mb
schedule = low_diskspace,5,60,close_low_diskspace=300M
peer_exchange = yes
session_on_completion = yes

Editiamo lo script di avvio per inserire il nome utente di chi lancerà l’eseguibile rtorrent. (se lo script non è presente nel pacchetto della vostra disto, potete prenderlo da qui e salvarlo col nome sotto indicato)

Intorno alla riga 41 troveremo user=”p2p”, modificatelo in base al vostro username. Qui ad esempio ho inserito “admin”.

Riavviamo i demoni lighttpd e rtorrent

usiamo “ps” per contollare la corretta esecuzione di rtorrent

Questo dovrebbe essere un output corretto.

Se viene visualizzata una cosa simile in particolare la 3° riga vuol dire che rtorrent è avviato corretamente e usa il file di configurazione precedentemente modificato. Procediamo quindi testando il dialogo tra rtorrent e il gateway.

Apriamo il browser su

http://ipdellamacchina/RPC2

oppure

http://ipdellamacchina:porta/RPC2

Nel caso in cui lighttpd usi porta diversa da 80, inserirla nell’indirizzo. esempio http://192.168.1.1:8081/RPC2

Se visualizziamo una pagina completamente bianca vuol dire che funziona (rtorrent passa i dati via xmlrc al gateway scgi). Se visualizziamo “500 internal server error” è facile che rtorrent non sia avviato o che ci siano degli errori nel file di configurazione di lighttpd (in particolare nel modulo scgi).

Ora procediamo con la WebUI,

scarichiamo la webui e scompattiamola nella document root di lighttpd che dovrebbe essere /opt/share/www sui sistemi optware e /etc/www per tutti gli altri.

aprire il browser e puntare a

http://ipdellamacchina/rtorrent

http://ipdellamacchina:numeroporta/rtorrent se lighttpd usa una porta diversa da 80

verrà visualizzato brevemente “Apertura…” e subito dopo, se tutto va bene, l’interfaccia. In caso di mancata connessione con rtorrent, verrà segnalata da un messaggio.

Se visualizziamo “500 internal server error”, “404 Error”, “Forbidden” o simili vuol dire che lighttpd è mal configurato. In questo caso controlliamo la parte finale dei log di lighttpd e rtorrent e interroghiamo mastro google.

Sistemi optware

Tutti gli altri, uguale senza “/opt”.