For those who know me in a professional fashion, you know I’m a big fan of Application Delivery. For those who don’t know me in that arena, Application Delivery is essentially the process by which content/applications get from a web server to your client machine, be it a Desktop, Laptop, or phone. Since most web-sites require constant availability, many companies utilize load balancing hardware that ensures requests are only sent to servers that can properly serve the content. This requires that load balancers are able to monitor which servers are up, active, and able to send that content. The reason most load balancers are now considered Application Delivery Controllers (ADCs) is because they now do much more than just load balancing. They can offload SSL (HTTPS) functionality from servers, rewrite requests, and even assist in compressing and caching data to minimize the load on servers while speeding up delivery time.

There are many methods by which ADCs monitor server availability. Some simply ping servers while others post actual data to ensure the resource responds correctly. Due to the company for which I work’s desire to add more intensive applications to our sites, simply load balancing requests in a round-robin fashion is no longer good enough. We now want our ADC to poll a server for information such as CPU Load, Requests Per Second, Virtual Memory Usage, and TCP Errors. By obtaining this information, the ADC will be able to send sessions to whichever system is best capable of serving them. Essentially, we want the ADC to have as much information as possible  when making its decisions.

Unfortunately, I’ve run into a bit of a complication. The ADC accesses a script on the web server which then gathers the necessary data via Windows Management Instrumentation (WMI). While granting access to the script is easy, it’s been difficult to find the necessary access to allow the script to grab the WMI information from the server. Fortunately, we’ve got a good security staff who will hopefully be able to help. I know granting the application Administrator access allows it to work, but that might have other implications. Once everything is figured out, we’ll be using “Dynamic Load Balancing.” Dynamic Load Balancing,  as opposed to static load balancing, continually gathers information from its resources and changes its load balancing algorithm accordingly. This is typically the best load balancing method because it allows for sending requests to the most capable servers.

For the upcoming release of CloverETL 2.9, we are working on improvements in CloverETL Server which will allow run transformations in parallel on multiple cluster nodes.

CloverETL Server already supports clustering, so more instances may cooperate to each other. Current stable version already implements common cluster features: fail-over/high-availability and scalability of lots of requests which are load-balanced on available cluster nodes. These features are actually implemented since version 1.3.

The basic concept of new parallelism
Transformation may be automatically executed in parallel on more cluster nodes according to configuration and each of these “worker” transformations processes just its part of data. Because there is one “master” transformation, which manages the other transformations and which gathers tracking data from “worker” transformations, the parallelism is transparent for CloverETL Server client. Client by default “sees” just one (master) execution and aggregated tracking data. However there are still logs and tracking data for each of “worker” transformations, so it’s still possible to inspect details of this parallel execution. “Worker” transformations outputs are gathered to the “master”, thus client has one single transformation output which may be processed further.

So how to get parts of input data?
Basically, transformation can process data which is already partitioned, which is the best case and there is no overhead with partitioning of data, or CloverETL Server itself can partition input data from one single source and distribute data on the fly (during the transformation) to several cluster nodes using the network connection. Overhead of this operation depends on the speed of network communication and other conditions.

Design changes in the graph
We aim to keep the transformation graph almost the same as it would be for “standalone” execution. Thus there will be just a couple of extra components in the graph which is intended to run in parallel. These components will handle partitioning/departitioning of data in case it’s not already partitioned.

Scalability
The new parallelism in CloverETL Server is a giant leap for scalability of the transformations. Ever since the graph is designed for paraller run, the number of computers which run this transformation depends just on cluster configuration. Graph itself is still the same. Configuration of the parallelism includes:

• working CloverETL Server cluster, thus standalone server instances won’t be able to handle such execution
• “partitioned” sandbox(see below) with list of locations

New sandbox types
On server side, graphs and related files are organized in so-called sandboxes. Until version 2.8, there was just one type: “shared” sandbox. It means that it contains the same files and directory structure on all cluster nodes. Since version 2.9 there will be two more types:

• “local” sandbox – is (locally) accessible on just one cluster node. It’s intended for huge input/output data which is not intended to be shared/replicated among multiple cluster nodes.
• “partitioned” sandbox – each of its physical location contains just part of data. It’s intended as a storage for partitioned input/output data of transformations which are supposed to run in parallel. List of physical locations actually specifies nodes which will run “worker” transformations.

Master – worker responsibilities
Master observes all related workers and when some transformation phase is finished on all workers, it’s master’s responsibility to allow the workers to process next phase. When any of the workers fails from any reason, it’s master’s responsibility to abort all the other workers and select whole execution as failed. Master/worker – These terms have meaning only in the scope of one transformation. Since 2.9 there is no privileged node configured as “master” in the cluster, but it doesn’t mean that all the nodes are equal. There may be differences between nodes in accessibility to physical sources. Configuration of sandboxes should reflect it.

If you have a problem like:

• One of a set of load balanced servers is behaving unexpectedly and
• Targetting that machine directly by IP address does not fail the same way

Then you may have a missing host header.

Your host name may be being routed to a different application.

This kind of thing has prompted us to think about writing smoke tests for this type of multi-machine configuration.

Nonius has launched an upgrade to its load balance satcom capable appliance – WGStargate.

On board internet from multiple links

This new version of the platform is capable of balancing traffic across up to 8 interfaces.
The appliance is being used in ships and emergency trucks.

WGStargate Appliance

For more info on the appliance www.noniussoftware.com

Richard Demongeot, un ami administrateur systèmes Unix, a conçu une architecture matérielle et logicielle taillée pour le web avec une forte résistance aux pannes et une qualité de service augmentée. Elle est fortement basée sur la virtualisation et peut donc être mise en place à partir de 2 serveurs (physiques). Son fonctionnement est illustré ci-dessous. Sur la page de l’auteur vous trouverez une liste détaillée des applications qui composent chaque service.

Un mot sur la virtualisation

Les deux serveurs physiques disposent à eux seuls de l’ensemble des services nécessaires à une architecture web tout en offrant de la répartition de charge et de la redondance (de services). Cela est possible grâce à la virtualisation.

En effet sur le serveur n°1 comme sur le n°2 un système hôte (dom0) ajoute une couche qui abstrait l’accès aux ressources de la machine (CPU, RAM, DISK, etc.) et permet notamment de les découper en parts (inégales) que l’on redistribue ensuite à des systèmes virtualisés.

Un système virtualisé est un système d’exploitation complet qui est lancé tel une application du système hôte (pour simplifier fortement). Ainsi un système hôte peut lancer plusieurs systèmes d’exploitation sur une même machine physique et cela permet d’isoler les différents services pour plus de sécurité et une meilleure allocation des ressources de la machine grâce au système de parts virtuelles.

L’avantage le plus important de la virtualisation est la souplesse qu’elle apporte et ici cela se traduit par une évolutivité à toutes épreuves (ou presque).

Haute disponibilité : résistance aux pannes

Chaque service est redondé, c’est à dire présent en double, un sur chaque serveur physique. Si l’un des deux vient à bugger, le second prend la relève immédiatement (un temps infime est nécessaire pour basculer de l’un à l’autre) et assure ainsi la continuité du service. Cela est valable quelque soit le service : web, filer, mail, etc.

Qualité de service : répartition de charge

En plus d’être dupliqués pour garantir une disponibilité constante, les services qui le nécessitent (web, sql, et mail) sont gérés par un répartiteur de charge. Cela permet de distribuer le service de manière efficace entre deux serveurs virtuels (chacun situé sur une machine physique différente) qui rendent le même service. L’objectif étant d’assurer le traitement d’un maximum de requêtes en un minimum de temps.

Service Web efficace : séparation du contenu dynamique – statique

En plus de la répartition de charge, le service web est divisé en deux pour une meilleure efficacité, l’un dynamique et l’autre statique (tous deux “load balanced”).

Le premier traite tout le contenu dit “dynamique” ce qui correspond typiquement à la génération de code HTML (via PHP) pour afficher les pages web.

Le second est en charge de délivrer tout le contenu dit “statique”, ce qui peut se traduire par “tout fichier qui n’est pas généré dynamiquement”. En général on retrouve dans l’aspect statique, des fichiers tels que des images, des mp3, des vidéos, etc.On pourrait également mettre ici les fichiers CSS, JS, XML.

Reprise sur panne simplifiée : politique de sauvegarde pertinente

En plus de la duplication des filers, ce qui limite déjà fortement les pannes, un service de backup est présent afin de réaliser une sauvegarde de chacun des systèmes virtuels complets de chaque service provenant de l’autre machine physique. Ainsi, si une des deux machines vient à ne plus fonctionner, on peut reconstruire une nouvelle machine très simplement à partir du service de backup de la machine restante qui contient les données de celle qui est down.

A venir : aspect réseau

Il y a déjà énormément à dire sur cette architecture en l’état, c’est pourquoi Richard (et moi-même) ajouterons dans un deuxième temps tous les aspect réseaux.

A noter : ceci n’est (pour l’instant) que de la théorie

Cette architecture web n’a pour l’instant pas encore été réalisée par nos soins, et cela ne reste donc que de la théorie. Cependant très prochainement l’auteur, Richard Demongeot, aura l’occasion de la réaliser à titre personnel. Bien entendu, il ne manquera pas de nous faire partager ses retours d’expérience sur ce projet très INtéressant.

La société SIVIT.fr, dans laquelle il travaille, réalise déjà ce type d’infrastructure et je vous conseille, pour ceux qui sont intéressés, d’aller voir leurs offres à très bon rapport qualité / prix.

Stay tuned

La rédaction de cet article a nécessité 2h14.

MindSource specializes in placing technology professionals with companies in the Silicon Valley Region and beyond. We have a need for a Network Administrator to work with our client in Mt. View, CA. This is a CONTRACT position, so let me know your interest ASAP!

Our client is seeking an energetic and experienced Network Engineer with Juniper router experience. If you have just done Cisco, you will not be a fit. Also, hands-on experience with Citrix netscalers and Netscreen VPN/firewalls is important.

You would join a heterogeneous team of 6 system and network engineers who support the production and development environments. Specifically, the team is tasked with taking project handoffs from engineering and implementing as well as providing ongoing support for said “services”. In addition to implementation, the team is tasked with carrying out smaller scale changes (setting up ACL’s, load balancing, VPN, caching, switching, routing, etc). Most client interaction is internal, though we do get involved with external clients for some configuration work as well as outage support. This team is on call 24/7 to provide tier 2/3 level support as necessary.

The ideal candidate would have a broad background in both network (primary) as well as system level troubleshooting. A generalist with solid troubleshooting skills is probably the best way to describe it.

Specific skills:

• System level network debugging (Solaris, Linux and a little bit of Windows)
• packet switching, packet routing
• in-depth knowledge of routers, switches, load balancing, VPN.
• familiar with JunOS, Citrix netscaler, Netscreen, Blue Coat, SBC’s, Cisco switches
• knowledge of shell scripting (regex)
• experience in HA multi-site networks
• data circuit troubleshooting (T1’s to OC3’s)
• familiarity with VOIP is helpful
• console/command line network config (we don’t use gui’s to configure network
  equipment)
• RCS (revision control system) and ACL*s
• perform and read/debug packet captures
• 5 plus years experience
• change management familiarity
• conceptual knowledge of 7-layer OSI model

If you are interested in this position, please forward your resume to kathy@mindsource.com TODAY!

A PHP Session Cookie (mostly defined as PHPSESSID) has a default life time until the browser is closed. This is OK for many web-applications, but in some cases this is not wanted and the lifetime can be changed – we will cover this later.

Another session-relevant issue is the PHP garbage collector. When a session is created, a flat-file is created on the server – e.g. in /tmp on linux servers. Since the session ID is a unique identifier, those session files will accumulate over time – the garbage collector will take care of these files and delete old files from time to time. PHP has a sort of built-in “load-balancing” feature for this garbage collector, so that old session files are not deleted on each and every session request, but with a certain probability. The default timeout for session files is 1440 seconds or 24 minutes. So a session file can be deleted after that timeout, but it may reside on the server longer, depending on the amount of sessions created – here comes the probability into the game.

//
//
So we have a session cookie with a lifetime until the browser is closed, but the garbage collector might delete the session file much earlier. In this case, and if there is a session request after the session file has been deleted, a new session is created and the old session information is lost. This can be very annoying if e.g. the users is writing some message in a web-based interface and this task takes longer than the session file on the server is available.

There are 3 PHP.ini variables, which deal with the garbage collector:

session.gc_probability in conjunction with session.gc_divisor is used to manage probability that the gc (garbage collection) routine is started. The probability is calculated by using gc_probability/gc_divisor, e.g. 1/100 means there is a 1% chance that the GC process starts on each request.

The most important variable is session.gc_maxlifetime:
As discussed earlier, this variable sets the timeout for session file deletion.
All three variables can be set in the php.ini configuration file, but more important, those can be also set during runtime due the PHP_INI_ALL permission.

//
//

$garbage_timeout = 3600; // 3600 seconds = 60 minutes = 1 hour
ini_set('session.gc_maxlifetime', $garbage_timeout);

So we just changed the garbage collection timeout to one hour. There is just another problem. Since PHP stores all session files into one directory (e.g. /tmp on Linux) and the garbage collector can’t tell the difference between the different gc_maxlifetime’s of the individual session files, the session files are deleted by the garbage collector with the lowest timeout.

The solution for this problem is another session file directory. This is best accomplished by creating another directory *within* the regular session file directory.
First we read the current save path for the session files and append our own directory to this path. Note: In order to make that platform independent, we need to check for the file-seperator first. Now we check if the directory already has been created, if not, create one. Then we set the new path for the session files.

strstr(strtoupper(substr($_SERVER["OS"], 0, 3)), "WIN") ?
	$sep = "\\" : $sep = "/";
$sessdir = ini_get('session.save_path').$sep."my_sessions";
if (!is_dir($sessdir)) { mkdir($sessdir, 0777); }
ini_set('session.save_path', $sessdir);

Now our session files are stored in the new directory and the regular garbage collector will not see them, so those files will survive at least that long as defined in gc_maxlifetime.

If you have enough disk space on the server you can set the session file timeout really long, in order to preserve most/all sessions until the browsers are closed. But in many cases the session should expire after a certain time, so we have to change the life-time of the session cookie itself.

$cookie_path = "/";
$cookie_timeout = 60 * 30; // in seconds
session_set_cookie_params($cookie_timeout, $cookie_path);

(the cookie path can be “/” or e.g. “/somedirectory” if the cookie should be valid for a certain directory only)

NOTE: All these changes MUST be made before the actual session is opened with session_start() or session_register().

Conclusion:
So we have a session cookie with a custom-defined lifetime, and we also redefined the garbage collector timeout. This ensures that the current session data (identified with the session id PHPSESSID) is available at least as long as the session cookie in the browser is valid.

Last but not least, we have the example source code session.php, which shows all steps together.

//
//
session.php

<?
// path for cookies
$cookie_path = "/";

// timeout value for the cookie
$cookie_timeout = 60 * 30; // in seconds

// timeout value for the garbage collector
//   we add 300 seconds, just in case the user's computer clock
//   was synchronized meanwhile; 600 secs (10 minutes) should be
//   enough - just to ensure there is session data until the
//   cookie expires
$garbage_timeout = $cookie_timeout + 600; // in seconds

// set the PHP session id (PHPSESSID) cookie to a custom value
session_set_cookie_params($cookie_timeout, $cookie_path);

// set the garbage collector - who will clean the session files -
//   to our custom timeout
ini_set('session.gc_maxlifetime', $garbage_timeout);

// we need a distinct directory for the session files,
//   otherwise another garbage collector with a lower gc_maxlifetime
//   will clean our files aswell - but in an own directory, we only
//   clean sessions with our "own" garbage collector (which has a
//   custom timeout/maxlifetime set each time one of our scripts is
//   executed)
strstr(strtoupper(substr($_SERVER["OS"], 0, 3)), "WIN") ?
	$sep = "\\" : $sep = "/";
$sessdir = ini_get('session.save_path').$sep."my_sessions";
if (!is_dir($sessdir)) { mkdir($sessdir, 0777); }
ini_set('session.save_path', $sessdir);

// now we're ready to start the session
session_start();

session_register('mytest');
print "mytest=".$_SESSION['mytest']."<br><br>";
$_SESSION['mytest'] = "captain";
?>

source: http://www.captain.at/howto-php-sessions.php

Di dalam dunia distributed system ada beberapa istilah yang mirip-mirip tapi sebenarnya digunakan dalam konteks yang berbeda.

Load sharing bertujuan untuk membagi-bagikan pekerjaan dari satu atau beberapa sumber pekerjaan kepada satu atau beberapa pekerja sehingga tidak ada satupun pekerja yang tidak mendapat pekerjaan dalam suatu waktu.

Load balancing bertujuan untuk menyamakan workload pada seluruh mesin pekerja.

Task scheduling adalah kegiatan untuk membagikan pekerjaan ke pada banyak mesin pekerja secara optimal dengan memperhitungkan beban kerja, kecepatan kerja, dan lain-lain.

Parallel computing bertujuan untuk melakukan sebuah pekerjaa dengan membagi-bagi pekerjaan besar tersebut menjadi pekerjaan-pekerjaan yang lebih kecil dan mengerjakannya secara paralel.

Apa mereka itu dan bedanya di mana akan gw coba jabarkan satu per satu di postingan selanjutnya. ^_^ (ntar kalo sempet)

Ever have one of those bugs that customers complain about, but you just cannot reproduce it? Here is a good one…

Customers were complaining about being logged out when clicking a download link.

This particular setup is a Cisco CSS 11501 series load balancer with 2 Dell Poweredge web servers sitting behind it.  Each webserver is running apache, as well as an application server (python) which handles authentication and processing for THAT server.

For weeks, I could not reproduce this bug.  So tonight when I finally got bit by it (at home), I was clueless for a while.  The code is so simple.  A simple key lookup in a simple dictionary, yet it just was not making sense.

Here is the story:

A while ago, we were having problems with Internet Explorer downloading content over SSL.  This turns out to be a common problem with IE, so to fix it, I caused the downloads to not use SSL, which is more efficient anyway.

About 50% of the time, users (like me tonight) were tossed to the other server, which knew nothing about the user login. That is why it was complaining about the “WB4_App::$DSEG and/or WB4_App::$AuthToken must be set in order to contact the     applications server.” error message, which is not one that should normally be shown.

To make matters worse, our IP address at work was apparently always using the same server, so I could not reproduce the problem.  I’m lucky that it happened to me at home, or I would still be banging my head against the desk…

Considering that Glassfish seems to be the very best next Application Server for Java Enterprise Applications, clustering and load balancing started to be a hot topic, especially considering the ease of configuring the Glassfish Cluster, both horizontal and vertical, clustering that is.  For those still asking themselves what is the difference between the two forms of clustering:

• Horizontal Clustering:

Grouping multiple physical servers into a cluster. A horizontal cluster consists of a cluster of servers that are exposed to browser clients as a single virtual server. Horizontal clusters help to increase application scalability, performance, and robustness

• Vertical Clustering:

A vertical cluster is like a horizontal cluster, except that rather than use several server machines linked together, vertical clusters use a single machine with multiple CPUs. Vertical clusters help to increase scalability on multiprocessor computers since they distribute work to several processes. Each process runs on a different CPU.

Now, although Glassfish has a pretty good integrated load balancing in its DAS (Domain Administration Server), one may work with the “older”, and mature Apache Web Server, for controlling and distributing the requests further on to the load balancers. This is what this article is about:Configuring the Glassfish Cluster, getting it ready for communicating with Apache, configuring Apache to talk to the Glassfish Cluster. Allright, that being said, let’s move on.

Prerequisites

This article has been written after using the following packages:

• Apache Web Server : Version 2.2.10
• Sun Glassfish Application Server: Version 2.1

Besides the “heavyweight” you will need some extra packages, which you can download from the locations specified together with the packages:

• Mod_JK: Version 1.2.2.8 — Get it HERE ( this is the version for x86_64 servers)
• tomcat-ajp.jar: You can get it from a Tomcat 5.5 distribution, after unpacking it.  I have used Tomcat 5.5.27 from HERE
• commons-logging.jar: Version 1.1.1 — Get it HERE
• commons-modeler.jar: Version 2.0.1 — Get it HERE

1 – Configure Apache for Mod_JK

The Apache Server’s loads the module files from a configuration file. In this case, working with a distribution of SuSE 10, the configuration file for loading modules resides under the following path:

/etc/apache2/sysconfig.d/loadmodules.conf

Open this file to get a view on where the modules are located. In this case, the file above tells us that all modules reside under:

/usr/lib64/apache2-prefork/

Ok, so we now know where the modules are. It is time to unpack our mod_jk package, that you just downloaded in the prerequisites section, and copy the contained mod_jk.so file in the modules folder. The filename of the module is “mod_jk.so”. Copy (unpack it) in the folder above.

Now you are ready to configure Apache for loading mod_jk. In order to do that, add the following line to the default module load configuration file (/etc/apache2/sysconfig.d/loadmodules.conf)

LoadModule jk_module /usr/lib64/apache2/mod_jk.so

Save the file and close it. In order to test your webserver, restart it by running:

/etc/init.d/apache2 restart

2 – Create your Glassfish Cluster

This article will describe the process of creating two glassfish instances, in a horizontal clustering mode ( each of the instance will be on a physical independent server) In order to understand some of the words we are going to user as we go along, a little introduction would be needed:

One server can host multiple cluster instances (vertical clustering) Combining two such servers, you will accomplish both a horizontal, and a vertical clustering. We will stick to the horizontal one, and limit ourselves to two instances, each of them located on one independent server.

In order to manage the instances, the Glassfish Administration Server (DAS), needs some “remote agents” on the servers where the instances resided. These agents are responsible for:

• starting / stopping instances
• deploying applications
• synchronizing repositories
• etc.
  

So, basically we have the DAS (Domain Administration Server) communicating with agents, which manage cluster instances

As you already know (and if you don’t, start all over with Glassfish Documentation), the basic concept that is at the root of the Glassfish Application Server, is the so called “DOMAIN”. You can have multiple domains configured, but only one domain can be running at the same time. We will:

• create a new domain
• create a cluster for the domain
• create agents for the cluster
• create instances for the cluster

In this article, all the steps will be performed from command line, so make sure you have enough privileges before starting.Ok, time to hit the road

1. Create a new domain:
   
   asadmin create-domain  –user admin –adminport 4848 domain1
   
2. Start domain:

   asadmin start-domain domain1

3. Create a new cluster for the domain “domain1”

   asadmin create-cluster -u admin cluster1

   Because no configuration was specified when creating the cluster, a new default configuration file named cluster1-config will be created. The heartbeat will be configured by default

4. Create a node agent on machine “slave1″Login on the “slave1″ machine as user “glassfish” ( or a user with privileges) and run the following command:
   
   asadmin create-node-agent -host master –port 4848 slave1_agent

   “master” is the name of the DAS (Domain Administration Server) and slave1_agent is the name of the agent on the “slave1″ machine, that will communicate with the DAS. 4848 is the default DAS administration port. If being asked for a password, accept the default one (Just push ENTER). The default Master Password for GlassFish clustering is “changeit” (again, you do not have to tip it in, just press ENTER when prompted)
   Start the agent you just created by running the following command:

   asadmin start-node-agent slave1_agent

5. Create a node agent on machine “slave2″Login on the “slave2″ machine as “glassfish” user and run the following command
   
   asadmin create-node-agent –host master –port 4848 slave2_agent

   Start the agent you just created:
   
   asadmin start-node-agent slave2_agent

6. Create instances on the DAS for the agents “slave1_agent” and “slave2_agent”Log into the DAS server as user “glassfish”
   Create the instance on “slave1_agent”
   
   asadmin create-instance –user admin –host master –port 4848 –cluster cluster1 –nodeagent slave1_agent –systemproperties HTTP_LISTENER_PORT=58294:HTTP_SSL_LISTENER_PORT=58297:IIOP_LISTENER_PORT=58300:IIOP_SSL_LISTENER_PORT=58303:
   IIOP_SSL_MUTUALAUTH_PORT=58306:JMX_SYSTEM_CONNECTOR_PORT=58309 slave1_instance

   Create the instance on “slave2_agent”

   asadmin create-instance –user admin –host master –port 4848 –cluster cluster1 –nodeagent slave2_agent –systemproperties HTTP_LISTENER_PORT=58294:HTTP_SSL_LISTENER_PORT=58297:IIOP_LISTENER_PORT=58300:IIOP_SSL_LISTENER_PORT=58303:
   IIOP_SSL_MUTUALAUTH_PORT=58306:JMX_SYSTEM_CONNECTOR_PORT=58309 slave2_instance

Ok. You have your clustered set-up. Now you need to configure your glassfish instances to communicate with Apache Mod_JK connector.

3 – Configure Glassfish for Mod_JK

Copy the rest of the connectors packages (tomcat-ajp.jar, commons-logging.jar, commons.modeler.jar) to the lib folder of your local Glassfish Installation (opt/glassfish/lib)

When started, the remote glassfish instances connect through the agents to their DAS (Domain Administration Server) and synchronize their configuration files. Therefore, we need to configure the options for Mod_JK on the DAS Server

Because we need each instance to communicate separately with the Apache mod_jk connector, we need to define the properties for each of the instances:

1. Login to the DAS (the “master” in the configuration above)
2. Stop your domain: asadmin stop-domain domain1
3. In order to be able to add multiple instance configurations, we need to make the configuration dynamic. We will first define the generic Routing property, and the listening port: asadmin create-jvm-options –target cluster1 “-DjvmRoute=\${AJP_INSTANCE_NAME}

   
   asadmin create-jvm-options –target cluster1 “-Dcom.sun.enterprise.web.connector.enableJK=\${AJP_PORT}

   Yes, we basically define a port for each instance that wants to communicate via mod_jk.

4. Configure the system properties for the instances defined in the previous step: asadmin create-system-properties –target slave1_instance AJP_INSTANCE_NAME=slaveinstance1
   asadmin create-system-properties –target slave1_instance AJP_PORT=8010 asadmin create-system-properties –target slave2_instance AJP_INSTANCE_NAME=slaveinstance2
   asadmin create-system-properties –target slave2_instance AJP_PORT=8015

   As you can see, we defined port 8010 to be used by instance 1, and port 8015 by instance 2. We will need these values, when we define our so called “workers list” (who is working in the cluster, how, and where)

   The commands above will append the DAS domain.xml configuration file (the file is to be found in the root folder of your domain. i.e. /opt/glassfish/domain1/domain.xml). The instances will copy this file when started by the cluster.

4 – Start Glassfish, the Agents and the cluster

1. Start domainLogin to DAS and run the following command: asadmin start-domain domain1
   
2. Start agent1Login on the “slave1″ machine as “glassfish” user and run the following commandasadmin start-node-agent slave1_agent
3. Start agent2Login on the “slave2″ machine as “glassfish” user and run the following commandasadmin start-node-agent slave2_agent
   
4. Start the cluster (it will start the instances too)Log into the DAS Glassfish Serverasadmin start-cluster cluster1

5 – Configure Apache Mod_JK to work with Glassfish instances

Log into the Apache Server

First you need to configure your “workers”. Create a file called “workers.properties” in the installation folder of apache:

Be sure to use the name of the AJP_INSTANCE_NAME defined in 3.4 (Configure Glassfish instances for Mod-JK) when defining the worker.name

worker.list=loadbalancer # Array of load balancers

#Set properties for worker1

worker.slaveinstance1.type=ajp13 # Protocol used. We use ajp13 cause it can handle ssl, as oposed to ajp12

worker.slaveinstance1.host=slave1 # Host name of the machine where the instance resides

worker.slaveinstance1.port=8010 # Port as we configured at point 4.4 (Configure Glassfish instances for Mod-JK)

worker.slaveinstance1.lbfactor=50

worker.slaveinstance1.cachesize=10

worker.slaveinstance1.cache_timeout=600

worker.slaveinstance1.socket_keepalive=1

worker.slaveinstance1.socket_timeout=300

#Set properties for worker2

worker.slaveinstance2.type=ajp13

worker.slaveinstance2.host=slave2

worker.slaveinstance2.port=8015

worker.slaveinstance2.lbfactor=50

worker.slaveinstance2.cachesize=10

worker.slaveinstance2.cache_timeout=600

worker.slaveinstance2.socket_keepalive=1

worker.slaveinstance2.socket_timeout=300

worker.loadbalancer.type=lb # The balancing will be done using round-robin algorithm

worker.loadbalancer.balanced_workers=slaveinstance1,slaveinstance2 # Which of the workers will be part of the load balancer

6 – Instruct Apache where to find the workers file

You can update the httpd.conf default configuration file of Apache, but i would reccommend creating a new configuration file, and including it in the defaul configuration file (httpd.conf) using the INCLUDE directive.

Create a new file called:

httpd.mod_jk.conf in the root apache installation

JkWorkersFile /etc/apache2/workers.properties
# Where to put jk logs
JkLogFile /var/log/apache2/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel debug
# Select the log format
JkLogStampFormat “[%a %b %d %H:%M:%S %Y] “
# JkOptions indicate to send SSL KEY SIZE,
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
# JkRequestLogFormat set the request format
JkRequestLogFormat “%w %V %T”
# Send all jsp requests to GlassFish
JkMount /*.jsp loadbalancer
# Now add all your applications that want the request to be sent to glassfish
JkMount /clusterjsp/* loadbalancer # We used the clusterjsp sample application in the glassfish installation for testing

Include the mod_jk configuration file in the default configuration of apache (httpd.conf)

Include /etc/apache2/httpd.conf.mod_jk

Restart Apache

/etc/init.d/apache2 restart

7 – Deploy sample application and test

Glassfish comes with a default clustered application called clusterjsp. Deploy the clusterjsp application on the cluster you created in step 3 (Create a Glassfish Cluster)

Open http://master:4848

Login using admin:adminadmin

Go to Enterprise Applications

Select Deploy

Select “Local packaged file or directory”

Browse to . /opt/glassfish/samples/quickstart/clusterjsp/clusterjsp.ear

Check the “Availability” checkbox

Add cluster 1 from the available targets

Deploy

If everything worked ok, you should be testing it on your apache server:

http://apacheserver/clusterjsp/ – The first instance should serve the application

Once again

http://apacheserver/clusterjsp/ The second instance should serve the application

That’s it! You have your Apache configured to load balance glassfish instances

Looking for a Web Systems Administrator, someone who wants to push their career and join an expanding thriving digital agency. Excellent beneifts.

Salary upto £35k

Job description:
Sys Admin
We are looking for a talented techie to help us with managing, releasing and maintaining our clients sites. You are keen to be involved in the fast pace of agency life and like to try and bring a little order to the creative chaos.  You can keep a complex HA server environment purring whether it be windows or linux based.

You understand the technologies of web development and like to dabble at scripting to make something work or be just that bit easier.

Chasing down the cause of a fault is a game and once you are on the trail you see it through to the end.

Key responsibilities:
• Maintaining our virtualised server environments hosted at a managed hosting provider.  (30%)
• Applying OS updates
• Hardening of environments
• Managing hardware maintenance and failure recovery in cooperation with hosting provider
• Scheduling maintenance
• Release management (30%)
• Working with the development team to schedule and plan releases and patches
• Maintaining deployment archives
• Configuring backups
• Tools and workflow (10%)
• Developing tools and systems to automate and improve development and deployment processes.
• Integrating systems such as Continuous integration (CruiseControl), Source control (SVN), issue tracking (JIRA)
• First line support and maintenance (30%)
• Investigating issues on production sites through fault finding
• Informing project managers and developer team of potential causes of issues
• Rectifying the issue, if applicable

Environments and systems:
• LAMP – Redhat, Apache, Mysql, PHP5,  cPanel
• Windows IIS 6 on server 2003, MSSQL 2005.
• Development, Staging and Production
• Load balanced and high availability windows hosting
• Akamai Content Delivery Network integration
• Client hosting environments (Windows and Unix)

send your CV to: neil@majorplayers.co.uk

I discovered lighttpd several months ago, and just recently had chance to implement it as http proxy balancer for a group of 4 DL3205Gp boxes.
I started with the 1.4.3, but  finished with 1.5.0-r1992 …
All the boxes has lighty and Apache running, I have one box as http proxy  balancing the rest, and in case its dead, one of the resting three will become the loadbalancer.  The balancing is done via lighttpd 1.5 and its mod_proxy_core and mod_proxy_backend_http modules.
Have to say, I use one of the boxes as NFS server,  and the resting three  mount the webserver directory in a view to serve the same contect …
The tricky part was to bypass the nagios url from balancing, for abious reasons.
The trick: in lighttpf.conf
$HTTP["url"] !~ “\.(js|css|gif|jpg|png|ico|txt|swf|html|htm)$|^\/(nagios|nagadmin)” {
#$HTTP["url"] =~ “\.php$” {
proxy-core.protocol = “http”
proxy-core.balancer = “sqf”
proxy-core.allow-x-sendfile = “enable”
proxy-core.backends = (
#  lighty pool
# dont use the box “127.0.0.1:80″
“10.99.99.11:80″
,”192.168.11.12:80″
,”192.168.11.13:80″
#  APACHE pool
#  dont use apache at the hosting box …,”127.0.0.1:8800″
,”10.99.99.11:8800″
,”192.168.11.12:8800″
,”192.168.11.13:8800″

)
proxy-core.max-pool-size = 3000
}

in the light we have include of nagios.conf:
alias.url =     (
“/nagios/cgi-bin” => “/usr/local/nagios/sbin”,
“/nagios” => “/usr/local/nagios/share”
)

$HTTP["url"] =~ “^/nagios/cgi-bin” {
cgi.assign = ( “” => “” )
}

$HTTP["url"] =~ “nagios” {
auth.backend = “htpasswd”
auth.backend.htpasswd.userfile = “/etc/nagios/.htpasswd”
auth.require = ( “” => (
“method” => “basic”,
“realm” => “Monitoring area”,
“require” => “valid-user”
)
)
setenv.add-environment = ( “REMOTE_USER” => “user” )
$HTTP["url"] =~ “\.php$” {
proxy-core.balancer = “round-robin”
proxy-core.protocol = “fastcgi”
proxy-core.allow-x-sendfile = “enable”
proxy-core.backends = ( “unix:/tmp/php-fastcgi.sock”)
proxy-core.max-pool-size = 16
}
}

In this way the nagios url is handled by the local box only.
same settings will be for interna box, but using the proxy-core http protocol  instead …

At the end, I would like to thank Carlos Silva Santin for giving me  the chance to wotk with him in the past 30 days ..
Carla  will grow up so fast Carlos … :p

เช้าวันนี้มีข่าวที่น่าตกใจคือ การเสียชีวิตของ Michael Jackson ราชาเพลงป๊อป ตอนแรกเห็นใน Twitter ส่วนของ Trending Topics ก่อนเลย

เนื่องจาก MJ ถือว่าเป็นสุดยอดคนดังคนหนึ่งของโลก (แม้ว่าในระยะหลังๆ จะตกไปบ้าง) ทำให้กระแส “ความต้องการข้อมูลเกี่ยวกับ MJ” หรือ “ความต้องการพูดคุยเรื่อง MJ” พุ่งสูงมากในวันนี้

TechCrunch พาดหัวว่า The Web Collapses Under The Weight Of Michael Jackson’s Death ส่วนตัวเลขดูได้จาก LATimes ซึ่งสรุปสั้นๆ ว่า คน tweet เพิ่มขึ้นสองเท่าตัว และคนเล่น Facebook เพิ่มขึ้น 3 เท่าตัว

ผลที่ตามมาคือ Twitter ต้องปิดฟีเจอร์บางส่วน (Trending Topics และ Search) ออกไปเพื่อให้ระบบยังทำงานได้

คำถามก็คือ เมื่อคนผูกพันกับเครือข่ายมากขึ้นเรื่อยๆ และเกิดเหตุการณ์เด่นในลักษณะนี้ขึ้น ด้วยพลังอำนาจของการสื่อสารผ่านเน็ตและจำนวนคนที่มากขึ้นเรื่อยๆ จะเกิดอาการ Big Head (อีกด้านของ Long Tail) ซึ่ง Head จะใหญ่มากๆๆๆๆ จนเครือข่ายรองรับไหวหรือไม่?

ปัจจุบันผู้ใช้เน็ตมีประมาณ 1 พันล้านคน อะไรคือทางออกในการรองรับ “ความต้องการที่จะบริโภคข้อมูล” ถ้าหากว่ามีเหตุการณ์ที่ชาวเน็ตทั้ง 1 พันล้านคนจะต้องการข้อมูลชิ้นเดียวกันพร้อมๆ กันในอนาคต (เช่น ลอบสังหาร ปธน. สหรัฐ?)

แล้วถ้าชาวเน็ตเพิ่มเป็น 2 พันล้าน 4 พันล้านล่ะ วิธีเดิมจะยังใช้งานได้หรือไม่ หรือว่าเราต้องใช้วิธีอื่น

ทางออกควรเป็นการปรับปรุงด้านโครงสร้างพื้นฐานทางเทคนิค? เช่น

• ปรับเซิร์ฟเวอร์ให้มี scalability
• ปรับซอฟต์แวร์
• เปลี่ยนสถาปัตยกรรมเป็น decentralized
• ปรับขนาดของท่อเน็ตเวิร์ค

หรือ เราจะมีวิธีทางอื่นๆ ที่แตกต่างไปจากนั้น เช่น วิธีการทางจิตวิทยาหรือสังคมศึกษา วิจัยดูว่าเมื่อมนุษย์ต้องการข้อมูล (แล้วดู empower by internet) จะมีพฤติกรรมที่ต่างออกไปจากปกติอย่างไร?

A big feature of the 10.x upgrade is the Application Template “wizard.”   BIGIP has their deployment guides, which walk you through creating all of the configuration objects you need to get set up and running, but they aren’t always the clearest map through the woods.  As discussed on the DevCentral site, the wizard/survey/form takes a lot of the page-turning out of the mix, and presents everything in a much clearer manner (as in all on one page, fill-in-the-blanks-and-click-go).

Here’s my take on using the Microsoft Sharepoint 2007 Application Template, what you need, and what it makes for you.

Prerequisites:

• 172.16.1.5 – Externally visible IP address
• 172.16.1.10, .11, .12 – Web front-end HTTP servers
• SSL Certificate and Key as discussed previously (SOL 6549 on tech.f5.com)

Steps to create a new configuration to front a Microsoft SharePoint 2007 system:

1. Logon to the Management Web Console, expand Templates and Wizards, select Templates, and then Microsoft SharePoint
   
2. Provide a familiar prefix for the template group, in this case SharepointDemo, and the IP address that will serve the URL, in this case 172.16.1.5
   
3. Scroll down to the SSL Offload Questions section to elect to offload SSL form the SharePoint Servers, and to select the appropriate certificate and key
   
4. Scroll down to the Load Balancing Questions to select your balancing method, and to add the WFE server IP addresses (Note: if you are offloading SSL on the BIGIP, use port 80 to pass the traffic to the back-end servers), and to provide the health check response string
   
5. Scroll down to choose WAN or LAN traffic, and click Finished
   
6. You’ll now see a list of the objects the template has created and configured for this Virtual Server instance
   

Merlin, or Module for Effortless Redundancy and Load balancing In Nagios, has reached beta stage! The Merlin project was initially started to create an easy way to set up distributed Nagios installations, allowing Nagios processes to exchange information directly as an alternative to the standard nagios way using NSCA.

The code has been available in our code library for some time and we have received a lot of comments and input as well as patches, speeding the project along. We are very grateful to everyone and anyone helping out and testing the module.

Download the Merlin beta 0.5 here or take a look at the code here.

BTW

We currently dont have any logo for the projet, but it is something we would like to have when the project reaches release. Do you like design? Whant to contribute to the project? Feel free to send your suggestion to klas.bernehjalt_at_op5.com