As part of my Intelligent Systems course in my Masters program, I am working on a project that joins two intelligent systems together. I won’t post many of the finer details of the project (check out my conference paper once I get around to writing it), but it is a rule-based and genetic algorithm hybrid.

The idea here is not create a “super-agent” that can win at games, but more to simulate personality in NPC characters. I’ve yet to generate a decent set of experiment results, but things are looking good so far. I have my test software written and I’m about to add my genetic algorithm library to the project to see what happens.

after months of talking about it, i finally finished a genetic algorithm to optimise reflector panels. this is general digital tool using simple ray-tracing methods. from initial tests using ODEON, it seems to work!

A random route can be shown like this,

After iteration,the result can be like this.

Method of generation routines is got from here:

http://www.psychicorigami.com/2007/04/17/tackling-the-travelling-salesman-problem-part-one/

'''
Created on 2009-10-29
@author: Administrator
'''
import sys, random
from math import sqrt
from pickle import *

PIL_SUPPORT = None
try:
   from PIL import Image, ImageDraw, ImageFont
   PIL_SUPPORT = True
except:
   PIL_SUPPORT = False 

def cartesian_matrix(coords):
   """ A distance matrix """
   matrix = {}
   for i, (x1, y1) in enumerate(coords):
      for j, (x2, y2) in enumerate(coords):
         dx, dy = x1 - x2, y1 - y2
         dist = sqrt(dx * dx + dy * dy)
         matrix[i, j] = dist
   return matrix 

def tour_length(matrix, tour):
   """ Returns the total length of the tour """
   total = 0
   num_cities = len(tour)
   for i in range(num_cities):
      j = (i + 1) % num_cities
      city_i = tour[i]
      city_j = tour[j]
      total += matrix[city_i, city_j]
   return total

def write_tour_to_img(coords, tour, img_file):
   """ The function to plot the graph """
   padding = 20
   coords = [(x + padding, y + padding) for (x, y) in coords]
   maxx, maxy = 0, 0
   for x, y in coords:
      maxx = max(x, maxx)
      maxy = max(y, maxy)
   maxx += padding
   maxy += padding
   img = Image.new("RGB", (int(maxx), int(maxy)),\
         color=(255, 255, 255))
   font = ImageFont.load_default()
   d = ImageDraw.Draw(img);
   num_cities = len(tour)
   for i in range(num_cities):
      j = (i + 1) % num_cities
      city_i = tour[i]
      city_j = tour[j]
      x1, y1 = coords[city_i]
      x2, y2 = coords[city_j]
      d.line((int(x1), int(y1), int(x2), int(y2)), fill=(0, 0, 0))
      d.text((int(x1) + 7, int(y1) - 5), str(i), \
        font=font, fill=(32, 32, 32)) 

   for x, y in coords:
      x, y = int(x), int(y)
      d.ellipse((x - 5, y - 5, x + 5, y + 5), outline=(0, 0, 0),\
                fill=(196, 196, 196))
   del d
   img.save(img_file, "PNG")
   print "The plot was saved into the %s file." % (img_file,)  

cm = []
coords = [] 

def eval_func(chromosome):
   """ The evaluation function """
   global cm
   return tour_length(cm, chromosome) 

def cities_random(cities, xmax=800, ymax=600):
   """ get random cities/positions """
   coords = []
   for i in xrange(cities):
      x = random.randint(0, xmax)
      y = random.randint(0, ymax)
      coords.append((float(x), float(y)))
   return coords

#Individuals
class Individual:
    score = 0
    length = 30
    seperator = ' '
    def __init__(self, chromosome=None, length=30):
        self.chromosome = chromosome or self._makechromosome()
        self.length = length
        self.score = 0  # set during evaluation  

    def _makechromosome(self):
        "makes a chromosome from randomly selected alleles."
        chromosome = []
        lst = [i for i in xrange(self.length)]
        for i in xrange(self.length):
            choice = random.choice(lst)
            lst.remove(choice)
            chromosome.append(choice)
        return chromosome

    def evaluate(self, optimum=None):
        self.score = eval_func(self.chromosome)

    def crossover(self, other):
        left, right = self._pickpivots()
        p1 = Individual()
        p2 = Individual()
        c1 = [ c for c in self.chromosome \
               if c not in other.chromosome[left:right + 1]]
        p1.chromosome = c1[:left] + other.chromosome[left:right + 1]\
                         + c1[left:]
        c2 = [ c for c in other.chromosome \
               if c not in self.chromosome[left:right + 1]]
        p2.chromosome = c2[:left] + self.chromosome[left:right + 1] \
                        + c2[left:]
        return p1, p2

    def mutate(self):
        "swap two element"
        left, right = self._pickpivots()
        temp = self.chromosome[left]
        self.chromosome[left] = self.chromosome[right]
        self.chromosome[right] = temp   

    def _pickpivots(self):
        left = random.randint(0, self.length - 2)
        right = random.randint(left, self.length - 1)
        return left, right    

    def __repr__(self):
        "returns string representation of self"
        return '<%s chromosome="%s" score=%s>' % \
               (self.__class__.__name__,
                self.seperator.join(map(str, self.chromosome)), self.score)  

    def copy(self):
        twin = self.__class__(self.chromosome[:])
        twin.score = self.score
        return twin

    def __cmp__(self, other):
        return cmp(self.score, other.score)

class Environment:
    size = 0
    def __init__(self, population=None, size=100, maxgenerations=1000,\
                 newindividualrate=0.6,crossover_rate=0.90,\
                 mutation_rate=0.1):
        self.size = size
        self.population = self._makepopulation()
        self.maxgenerations = maxgenerations
        self.newindividualrate = newindividualrate
        self.crossover_rate = crossover_rate
        self.mutation_rate = mutation_rate
        for individual in self.population:
            individual.evaluate()
        self.generation = 0
        self.minscore = sys.maxint
        self.minindividual = None
        #self._printpopulation()
        if PIL_SUPPORT:
            write_tour_to_img(coords, self.population[0].chromosome,\
                              "TSPstart.png")
        else:
            print "No PIL detected,can not plot the graph"

    def _makepopulation(self):
        return [Individual() for i in range(0, self.size)]

    def run(self):
        for i in range(1, self.maxgenerations + 1):
            print "Generation no:" + str(i)
            for j in range(0, self.size):
                self.population[j].evaluate()
                curscore = self.population[j].score
                if curscore < self.minscore:
                    self.minscore = curscore
                    self.minindividual = self.population[j]
            print "Best individual:", self.minindividual
            if random.random() < self.crossover_rate:
                children = []
                newindividual = int(self.newindividualrate * self.size / 2)
                for i in range(0, newindividual):
                    selected1 = self._selectrank()
                    selected2 = self._selectrank()
                    parent1 = self.population[selected1]
                    parent2 = self.population[selected2]
                    child1, child2 = parent1.crossover(parent2)
                    child1.evaluate()
                    child2.evaluate()
                    children.append(child1)
                    children.append(child2)
                for i in range(0, newindividual):
                    #replce with child
                    totalscore = 0
                    for k in range(0, self.size):
                        totalscore += self.population[k].score
                    randscore = random.random()
                    addscore = 0
                    for j in range(0, self.size):
                        addscore += (self.population[j].score / totalscore)
                        if addscore >= randscore:
                            self.population[j] = children[i]
                            break
            if random.random() < self.mutation_rate:
                selected = self._select()
                self.population[selected].mutate()
        #end loop
        for i in range(0, self.size):
                self.population[i].evaluate()
                curscore = self.population[i].score
                if curscore < self.minscore:
                    self.minscore = curscore
                    self.minindividual = self.population[i]
        print "..................Result........................."
        print self.minindividual
        #self._printpopulation()

    def _select(self):
        totalscore = 0
        for i in range(0, self.size):
            totalscore += self.population[i].score
        randscore = random.random()*(self.size - 1)
        addscore = 0
        selected = 0
        for i in range(0, self.size):
            addscore += (1 - self.population[i].score / totalscore)
            if addscore >= randscore:
                selected = i
                break
        return selected

    def _selectrank(self, choosebest=0.9):
        self.population.sort()
        if random.random() < choosebest:
            return random.randint(0, self.size * self.newindividualrate)
        else:
            return random.randint(self.size * self.newindividualrate,\
                   self.size - 1)

    def _printpopulation(self):
        for i in range(0, self.size):
            print "Individual ", i, self.population[i]      

def main_run():
    global cm, coords
    #get cities's coords
    coords =cities_random(30)
    cm = cartesian_matrix(coords)
    ev = Environment()
    ev.run()
    if PIL_SUPPORT:
        write_tour_to_img(coords, ev.minindividual.chromosome, \
                          "TSPresult.png")
    else:
        print "No PIL detected,can not plot the graph"
if __name__ == "__main__":
    main_run()

The paper talks about how to measure the amount of information required by evolutionary algorithms in order for them to work. The paper describes accounting processes for measuring where information is put into evolutionary algorithms, so that it becomes clear how much information is needed up front. The paper shows that the information present in evolutionary algorithms does not emerge as a result of evolutionary algorithms – the “active information” has to be put into the process at different points for the process to work.

The paper:

“Conservation of Information in Search: Measuring the Cost of Success,” published in IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans.

The MP3 file for Part 1 is here.

Part 1 Topics:

• about the IEEE, the publisher of the paper Dembski and Marks
• the search problem and evolutionary algorithms
• the concept of “active information”
• blind search versus targeted searches
• No Free Lunch theorems
• Conservation of information
• Responding to an objection: is Darwinism analogous to the search problem?
• does evolution have mathematical underpinnings?

The MP3 file for Part 2 is here.

Part 2 Topics:

• fitness functions and fitness landscape
• constraints as information
• does the paper support intelligent design?
• how have critics responded to this paper?

You can read the entire paper for free here.

(This is going to be another long one.)

When we last left our heroes they were looking into JGAP as a cool framework for the creation of genetic algorithms. One simple example left them warm and fuzzy and the second left them temporarily bewildered. Taking a better look at the Watchmaker Framework helped clear their minds and made them realize that more thought was needed. They stopped, took a deep breath and ordered cappuccinos.

In the course of human events, research is inevitable. In my research I ran across Mehdi Khoury’s tutorial page which included a comparison of different genetic programming packages as of June 2007. The package he rated the highest? A package named ECJ developed at George Mason University’s Evolutionary Computation Laboratory.

In the course of my research in GP frameworks I decided I wanted to give ECJ a try as well. Nothing like a recommendation from a web site I’ve never heard of to influence my decision making capabilities about cool technologies.

I will be reversing the examples as the Hello World example is really about genetic algorithms while the Simple Math Test, taken from Toby Segaran’s Programming Collective Intelligence, is about genetic programming. It just makes sense to do GA before GP.

If you disagree I look forward to reading your blog posting where you do the reverse.

Hello World – A Genetic Algorithm

Input: the alphabet + a comma + an exclamation point + a space
Output: “Hello, world!”

The ECJ Properties File

The use of properties files is a very Java thing. Not to say that other programming languages don’t use something similar, but Java has made their use a virtue. In some cases, properties files are fantastic, while in other cases XML files are better. Folks who prefer verbose descriptions prefer XML and all others prefer the key=value format of properties; which you decide to use should be based on the problem at hand and not a knee-jerk reaction. Good luck figuring out if you’re having a brain-based knee-jerk reaction right now.

For this ECJ example, I rearranged the various properties so that related key/value pairs would be together. I hope this will make the explanation more coherent.

The first thing to bear in mind with ECJ is that you don’t get to write main(). The properties file tells the controller class ec.Evolve what to do and it does it with great verve and joie de vivre. I am sure you could write your own controller class to execute ECJ within your own applications, but that is left as an exercise for the reader.

The properties file is called from the command line like so:

java ec.Evolve -file [properties file]

As long as the ECJ framework is in the classpath (there is no JAR file to speak of, but you can always make your own), your custom classes are in the classpath, and the path to the properties file is accurate you should be good to go. I have run this example from within Eclipse and on the command line (using Kubuntu) and the results are the same.

verbosity    = 0

breedthreads = 1
evalthreads  = 1
seed.0       = 4357

The first batch of properties are global framework configurations:

• log file level verbosity (0 – output everything, all the way to 5000 – output nothing). More detail here.
• number of threads used for crossover/breeding
• number of threads used for population evaluation
• random number seed

state       = ec.simple.SimpleEvolutionState

pop         = ec.Population
init        = ec.simple.SimpleInitializer
finish      = ec.simple.SimpleFinisher
breed       = ec.simple.SimpleBreeder
eval        = ec.simple.SimpleEvaluator
stat        = ec.simple.SimpleStatistics
exch        = ec.simple.SimpleExchanger

generations          = 100
quit-on-run-complete = true

Next comes a number of existing classes to accomplish basic tasks. I am eternally grateful to them for the amount of work they saved me. The ones deserving of an explanation are:

• ec.simple.SimpleEvolutionState – a subclass of EvolutionState. It contains the configuration information needed by the framework to get your genes evolving.
• ec.Population – contains the current collection of Individuals/chromosomes that have been bred and/or evaluated.
• ec.simple.SimpleBreeder – creates/breeds Individuals. While ECJ can handle multiple sub-populations the SimpleBreeder does not.

The generations and quit-on-run-complete are complementary; continue to evolve until 100 generations are completed or our fitness function has a perfect match.

checkpoint           = false
prefix               = ec
checkpoint-modulo    = 1

stat.file            = $out.stat

breed.elites.0       = 1

Checkpoint file configurations are defined here. The non-obvious ones are:

• prefix – the prefix used for the checkpoint file (not used in this example, but necessary to run Evolve)
• checkpoint-modulo – run a checkpoint every generation or after every N generations?
• stat.file – name of the output file. The $ means write it in the folder where the Java process started. If a relative path is used then use the folder where the process started as the anchor for the relative path.

pop.subpops  = 1
pop.subpop.0 = ec.Subpopulation

pop.subpop.0.size                   = 100
pop.subpop.0.duplicate-retries      = 0
pop.subpop.0.species                = ec.vector.GeneVectorSpecies
pop.subpop.0.species.ind            = ec.vector.GeneVectorIndividual
pop.subpop.0.species.fitness        = ec.simple.SimpleFitness
#
# Hello, world! is 13 characters long
#
pop.subpop.0.species.genome-size    = 13
pop.subpop.0.species.crossover-type = two
pop.subpop.0.species.crossover-prob = 1.0
pop.subpop.0.species.mutation-prob  = 0.05

pop.subpop.0.species.pipe          = ec.vector.breed.VectorMutationPipeline
pop.subpop.0.species.pipe.source.0 = ec.vector.breed.VectorCrossoverPipeline
pop.subpop.0.species.pipe.source.0.source.0 = ec.select.TournamentSelection
pop.subpop.0.species.pipe.source.0.source.1 = ec.select.TournamentSelection

select.tournament.size = 2

The next group defines the configuration of the population:

• Only one sub-population
• Use the ec.Subpopulation class as its container
• Create 100 Individuals
• Use the GeneVectorSpecies and GeneVectorIndividual as the container for my custom gene
• Use SimpleFitness to determine who are the current winners. Our custom fitness function will configure this for every individual
• The genome size matches the length of our string…in this case 13. One character per gene
• The crossover-type defines one of five possible ways to breed between the selected individuals. Type two means that the genes between two points in the chromosome will be swapped out with each other.
• The crossover probability and mutation probability defines how often a crossover and mutation will occur: a one means all the time, 0.05 means not so often.
• The mutation and crossover pipelines contain selection objects that decide who gets to breed and who gets mutated. TounamentSelection selects a number of individuals at random (how many is defined in the property select.tournament.size) and picks a winner based on the individual’s fitness value.

# each of these is really on one line
pop.subpop.0.species.gene
                = hiddenclause.example.ecj.CharVectorGene
pop.subpop.0.species.gene.alphabet
                = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXY Z,!

eval.problem = hiddenclause.example.ecj.HelloWorld

Finally, I defined my 2 implementation classes (my gene and my fitness function) and their parameters.

There was no gene/individual type I could use so I defined a subclass of VectorGene and called it (wait for it) CharVectorGene. Since that gene is responsible for creating more genes, a Prototype for you Design Patterns geeks, it will also contain the alphabet containing the allowed characters to be used in this example.

HelloWorld is the fitness function that will push the population into evolving into a greeting.

Fitness Function

ECJ has the concept of an Individual where JGAP has a Chromosome and Watchmaker has AbstractCandidateFactorys to create the chromosomes out of any type you want. In the code I do the same check as before: check that the proper character is at the proper location; give it a point for every hit.

        int fitnessValue = 0;

        GeneVectorIndividual charVectorIndividual
                              = (GeneVectorIndividual) individual;
        long length = charVectorIndividual.size();
        for (int i = 0; i < length; i++) {
            CharVectorGene charVectorGene
                      = (CharVectorGene) charVectorIndividual.genome[i];
            char actual = charVectorGene.getAllele();
            if (actual == _expected[i]) {
                fitnessValue += 1;
            }
        }

In ECJ I am responsible for configuring the Fitness object as we get closer to the perfect message so my fitness function is not the ultimate arbiter of a gene’s fitness.

    SimpleFitness fitness = (SimpleFitness) charVectorIndividual.fitness;
    fitness.setFitness(evolutionState, fitnessValue,
                    fitnessValue == charVectorIndividual.genomeLength());

CharVectorGene – A Gene for Chars

This is where the gene is both created and initialized. The setup() method is only called once for the entire run to load up any parameters, in this case the desired alphabet.

    public void setup(final EvolutionState state, final Parameter base) {
        super.setup(state, base);

        Parameter def = defaultBase();

        String alphabetStr = state.parameters.getStringWithDefault(
                     base.push(P_ALPHABET), def.push(P_ALPHABET), "");
        if (alphabetStr.length() == 0)
            state.output.fatal(
                  "CharVectorGene must have a default alphabet",
                  base.push(P_ALPHABET));

        alphabet = alphabetStr.toCharArray();
    }

Every gene has to have a character. This method randomly assigned a character to itself.

    public void reset(EvolutionState state, int thread) {
        int idx = state.random[thread].nextInt(alphabet.length);
        allele = alphabet[idx];
    }

There are four standard Java methods that turn out to be quite important to ECJ.

    public boolean equals(Object other)
    public int hashCode()
    public Object clone()
    public String toString()

Standard Java rules apply as to how you should implement them. I probably did not do such a good job.

ECJ’s HelloWorld Output

After running HelloWorld I found that the output to stdout doesn’t do anything more than tell you how many generations have passed and that (maybe) a solution was found. For the really interesting output you want to look at the out.stat file (ECJ automatically adds a space between each letter when it collects the stringified version of each gene):

...
Generation: 45
Best Individual:
Evaluated: T
Fitness: 13.0
 H e l l o ,   w o r l d !

Best Individual of Run:
Evaluated: T
Fitness: 13.0
 H e l l o ,   w o r l d !

The most interesting thing about the above is that ECJ came up with the solution in the least number of generations (JGAP: 233, Watchmaker: 125, ECJ: 45). It would appear that ECJ, though it took me longer to figure out, is the best petri dish so far. I am sure there is fine tuning that could be done to make the various toy example behave similarly, but I think I’ll leave that to someone who cares as an exercise for the reader.

Miscellaneous Comments

Okay, maybe I should have titled this section Miscellaneous Complaints.

The above example took me about a week of on-and-off thought. There was no simple way for me to discover what I did not know about ECJ, but it did a great job of highlighting over and over again that I knew nothing. Since there was no existing class to handle strings I had to figure out, sans documentation, what I needed to do. While intellectually interesting, it was quite frustrating. As you might expect there were a lot of dead-ends in my search.

In addition, due to the flexible method of assigning classes to various categories there should be a lot of checks for class types to insure the run doesn’t crash due to a bad case of downcasting. The ECJ examples used them; I removed them from my code.

While I was happy to finally figure out how to make ECJ work I also have to admit I was exhausted by the time that happened. Hunting through the Javadocs and various README files was decidedly unsatisfying.

Without going into a lot of detail, and I won’t, the ECJ framework appears to be quite powerful. While it has self-admitted warts the most interesting design choice I found was the use of properties files to glue everything together. JGAP and Watchmaker don’t use properties files at all though they might; I just didn’t run into them. ECJ’s use of them is very Spring-like only without the XML and the strict usage of interfaces. I am a big fan of the Spring Framework so ECJ gained points on the use of properties as a pseudo-dependency-injection file, but lost points by not using interfaces properly.

My big suggestion to the ECJ team: port ECJ to Spring and use more interfaces in the implementation code (or more accurately, stop downcasting the interfaces if you know that someone could mess with the properties files). Yeah, the use of XML is ugly, but it makes extending ECJ more consistent and should make writing tests for the various framework components easier.

Or not. I know it is going to be a lot of work and who knows how much code you want to maintain backward compatibility with.

On second thought, leave it alone. How about some more documentation?

The Code

helloworld.params

verbosity    = 0

breedthreads = 1
evalthreads  = 1
seed.0       = 4357

state  = ec.simple.SimpleEvolutionState

pop    = ec.Population
init   = ec.simple.SimpleInitializer
finish = ec.simple.SimpleFinisher
breed  = ec.simple.SimpleBreeder
eval   = ec.simple.SimpleEvaluator
stat   = ec.simple.SimpleStatistics
exch   = ec.simple.SimpleExchanger

generations             = 100
quit-on-run-complete    = true
checkpoint              = false
prefix                  = ec
checkpoint-modulo       = 1

stat.file       = $out.stat

pop.subpops     = 1
pop.subpop.0    = ec.Subpopulation

pop.subpop.0.size                  = 100
pop.subpop.0.duplicate-retries     = 0
pop.subpop.0.species               = ec.vector.GeneVectorSpecies
pop.subpop.0.species.ind           = ec.vector.GeneVectorIndividual
pop.subpop.0.species.fitness       = ec.simple.SimpleFitness

# Place on one line
pop.subpop.0.species.gene
    = hiddenclause.example.ecj.CharVectorGene
# Place on one line
pop.subpop.0.species.gene.alphabet
    = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXY Z,!
#
# Hello, world! is 13 characters long
#
pop.subpop.0.species.genome-size    = 13
pop.subpop.0.species.crossover-type = two
pop.subpop.0.species.crossover-prob = 1.0
pop.subpop.0.species.mutation-prob  = 0.05

# Place on one line
pop.subpop.0.species.pipe
    = ec.vector.breed.VectorMutationPipeline
# Place on one line
pop.subpop.0.species.pipe.source.0
    = ec.vector.breed.VectorCrossoverPipeline
pop.subpop.0.species.pipe.source.0.source.0 = ec.select.TournamentSelection
pop.subpop.0.species.pipe.source.0.source.1 = ec.select.TournamentSelection

select.tournament.size = 2

eval.problem = hiddenclause.example.ecj.HelloWorld

breed.elites.0 = 1

HelloWorld.java

/**
 * HelloWorld.java
 *
 * This is an example only! Use it for anything else at your own risk!
 * You have been warned! Coder/user beware!
 */
package hiddenclause.example.ecj;

import ec.EvolutionState;
import ec.Individual;
import ec.Problem;
import ec.simple.SimpleFitness;
import ec.simple.SimpleProblemForm;
import ec.vector.GeneVectorIndividual;

public class HelloWorld extends Problem implements SimpleProblemForm {
    private char[] _expected = "Hello, world!".toCharArray();

    public void evaluate(final EvolutionState evolutionState,
                                    final Individual individual,
                                    final int subPopulation,
                                    final int threadNum) {
        if (individual.evaluated)
            return;

        int fitnessValue = 0;

        GeneVectorIndividual charVectorIndividual = (GeneVectorIndividual) individual;
        long length = charVectorIndividual.size();
        for (int i = 0; i < length; i++) {
            CharVectorGene charVectorGene
                    = (CharVectorGene) charVectorIndividual.genome[i];
            char actual = charVectorGene.getAllele();
            if (actual == _expected[i]) {
                fitnessValue += 1;
            }
        }

        SimpleFitness fitness
                         = (SimpleFitness) charVectorIndividual.fitness;
        fitness.setFitness(evolutionState, fitnessValue,
                fitnessValue == charVectorIndividual.genomeLength());

        charVectorIndividual.evaluated = true;
    }

    public void describe(final Individual individual,
                         final EvolutionState state,
                         final int subPopulation,
                         final int threadNum,
                         final int log, final int verbosity) {
        // Do Nothing
    }
}

CharVectorGene.java

/**
 * CharVectorGene.java
 *
 * This is an example only! Use it for anything else at your own risk!
 * You have been warned! Coder/user beware!
 */
package hiddenclause.example.ecj;

import ec.EvolutionState;
import ec.util.Parameter;
import ec.vector.VectorGene;

/**
 * @author carlos
 */
public class CharVectorGene extends VectorGene {
    public final static String P_ALPHABET = "alphabet";

    private static char[]      alphabet;
    private char               allele;

    @Override
    public void setup(final EvolutionState state, final Parameter base) {
        super.setup(state, base);

        Parameter def = defaultBase();

        String alphabetStr = state.parameters.getStringWithDefault(
                          base.push(P_ALPHABET), def.push(P_ALPHABET), "");
        if (alphabetStr.length() == 0)
            state.output.fatal(
                       "CharVectorGene must have a default alphabet",
                       base.push(P_ALPHABET));

        alphabet = alphabetStr.toCharArray();
    }

    /*
     * (non-Javadoc)
     * @see ec.vector.VectorGene#reset(ec.EvolutionState, int)
     */
    @Override
    public void reset(EvolutionState state, int thread) {
        int idx = state.random[thread].nextInt(alphabet.length);
        allele = alphabet[idx];
    }

    public char getAllele() {
        return allele;
    }

    /*
     * (non-Javadoc)
     * @see ec.vector.VectorGene#equals(java.lang.Object)
     */
    @Override
    public boolean equals(Object other) {
        if (!this.getClass().isInstance(other)) {
            return false;
        }

        CharVectorGene that = (CharVectorGene) other;

        return allele == that.allele;
    }

    /*
     * @see ec.vector.VectorGene#hashCode()
     */
    @Override
    public int hashCode() {
        int hash = this.getClass().hashCode();

        hash = (hash << 1 | hash >>> 31) ^ allele;

        return hash;
    }

    @Override
    public Object clone() {
        CharVectorGene charVectorGene = (CharVectorGene) (super.clone());

        return charVectorGene;
    }

    @Override
    public String toString() {
        return Character.toString(allele);
    }
}

Next time: the ECJ version of Toby Segaran’s Simple Math Test.

Or maybe I will do that first in Watchmaker.

I enjoyed a post today at Hidden Clause talking about an attempt to generate the string ‘hello, world!’ using a genetic algorithm.

In the first chapter of the documentation for the Watchmaker Framework Daniel W. Dyer presents an interesting genetic algorithm example: what if we wanted to evolve the string “HELLO WORLD”? … My first thought: implementing this in JGAP shouldn’t be that hard.

The setup reminds me of the game ‘Mastermind’ — we’re dealing with a fixed-length string and scoring based on how many of the correct characters occur in the string and if they are in the right position. You combine some better strings and sometimes mutate them to generate new strings, and repeat that process until, hopefully, you arrive at the target string, ‘hello, world!’.

Did it work? Well, almost. The first result, “‘!ello, or!d!”,  was reached after 1 million iterations on a population size of a thousand — and, it used a limited alphabet containing only the letters and punctuation in the target string (and not, for example, ‘z’).

The reason the solution was not reached, the post’s author notes, is that the algorithm doesn’t have the breadth or the convergent ability to match the statistical odds involved. Recalling the old problem of monkeys on typewriters producing Shakespeare , the author notes how unlikely it is to produce the desired result.

Of course, we’re not simply monkeying at typewriters. We’re making random changes and mutations, but we’re intelligently selecting the parents. Apparently, the method first tried wasn’t intelligent enough to arrive at the desired solution within that number of iterations, and increasing iterations or population size, apparently, isn’t going to be practical.  So we need to use a better way of selecting the parents.

We could improve the fitness function. Let’s say we take the whole alphabet and the punctuation and number them (for instance: A=1… Z=26 and space and comma and exclamation could be 27, 28, and 29, respectively). Now, assign fitness based on the difference between actual and target values of each character. Is this cheating? Well, no. Our new fitness function requires no a priori knowledge of the solution and will work equally well for a different target string. This is providing our GA with a lot more information per epoch, and therefore we should see a much faster convergence.

(Aside: there are much better way of numbering the letters and punctuation to encourage faster convergence, based on the frequency the characters are used in the English language. In fact, one could develop a system that took advantage of commonly grouped letters like ‘th’ and ‘ing’, and if dealing with sentences, commonly grouped words…  Is that cheating? Well, we must remember that the more such information we encode, the more pre-disposed or biased our output will be. Also, the point of a using a genetic algorithm is to substitute computation time for programming complexity. In a contrived problem, it’s up to us where that line lies. )

There are a number of other ways to improve the fitness function, but my instinct is that this suggestion or a similar modification would be sufficient. I hope the Hidden Clause crew reports the result of another attempt.

In the first chapter of the documentation for the Watchmaker Framework Daniel W. Dyer presents an interesting genetic algorithm example: what if we wanted to evolve the string “HELLO WORLD”?

The fitness function would be straightforward: check every letter of a possible solution and every matching letter at a matching position would raise that chromosome’s fitness value.

I highly recommend you read the chapter referenced above before continuing. It is quite well written and will make the pain I describe below clearer.

My first thought: implementing this in JGAP shouldn’t be that hard.

Implementing it: easy. Getting a good solution: not so much. This is not a problem with JGAP so much as it is a problem with monkeys and Shakespeare. Before I go into that let me present the code.

My target was a little bit more ambitious: instead of hello world in all upper case characters I wanted to evolve the string “hello, world!” all in lower case and containing a space, a comma and an exclamation point.

The fitness function was easy: check every letter and its position and if it matches the target string increment the fitness value by 1. The higher the fitness value the better the match.

Here is my fitness function:

package hiddenclause.example;

import org.jgap.FitnessFunction;
import org.jgap.IChromosome;

public class HelloWorldFitnessFunction extends FitnessFunction {

    private char[] _expected;

    public HelloWorldFitnessFunction(String desiredMessage) {
        _expected = desiredMessage.toCharArray();
    }

    @Override
    protected double evaluate(IChromosome aSubject) {
        int fitnessValue = 0;

        String msg = (String) aSubject.getGene(0).getAllele();
        char [] actual = msg.toCharArray();
        for (int i = 0; i < actual.length; i++) {
            if (actual[i] == _expected[i]) {
                fitnessValue += 1;
            }
        }

        return fitnessValue;
    }

}

Next step: implement the chromosome that will contain the letters that will evolve into the target string.

In order to accomplish the next step I decided to use the JGAP genetic algorithm* class StringGene. I configured it to randomly generate a string of exactly the length desired and use an alphabet of just the letters and punctuation from the target string. Why not add the entire alphabet and punctuation to add to the diversity? Monkeys, remember? More on that later.

The chromosome creation code is:

package hiddenclause.example;

import org.jgap.Chromosome;
import org.jgap.Configuration;
import org.jgap.FitnessFunction;
import org.jgap.Gene;
import org.jgap.Genotype;
import org.jgap.IChromosome;
import org.jgap.InvalidConfigurationException;
import org.jgap.impl.DefaultConfiguration;
import org.jgap.impl.StringGene;

public class HelloWorld {

    private static final String MESSAGE = "hello, world!";
    private static final int MAX_EVOLUTION = 1000000;

    public static void main(String[] args) throws Exception {
        long startTime = 0;
        long endTime = 0;
        HelloWorld hello = new HelloWorld();

        Genotype population = hello.create(1000);

        startTime = System.currentTimeMillis();
        int i = 0;
        for (i = 0; i < MAX_EVOLUTION; i++) {
            population.evolve();
            IChromosome solution = population.getFittestChromosome();
            if (solution.getFitnessValue() == MESSAGE.length()) {
                break;
            }
        }
        endTime = System.currentTimeMillis();

        outputSolution(population, startTime, endTime, i);
    }

    private Genotype create(int popSize) throws InvalidConfigurationException {
        Configuration conf = new DefaultConfiguration();
        conf.setPreservFittestIndividual(true);

        FitnessFunction myFunc = new HelloWorldFitnessFunction(MESSAGE);
        conf.setFitnessFunction(myFunc);

        Gene stringGene = new StringGene(conf, MESSAGE.length(), MESSAGE.length(),
                "!dehlorw ,");
        IChromosome sampleChromosome = new Chromosome(conf, stringGene, 1);
        conf.setSampleChromosome(sampleChromosome);
        conf.setPopulationSize(popSize);

        Genotype population;
        population = Genotype.randomInitialGenotype(conf);

        return population;
    }

    private static void outputSolution(Genotype population, long startTime,
            long endTime, int evolutionIdx) {
        System.out.println("Stopped at generation " + evolutionIdx);

        long totalSeconds = (endTime - startTime) / 1000;
        long actualSeconds = totalSeconds % 60;
        long actualMinutes = totalSeconds / 60;
        System.out.println("Total evolution time: " + actualMinutes + ":"
                + actualSeconds);

        IChromosome solution = population.getFittestChromosome();
        int fitnessValue = (int) solution.getFitnessValue();
        System.out.println("The best solution has a fitness value of " + fitnessValue);

        System.out.println("Looking for '" + MESSAGE + "':");
        String message = (String) solution.getGene(0).getAllele();
        System.out.println("\t '" + message + "'");
    }
}

Notice that the HelloWorld class will handle any kind of incoming string; I just happened to use a hard coded constant. One of its assumptions (euphemism for bug) is that the incoming string and the solution string are the same length. If the two strings are not the same length the loop might try to access part of an array that doesn’t exist and there are very few exceptions that cause as severe a level of mental anguish as the ArrayIndexOutOfBoundsException (however, NullPointerException is right up there).

The above gave the following output after 1 million generations:
Break: Executed evolution 1000000 times.
Total evolution time: 76:45
The best solution has a fitness value of 10
Looking for 'hello, world!':
'!ello,  or!d!'

Not pretty.

What happened?

While the use of crossover should make the evolution progress faster there are certain statistical issues that must be taken into account. For a chromosome to get a high value it has to have 2 things in its favor:

1. it has to have one or more letters in the new string
2. the letter(s) have to be in the proper position(s)

This is where the monkeys come in.

The more I thought of it the more I thought about the scenario of an infinite number of monkeys trying to write Shakespeare. To begin their arduous task one or more of the monkeys has to hit the proper first letter. Assuming a keyboard with just the alphabet, and no punctuation, they have a 1 in 52 chance of hitting the right first letter. Shouldn’t be too hard with an infinite number of primates.

The next letter becomes a little harder. Of the subset of monkeys that hit the proper first letter the chances of any of them hitting that second letter has a statistical probability of 1 in 2704. The third: 1 in 140608. The fourth: 1 in 7,311,616. The fifth: 1 in 380,204,032.

And that doesn’t include punctuation and white space.

That mean that if the letter h never makes it to the first character the system will never (never, as in…well, never) evolve the target string. Never. In order to up the statistical odds of my chromosome succeeding I created an alphabet of just the letters that comprise the target string: “!dehlorw ,”.

And yet, with that small set of characters, an ever changing population of 1000 chromosomes at 1 million generations was unable to generate the string. Can it succeed? Of course. The proper population (random generation of strings) in the petri dish that is JGAP would generate the target string. But it is random and random does not mean regular.

So, adding more letters and punctuation adds more noise to the mix and just makes the combinations even worse (though possibly better as more randomly mixed letters could elevate the use of letters from the target string).

When I changed the StringGene’s alphabet to include a few more letters, “!abcdefghlmnorstwxyz ,”, this is what the output was:

Stopped at generation 1000000
Total evolution time: 77:20
The best solution has a fitness value of 7
Looking for 'hello, world!':
'zblxo, ,orrdx'

Defintely not an improvement.

Is there a way of configuring this differently to encourage a solution? Perhaps a better fitness function? Custom gene? Better randomizer? I will gladly take any suggestions and try them out. Maybe.

Exercise for the reader: up the population value to create more opportunities for the desired letters to end up in the desired positions. Bake overnight.

When I increased the population number to 800K and ran it again my Kubuntu box locked up after 2 days because I continued to do things like run Eclipse, read PDFs and back-up my work. I guess 800 trillion iterations of the fitness function was exhausting.

No joy in Mudville.

Update 8/15/09: the problem solved.

* To create a genetic program using JGAP you would create a GPGenotype using a class that inherits from GPProblem and a fitness function that inherits from GPFitnessFunction.

Open Software Challenge Nepal 2009 was announced somewhere during early 2009. And somehow, computer-half of the CSIDC team (me-Prajwol and Dipesh) decided to participate in it.

Screen blurred, grayscale stars – To talk about CSIDC, it was CSIDC 2006, organized by the IEEE under the theme of “Preserving, Protecting and Enhancing the Environment”. Saurav Ratna Tuladhar and Nilesh Shakya (electronics), and Dipesh Karki and Prajwol Kumar Nakarmi –me (computer) were selected from Pulchowk Campus to participate in it. We had presented our project “iForest – A Monitoring and Management System for Sustainable Forestry”. The certificate of participation of this competition has definitely increased the weight of our CVs – Screen normal, colorscale resumes. he he

Dipesh and I talked in phone, texted in MSN, brainstormed in Eiden Garden and came out with what we christened “Computational Intelligence in Inflation Forecasting” (CIIF). Of course Carlsberg was there to accompany us whenever possible. LOL

Now on August, we both are proud to say that we made to the prestigious TOP 10 among some 80+ projects. My heartfelt congratulations to the guys who made it to the TOP 3 (Y).

Many were expecting our project to do better before the result announcement, and of course, we were hopeful too.  But probably, the fact that our development platform was .NET pulled our legs and we didn’t make till the TOP 3 mark. We had some people asking “how can your software be open when I have to pay for it” LOL LOL a big disgrace to such a question. We tried to explain – they seemed to agree – it was apparent that they didn’t understand :’(. May the true meaning of Openness of the Software come to light to all of us, AMEN.

Let me, now, say some about CIIF. As we know, Inflation is a persistent and significant increase in general price level and it is calculated by averaging the percentage growth rate of the prices of selected sample of commodities. It is one of the most essential macroeconomic indicator as many other economic variables like wage rate, salaries , gross domestic product, interest rate are either directly or indirectly related to change in price. However the inflation can be calculated only in hindsight by comparing present commodity price with the past. As this isn’t much help while formulating the future pricing policy, a forecasting mechanism must be introduced to estimate the future expectation of the inflation rate.

In this light, the software, that attempts to forecast one of the key macroeconomic indicator- ‘Inflation’ using both Neural Network and Genetic Algorithm, was developed. Ladies and Gentlemen, we give to you CIIF . The system consists of two main engines, viz. Artificial Neural Network and Genetic Algorithm. Two flavors of the later are available as Encoding and No Encoding. Statistics are present to measure the fitness of the predicted data. For easy visualization of the output, Graph component is also available.

The input data consisted of inflation rates of USA for last 1143 months, dating back to January of 1914 to March of 2009.  Before beginning the training, the data set is divided into the training set and the validation (Prediction set). First 1043 months data are kept in training set while last 100 are kept in validation set. The application is supple in sense that by simply changing the parameter in the input file the size of validation set as well as training set can be changed. The value of R2 >= 0.5 is acceptable for our system, which implies that at least 50pc of total variation in the data should be explained by the system. Our tests show that the results are over par.

You can follow CIIF under http://collaborate.d2labs.org/projects/ciif/

Cheese and Cheers

Prajwol

http://4volt.com/projects/jansen/

This is what i’m talking about, this project unites everything most nerdy in the world, Genetic algorithms/Arduino/Servo motor modification. It also makes for the definitive science fair project, mwahahahaha.

This little bastard converts rotary motion into leg-like motion, the mechanical segments were developed by 12 numbers that means the relative distances between joints, and these numbers were found through a genetic algorithm running on an Atari STe computer for months, Jansen himself call these numbers “The 12 holy numbers”.

The site contains schematics and arduino code, so, turn off you TV and start roboting!

Can you place three white queens and five black queens on a 5×5 chessboard so that no queen of one color attacks a queen of the other? The solution is unique, except rotations and reflections.

It took me longer than I first expected to solve this problem. The solution is:

Since a queen attacks along rank, file, and diagonal, it leaving triangles between them unattacked, with a knight-move as the closest vertex of each triangle. While that is immediately obvious to anyone familiar with chess, I was able to solve the problem by thinking about placing white queens in such a way as to allow the largest triangles. Before searching for a solution in that manner, I tried other methods unsuccessfully: hit-and-miss attempts (having at first underestimated the problem); placing queens at knight-moves as one might do for the classic eight-queen problem (see below); utilizing symmetry by placing queens at, say, each corner and at the center; and so on.

How would a computer solve the problem? Well, a wikipedia article suggests, among other methods, a genetic algorithm, which struck my fancy.  So I started writing one.

After the skeleton of the algorithm was written and I started tinkering with parameters and the fitness function, I realized this problem would make a great introduction to genetic algorithms and also provide a simple platform for various discussions about the nuances of GAs.

OK, I have always loved the idea of genetic algorithms, and specifically how they apply to solving problems in unorthodox ways.

As anyone who has read this blog knows, I am very slowly writing a game, very much like galactic arms race (http://gar.eecs.ucf.edu) in look and feel, but what these guys are doing is mind blowing. They are using AI to develop new weapons based on what users use and what they like to use.

There is also an interview with the creator at AIGameDev (http://aigamedev.com/interviews/galactic-arms-race/).

Both of these links are very worth a read, and while it will be a while before I get to this point, I plan to follow this project very closely.

Now go away – I have a game client to download…

Yann Le Cun was recently featured on NYAS’s Science and the City podcast. He spoke about visual processing using Artificial Neural Networks (ANNs) and in particular his work on the system that reads the amounts on your cheques at the ATM.  The host, Alana Range, notes that her ATM has never gotten her cheque amounts wrong. I myself no longer bother to check.

The technology behind the ATMs was developed by Le Cun and others almost 10 years ago, at AT&T Bell Labs [which, tragically, has been closed down]. The algorithm they developed [now] goes under the name LeNet, and is a multi-layer backpropagation Neural network called a Convolution Neural Network.  I will explain this terminology in my next post. [Update: explanation now posted.]

In the object recognition demonstration, Yann Le Cun describes four output interpretations in the LeNet algorithm: 1) edges and contours, 2) motifs, 3) categories, and finally 4) objects. By narrowing down options in several steps LeNet can arrive at the final outputn (identifying the object) far more rapidly — the demonstration on the podcast proceses 4-5 pictures each second, and can recognize five different objects.  There are also demonstrations online on Yann Le Cun’s website.

I wondered as I listened to this podcast about the comparisons drawn between mammalian visual processing and Le Cun’s algorithm. There were some very pronounced differences that suggest that our brains utilize a totally different technique, not simply a more complex version of LeNet, as was implied in the interview. These differences were:

1. LeNet utilizes supervised learning. Our learning is largely unsupervised.
2. We extrapolate, not just interpolate. So while LeNet needs tens of thousands of samples before it starts recognizing an object, babies might see a few toy planes and recognize the one in the sky. In interview, Le Cun notes that his algorithm, when used for letter recognition, needs to be trained with both printed and handwritten samples and is unable to extrapolate from one to the other.

I think there were some other differences that I do not now recall.  I got flashbacks of the TED-talk in which Handspring founder Jeff Hawkins cracked some jokes about the fundamental differences between computer and human visual processing. I’ll try to post about that; it was pretty entertaining.

[Update: check out a Matlab class for CNN implementation on the Matlab file exchange, by Mihail Sirotenko.]