Jogging and lifting, my MamaRuth, getting a job at J.Crew, finishing my Alogorithm’s class, moving onto to things that will last!! (Sing the melody of “My Favorite Things” when you read this line)

GYM – It has been a wonderful month of consistent Gym time! I am so excited with my results! Although weight has not decreased, I know my muscle has increased and my fat is being burned! I now lift 20lbs upper body and lower body 70lbs! For the next month I am not going to increase my weight, instead I am going to increase my reps. Right now I do 3 sets of 10. My jogging is the same, 2 to 3 miles. However, I may increase my incline. My incline has gone from 0 to 2.

MAMARUTH – I think about her often, there is not a day that goes by that I don’t. Anything and everything it seems reminds me of her. At times I cry a river and other times I smile knowing she’s in Heaven. I can only imagine the pain my mother and her brothers and sisters are going through. I fear now for the health of my parents, aunts, and uncles. I can’t help but think “Who’s next?”. I simply loathe Cancer. I signed up to be an advocate for Cancer. I am to recieve constant emails on events and things. So I’ll keep you all updated when I get more info! I have deeper future plans in this area but I do not feel the need to go into detail just yet. I was going to go visit her gravesite this week, she got her headstone. I just couldn’t muster up the courage too. I’m not sure if it’s that I don’t want to face reality once more, or that it’s all too real in general. I miss that classy lady, she taught me well!

J.Crew – On a different tone, I got a seasonal position with J.Crew! Today was my first day, just orientation and such. My manager said we could shop today if we wanted and I was sooo tempted, but I didn’t I’ll probably stop by tomorrow instead! Hahah! It was fun and I met some cool people! I can’t wait to work! J.Crew is one of my favorite stores, so it makes it all the better. Working here also brings up some more future plans! We’ll see how everything runs though! Hubby’s job is still good (Aviation Analyst) Thank God!, good ol’ San Diego! Things just keep getting better!

ALGORITHMS & LOGIC CLASS – Wow, what can I say? I thank God that he has given me such a good outlook on this class. I have been reading and understanding. Although at times it gets a bit confusing, it doesn’t overwhelm me. Next week is my final for this class! I’m grateful that it is here and I am also grateful that as of now I am still holding an A! God is good all the time!

Well my friends, it seems my close is here! I will attempt to write once before Thanksgiving! However, if I don’t make it…..HAPPY THANKSGIVING EVERYONE!!

XoXo, J

How many buzz words can you fit in a title?

Microsoft and other companies often request you to complete some algorithms in their interviewing process. Today I completed an attempt at an ArrayBasedStack to be able to find a palindrome. Something I must admit I have not touched since early college! This is going to be the first of some interview questions I got. Plus it is about time I wrote another blog post.

Using the excellent: Data Structures and Algorithms in Java by G&T to define my stack I created an array based stack similar to what they have posted. Each developer has their own style. I prefer using return size() == 0 rather than their (top < 0), but that is my preference.

Just remember a few things

A palindrome is a word, phrase, number or other sequence of units that can be read the same way in either direction (the adjustment of punctuation and spaces between words is generally permitted) – from wiki

Trim whitespace and lowercase the characters.

Stacks work on last in, first out principle. What? Well this comes in handy for the algorithm. LIFO gives you a reverse effect.

So – here it is, the algorithm. Hope it will help you with your stack based palindromes.

private boolean isPalindrome(String str) {

 Stack<Character> stack = new ArrayBasedStack<Character>(str.length());
 str = str.toLowerCase().trim(); //format the data
 while(stack.size()<(str.length()/2)) {
  //When the stack is less than half the length
  //(even numbers great, odd numbers ignore the middle!
stack.push(str.charAt(stack.size()));
//push the character at the position indexed by the size of the stack

 }

 while(stack.size()>0) { //lets go back

if(stack.top() == str.charAt(str.length() - stack.size())) {
//again use the stack size
//to index the position, minus the length
//so this counts down every time something is popped from the stack.
//You are checking the top of the stack is equal to the character
//in 2nd half of the word. since characters go in first, the characters,
//abba would have b on top and a on the bottom when hitting this loop -
//the first two characters - they are the same as the last two. Get it now? 
stack.pop(); //remove palindromic characters

} else {

return false; //it is not a palindrome -
//you don't want to keep going and be inefficient if it is wasting time!

}

 }

 return true; // if it gets here there can only be on conclusion.

}

Hope this helps and you learn my thinking behind this. Good luck with the interviews.

Visiting the Society of the Query Conference I had the chance to listen to many interesting talks on the impact of search engines (not only Google) on our lives. The two sessions on alternative search recalled Alexander Galloway’s call for the creation of alternative algorithms to my mind. In Mary Joyce’s blog I found a nice entry on Power & the Network that points out the relevance of such an endeavor. She writes that

“[...] the challenge for people who want to encourage more effective digital activism is to figure out which content will both assist activists in creating effective campaigns and which can be disseminated most effectively online.”

One way to do so and also to account for her second request, namely to provide people with

[...] digital content that not only inspires and directs citizens to act, but also tells them how to act[,]

could be in the creation of the algorithms mentioned above. It would hence not only be the content, as Joayce writes, but also about the techniques of dissemination. Rather than merely adapting to the protocols and pathways provided by the Web one should also open the eyes for alternatives. The Society of the Query conference provided some interesting hints in this direction. Particularly interesting to me were the misspelling generator that can for instance be used to temporarily circumvent censorship and the anxiety monitor, which provides a tool to visually compare the connotations of different keywords towards diverse cultural backgrounds. While these examples fall more in the first category and are dealing with using the available tools to provide alternatives to the default search options, the approach presented by Daniel von der Velden was different. His main argument for alternative search is that the interesting challenges to main stream assumptions are found at the periphery of the usual search results. The problem with thos results is that they

“[...] are often not directly connected to the statement and exist isolated as isolated worlds away from the powerful, reigning opinion. Metahaven wants to develop a search engine that connects these different spheres, to provide different points of views on particular issues and be able to put emphasis on the marginal forces.

To achieve these goals Metahaven strives to create maps that reveal the networks of power and provide a better overview on where the information can be found and how it is distributed. Through this they want to open up the black-boxes of cloud-computing and current search engines. Even though Metahaven attempts to provide a truly alternative search and wants to provide a greater visibility of alternate accounts something from Joyce’s requests seems to remain open. It is certainly not enough to provide the people with the information. Alternative algorithms also have to show their users how to act, the next step so to say. This is what will be one of the great challenges for alternative search in the future.

Big Oh notation is used in computer science to decribe the complexity of an algorithm in terms of time and space (memory). Big Oh notation describes the worst case scenario of what happens when an algorithm is run with N values and is really only useful when talking about large sets of data.

Big Oh Complexity Over Time

Source: table source here also big thanks to Dean who made the graph for me

Explanation

O(1) = irrespective of changing the size of the input, time stays the same

O(N) = as you increase the size of input, the time taken to complete operations scales linearly with that size

O(log N) = growth curve that peaks at the beginning and slowly flattens out as the size of the data sets increase

O(N²) = growth will double with each additional element in the input

Java Example for O(1):

    public static boolean isArrayOver100(String[] args) {
        if (args.length > 100)
            return true;
        return false;
    }

Java Example for O(N):

    public static boolean contains(String[] args, String value) {
        for (int i = 0; i < args.length; i++) {
            if (args[i] == value)
                return true;
        }
        return false;
    }

Java Example for O(log N):

public static int binarySearch(int[] toSearch, int key) {
    int fromIndex = 0;
    int toIndex = toSearch.length - 1;

    while (fromIndex < toIndex) {
        int midIndex = (toIndex - fromIndex / 2) + fromIndex;
        int midValue = toSearch[midIndex];

        if (key > midValue) {
            fromIndex = midIndex++;
        } else if (key < midValue) {
            toIndex = midIndex - 1;
        } else {
            return midIndex;
        }
    }
    return -1;
}

See my other post on binary search here.

Java Exmaple O(N²)

public static boolean containsDuplicates(String[] args) {
        for (int i = 0; i < args.length; i++) {
            for (int j = 0; j < args.length; j++) {
                if (i == j)
                    break;
                if (args[i] == args[j])
                    return true;
            }
        }
        return false;
    }

I took this post as my inspiration as I found it to be a great introduction but have instead used java for the code examples.

I don’t talk much about the After the Deadline spell checker. Many people believe spell checking is a solved problem and they must “work the same way”. AtD does more than spell checking, so I talk about these other things.

Despite the lack of talk, AtD’s spell checker is awesome and I’m constantly improving it. That’s what I’m going to write about today. This past weekend I was writing a post and I tried to spell “conoisur”. As you call tell, I don’t know how to spell it and my attempt isn’t even close. It’s bad enough that I make spell checkers and can’t spell, it’s terrible that AtD couldn’t give me any suggestions. I felt left out in the cold.

Suggestions before this post

Most spelling errors are one or two changes away from the intended word. AtD takes advantage of this and limits its suggestion search to words within two changes. By limiting the search space for words, AtD has less to choose from. This means AtD give you the right answer more often.

Peter Norvig conducted a quick experiment where he learned that 98.9% of the errors in the Birkbeck spelling error corpus were within two edits and 76% were one edit away. These numbers are close to what I found in my own experiments months ago. I’d present those numbers but I… lost them. *cough*

From these numbers, I felt it was safe to ignore words three or more edits away from the misspelling. This works well except when I try to spell words like connoisseur or bureaucracy. For my attempts, AtD generated no suggestions.

Then it hit me! If there are no suggestions within two edits… why don’t I try to find words within three edits.  And if there are still no suggestions then, why I don’t I try words within four edits. I could go on like this forever. Unfortunately nothing is free and doing this would kill AtD’s performance. So I decided to limit my experiment to find words within three edits when no words are within two edits.

AtD’s spellchecker uses a neural network to score suggestions and compare them to each other. During training the neural network converges on an optimal weighting for each feature I give it. One of the features I give to the neural network is the edit distance normalized between 0.0 and 1.0. Edit distance is weighted highly as a value close to 1.0 is correct 76% of the time. I feared that introducing high edit distances that are almost always correct would create something that looks like a valley to the neural network. Meaning a high value is usually correct, a low value is usually correct, and the stuff in the middle is probably not so correct. Neural networks will make good guesses but they’re dependent on being told the story in the right way. This valley thing wasn’t going to work. So I modified the features so that an edit distance of 1 is a weight of 1.0 and everything else is 0.0.

Here are the numbers on AtD’s spell checker before I made these changes:

Word Pool Accuracy

Spelling Corrector Accuracy

The word pool accuracy is a measure of how often the pool of suggested words AtD generates has the intended word for a misspelled one in the test data. If the correct word is not here, AtD won’t be able to suggest it. The corrector accuracy is a measurement of how accurate AtD is at sorting the suggestion pool and putting the intended word on the top. The neural network score is what AtD uses in production and includes context. The joint frequency/edit distance score is similar to the statistical corrector in How to write a spelling corrector. Without context I don’t expect most spell checkers will do much better than that.

Here are the numbers with the changes:

Word Pool Accuracy

Spelling Corrector Accuracy

What this means for you

These little changes let AtD expand its search for suggestions when none were found. This means better recommendations for hard to spell words without impacting AtD’s stellar correction accuracy. I hope you enjoy it.

Suggestions you'll get now

How does this compare to MS Word, ASpell, and others?

If you want to compare these numbers with other systems, I presented numbers from similar data in another blog post. Be sure to multiply the spelling corrector accuracy with the word pool accuracy when comparing these numbers in the ones in the other post. For example: 0.9451 * 0.9891 = 0.9348 = 93.48%

If you’d like to play around with spelling correction and neural networks, consider downloading the After the Deadline source code. Everything you need to conduct your own experiments is included and it’s open source. The data sets used here are available in the bootstrap data distribution.

How to find approximate Nash equilibrium for sparse games

Constantinos Daskalakis is one of the experts in modern game theory, especially the structure of Nash Equilibrium for non-zero sum games. He has written a wonderful paper with Christos Papadimitriou On oblivious PTAS’s for nash equilibrium. Also see his nice survey for more information.

Today I want to talk about Costis and Christos’ paper as it relates to sparse games. This is another example of the Iceberg Effect: their paper has a beautiful result on game theory, but I missed another of their results.

Constantinos, Costis, was kind enough to point out two things about sparse games after my earlier discussion on games. First, that in his paper with Christos at STOC 2009, they stated a theorem that solves the sparse case of symmetric games. He went on to sketch the proof, which they did not include in their paper: I will give their proof in a moment.

Second, he points out that Shanghua Teng has observed that the uniform distribution is trivially a -Nash equilibrium of a -sparse game—the expected payoff from any row is at most for the row player and similarly for the column player.

His second point is one that I would like to discuss, since it raises an issue by what we mean to approximate a Nash Equilibrium (NE).

Let’s turn to discuss first NE and then their theorem.

Nash Equilibrium: Exact and Approximate

What does it mean to find an approximate NE? I think it will be easier to first explain the main issue involved by an analogy. Anyway I love analogies.

Suppose that you have a polynomial and want to find an so that

of course, we call a root. Usually is not a rational number, so we cannot write it down exactly. Thus, what we do is try to find a so that

where is the required precision and are integers.

There is a fundamental problem: simply because is small does that mean that it is near an actual root of ? It does not, in general. What we want are the stronger statements:

1. The value of is near ;
2. There is a so that and is also small.

In this case we can say that is an approximate root of the polynomial. If there is no such , then is an interesting point, but it is not correct to call it an approximate zero.

This behavior is analogous to what happens with “approximate NE’s.” There are two types of interpretations of what an approximate NE is:

An -near NE: This is a pair of strategies that are within an of an exact NE.

An -NE: This is a pair of strategies such that a player who defects from their strategy can gain at most an .

Costis, in his survey, points out that the first notion is reasonable, but the latter is the one that we usually use: Note also that it is easy for a player to check the approximate optimality of the actions used by her mixed strategy. So the notion of an approximate Nash equilibrium is much more appealing and, in fact, this is the notion most commonly used.

Note, the analogy to the root problem: in the first we are near a root, and in the second we are at a small value. As Costis points out the latter can be checked easily, while checking that a point is near an exact root is not as easy.

Sparse Games

I will now state their theorem:

Theorem: Suppose that is an matrix with entries in , such that each row and column in has at most ’s. Then, for the symmetric game defined by , we can find an -NE in time bounded by

Here is a sketch of their proof:

Proof:

The key insight is that it is enough to consider strategies for the row and column player where the probabilities are integer multiples of . The number of such strategies for either player is at most

This follows since the number of strategies are bounded by the number of ways to place objects into “bins.”

We now have a “small” set of strategies, the following argument will show that one of these strategies contains a pair that is an -near NE. Consider an exact NE say . Notice that there exist strategies such that the following are satisfied:

1. For all : and .
2. For all : if then , and if then .

Consider . If , then set to . Otherwise, set to the nearest multiple of . This almost works: the small issue is that if is not a exact multiple of , then do we round up or down? Either rule will make satisfy the needed conditions above. However, we also need to be a probability vector: it is easy to see that by rounding some up or down we can adjust to satisfy

which will make it a probability vector. The same method works for . Now we need to prove that this strategy pair is an -near NE. The payoff for the row player is and for the column player is . Let denote a unit vector with at -th coordinate. For notational convenience, we omit the transform symbols on the vectors. The usage should be clear from the context.

Notice that the pair of strategies and the NE satisfy:

1. , for all .
2. , for all .

The first claim follows from the fact that

The second claim is analogous. This is where we use that has at most ’s in each row/column. Therefore, the total error from the NE is at most

It follows from the above that is an -near NE of the game.

The above proof shows the existence of an -near NE in the search space formed by the strategies that are multiples of . This strategy pair is near the exact NE . However, because of the difficulty of searching for an -near NE, an exhaustive search in this space can only guarantee that we would find an -NE, and not an -near NE.

Open Problems

An interesting open problem is to try to found an -near NE. The above proof shows that it is in a relatively “small” list of strategies. Is it possible to discriminate and determine which are near exact NE’s?

The above proof gives us a PTAS for games with a bounded number of 1’s in each row and column. However, for matrices with 1’s in all, the problem of finding a PTAS is still, I believe, open.

What is it?

Binary Search is a technique used to search sorted data sets. It works by comparing the middle element against the search value, if the search value is greater than the middle element a subset is created out of the upper half of the set. The same operation is then performed on the subset recursively until the value is either found and the index returned, or not found and -1 returned.

eg. If we start with with a set of ints: [1, 4, 6, 8, 9, 10, 11, 13, 15] and our search value = 11

• Choose the middle element: 9
• Is 9 > 11
• No, so take the upper half and create a subset [10, 11, 13, 15]
• Choose the middle element: 13 (4/2 =2 which is the third element in the array as arrays are zero based)
• Is 13 > 11
• No, so take the lower half and create a subset [10, 11]
• Choose the middle element: 11
• Is 11 >= 11 yes – we have found our element

Binary Search in Java

I have created my own java binary search method:

    public static int binarySearch(int[] toSearch, int key) {
        int fromIndex = 0;
        int toIndex = toSearch.length - 1;

        while (fromIndex < toIndex) {
            int midIndex = (toIndex - fromIndex / 2) + fromIndex;
            int midValue = toSearch[midIndex];

            if (key > midValue) {
                fromIndex = midIndex++;
            } else if (key < midValue) {
                toIndex = midIndex - 1;
            } else {
                return midIndex;
            }
        }
        return -1;
    }

Take a look at Arrays.binarySearch() method to see the java source code implementation of the search. If you don’t have the java source code attached to your favourite IDE I suggest you do so, it’s a great way to learn how everything works.

Complexity

The complexity of the binary search is O(log n).

Why use Binary Search?

Binary Search is quicker than searching through every element (O(n)) as it halves the amount of elements to search each iteration. This halving means that after a strong growth curve at the beginning, it slowly flattens out as the size of the input data set increases, therefore it is good for large data sets.

More Info

There is a good introductory explanation of binary search and O notation here.