Tomorrow i have Exam on Data Structures. For preparing for that i found the following study materials to be very useful:
1. AVL Trees – Wikipedia
2. Red-Black Trees – Wikipedia
3. (i) 2-3 Trees. Insertion & deletion links are given in this page.
(ii) There is one more paper whose ideology of 2-3 tree is little different from the one mentioned above. Here it is. I would like to mention that the 3.(ii) ideology is taught generally. But we(IIT Kharagpur & IISc Bangalore) were taught the 3.(i) one.

There is one very good Java Applet where you can insert, delete, search etc. an element in different kinds of trees STEP-BY-STEP. Here’s the link:
Search Tree Java Applet
Enable Java in your browser to view it.

Programmers aren’t always the very rational beings they please themselves to believe. Very often, we close our eyes and take decisions based on what we think we know, and based on what have been told by more or less reliable sources. Such as, for example, taking red-black trees rather than AVL trees because they are faster, while not being able to quite justify in the details why it must be so. Programming using this kind of decision I call cargo cult programming.

Originally, I wanted to talk about red-black vs. AVL trees and how they compare, but I’ll rather talk about the STL std::map that is implemented using red-black trees with G++ 4.2, and std::unordered_map, a hash-table based container introduced in TR1.

TR1 std::unordered_map is a map that does not maintain any particular order between the keys of the data it contains. It is therefore implemented as a hash table. The std::map is implemented, at least in G++ 4.2, as a red-black tree. Self-balancing trees change their shape with each insertion in order to maintain most leaves at an equal depth, ensuring an access time. The difference between an AVL tree and a red-black tree is the way the tree is rebalanced, and the red-black tree does about half as much operations as an AVL tree on insertion, making somewhat faster—how much faster remains to be quantified.

So I ran a few simple experiments to compare std::unordered_map and std::map. First, I got hold of the Zingarelli word list, containing some 585,000 Italian words (I can’t find a working URL, but I got the list a few years ago on a Scrabble-related page). I ran three tests: insertion, successful search, and failed search. The list is broken into two parts, one containing 90% of the words, to be inserted, and 10% of the words where kept for the unsuccessful search test.

The insertion test consisted into inserting all of Zingarelli’s list into both data structures, in the most simplistic way: scanning the list sequentially and inserting items one by one.

Insertion times are about the same; so we cannot conclude anything special here except maybe that the insertion time is largely dominated by memory allocation and copy. For the successful search (10,000 tries in each case):

Map look-up is immensely faster with std::unordered_map, a ratio of about 8:1! Failed searches exhibit the same behavior. For ~60,000 failed searches:

We see the same kind of differences here again, but the failed searches are much slower than the successful searches.

When we repeat the experiment with integers (with the same kind of numbers; 500,000 integers of which 10% are randomly chosen for the failed searches), we get essentially the same picture but a massive speedup as strings are rather costly to copy. Indeed, for the insertion:

Now, we see the algorithmic difference between the two algorithms as the time spent allocating and copying strings is eliminated. For successful and failed searches, we get (again for 10,000 look-ups):

and:

The std::unordered_map therefore seems to be much faster than std::map. What have we lost in order to gain this speed? First, a lot of memory as hash table must retain a certain sparseness to sport their average constant-time look-ups; and the ordering of keys. Listing the items sorted in a hash map won’t produce an ordered list, but rather a randomized version of the list. std::map on the other hand, allow lexicographic enumeration of its contents.

*
* *

So you’re reading this and are thinking, mmppfh, of course, it’s a hash table, you nitwit. Well, yes. Maybe so.

But here’s what prompted me to do the test. Red-black trees offer access. But that’s assuming (and quite wrongly) that comparing keys can be performed in constant time. This may be true for simple keys, such as machine-size integers (known in C and C++ as int) but not so for more complex data, like strings and other structures. For string, for example, comparison may still be very fast because the cost of comparing two string only depends on the longest common prefix; if two long strings have differences in the first few characters, comparison terminates rapidly and the cost is moderate. If, on the other hand, the string share a very long prefix, then the comparison algorithm must scan both strings until the end is reached or a difference is found; this can be very long. So, lets be the average common prefix length. The expected search time is now which can grow large if is large.

For a hash table look-up, you must first compute the hash key which can be at best done in linear time in the average key length. In our first case, this length is the average string length. Let this average length be . The number of probe is a small constant that depends on the sparseness of the table and the number of items, , it contains. For each of those probes, a string comparison is performed, leading to an expected complexity of .

Now, it is not clear when . Both cancel, which boils down to . So, in some conditions, it may be possible to make the tree appear faster than the hash table.

On the average, however, the hash map should win becase we expect .

*
* *

A cargo cult is described as:

A cargo cult is a type of religious practice that may appear in primitive tribal societies in the wake of interaction with technologically advanced, non-native cultures. The cults are focused on obtaining the material wealth of the advanced culture through magical thinking, religious rituals and practices, believing that the wealth was intended for them by their deities and ancestors.

Except from the part with ancestor spirits but especially when considering magical thinking, cargo culting applies very often to how programmers write code and take decisions about data structures and algorithms. Choosing a red-black tree over an AVL, or over a splay tree, because we think that it is somehow always better—sometimes because Ancestor X (a more or less reliable source or authority, such as a more experience programmer, a teacher, or some Internet dude) said so—is a form of cargo cult where the programmer does not use rationality to its full extent to make a decision.

When choosing data structures, one must be fully aware of the dual cost of data structures. The first cost is the run-time cost. Theoretical complexity and actual implementation-dependant run-times may be quite different. The second cost is memory usage. Memory is large on modern systems but not infinite. A particularly wasteful method that offers constant-time access to the data may use as much as, say, ten times the memory used by a method that gives you access. If for small data set this 10× memory usage pose no particular problem, it may be quite different with a largish data set. It’s all a very delicate balancing act between run-time performance and scalability.

It’s very hard to trust one’s gut feelings about a data structure and the data put in. Combined with the access patterns, the data structure may yield a very counter-intuitive performance. Rather than giving into magic thinking cargo cult programming, you should always take a little time to validate our assumptions and hypotheses about the data, the data structure, and the access patterns, as data structure behavior is clearly not independent from the data and the access patterns.

Consider this very simple example. We have a simple binary tree and a list of strings. Binary trees offer insertion and access times, so it is, a priori, a good choice. Now, the list of string is composed of words, but it so happens it’s already sorted. If we insert the strings in the order they are in the list, we get a degenerate binary-tree that’s in fact a list! Indeed so: insertions are always performed at the far right of the tree, causing the tree to degenerate into something we could call a vine and then all operations degenerate to linear time. If we randomize the list and insert the words into the tree in that randomized order, we get a ragged tree, but about equally deep everywhere, leading to the expected access and insertion time.

The sorted list / binary tree example is an over-simplistic one, I admit, but it gets to the point: the programmer used a data structure cargo-culted to offer access time, but due to his incomplete comprehension of the data (the sorted list) and of the access pattern (inserting items sequentially from the list), the result was disastrous.

But the thing is, we all do that to a certain extent!

Gli alberi AVL sono degli alberi binari bilanciati in altezza. Un albero binario si dice bilanciato in altezza se, per ciascun nodo dell’albero, l’altezza del sottoalbero sinistro differisce dall’altezza del sottoalbero destro al più di una unità.

Per gli alberi AVL si parla anche di fattore di sbilanciamento di un nodo e lo si definisce come la differenza tra l’altezza del sottoalbero sinistro e l’altezza del sottoalbero destro. Banalmente, in un albero AVL il fattore di sbilanciamento di ciascun nodo è, in valore assoluto, .

A differenza di un albero binario “tradizionale”, un albero AVL mantiene la proprietà di bilanciamento in qualsiasi circostanza, sia dopo un inserimento di un nuovo valore che dopo la cancellazione.

Si può dimostrare che l’altezza di un albero AVL di n nodi è sempre e quindi tutte le operazioni di gestione dell’albero (ricerca, inserimento e cancellazione) hanno costo .

Per mantenere il bilanciamento dell’albero in seguito a inserimenti o cancellazioni si devono effettuare delle particolari rotazioni, semplici o composte, che permettono di riequilibrare l’albero.

Abbiamo 4 tipi principali di rotazioni, che vengono applicate in base alla tipologia di sbilanciamento che si vuole corregere. Supponiamo di chiamare v il nodo critico, ovvero il nodo più profondo dell’albero che ha un fattore di sbilanciamento in modulo uguale a 2; possiamo trovarci in 4 casi distinti, che necessitano di rotazioni differenti:

• Rotazione SS: si verifica quando a determinare lo sbilanciamento è il figlio sinistro del sottoalbero sinistro del nodo v
• Rotazione DD: si verifica quando a determinare lo sbilanciamento è il figlio destro del sottoalbero destro del nodo v
• Rotazione SD: si verifica quando a determianre lo sbilanciamento è il figlio destro del sottoalbero sinistro del nodo v
• Rotazione DS: si verifica quando a determianre lo sbilanciamento è il figlio sinistro del sottoalbero destro del nodo v

I 4 casi sono simmetrici a coppie (SS con DD, SD con DS).

Nel caso sia necessaria una rotazione SS, per ribilanciare il nodo v occorrerà applicare una rotazione semplice verso destra su v.

Nel caso sia necessaria una rotazione SD, per ribilanciare il nodo v occorrerà una doppia rotazione: sia z il figlio sinistro di v; l’abero T che sbilancia v è il sottoalbero destro di z, radicato in w. La rotazione SD consiste in una rotazione semplice verso sinistra con perno in z seguita da una rotazione di base verso destra con perno in v.

La precedente immagine mostra un’esempio di rotazione semplice verso destra, imperniata sul nodo di etichetta b; la rotazione verso sinistra è perfettamente simmetrica.

Di seguito troverete il codice C per eseguire le 4 rotazioni.

struct node{
    void *data;
    size_t data_size;
    //h è l'ltezza del nodo
    int h;
    struct node *left;
    struct node *right;
    struct node *father;
};

static struct node* ss( struct node* root ){
    printf("Rotazione ss\n");
    struct node* x;

    //effettuiamo la rotazione
    x = root->left;
    root->left = x->right;
    x->right = root;

    //aggiorniamo le altezze dei nodi coinvolti
    root->h = MAX( HEIGHT( root->left ), HEIGHT( root->right ) ) + 1;
    x->h = MAX( HEIGHT( x->left ), x->h ) + 1;

    x->father=x->right->father;
    if(x->father){
        if(x->right->father->left==x->right)
            x->right->father->left=x;
        else if(x->right->father->right==x->right)
            x->right->father->right=x;
    }

    x->right->father=x;
    if(x->right->left)
        x->right->left->father=x->right;

    return x;
}

/*
 * La funzione dd effettua una rotazione semplice verso sinistra.
 * La funzione prende in ingresso il nodo perno
 */

static struct node* dd( struct node* root ){
    printf("Rotazione dd\n");
    struct node* x;

    //effettuiamo la rotazione
    x = root->right;
    root->right = x->left;
    x->left = root;

    //aggiorniamo le altezze dei nodi coinvolti
    root->h = MAX( HEIGHT( root->left ), HEIGHT( root->right ) ) + 1;
    x->h = MAX( HEIGHT( x->right ), root->h ) + 1;

    x->father=x->left->father;
    if(x->father){
        if(x->left->father->left==x->left)
            x->left->father->left=x;
        else if(x->left->father->right==x->left)
            x->left->father->right=x;
    }
    x->left->father=x;
    if(x->left->right)
        x->left->right->father=x->left;

    return x;
}

/*
 * La funzione sd effettua una rotazione composta, prima verso sinistra con perno nel figlio sinistro del nodo crtico,
 * poi verso destra con perno nel nodo critico stesso.
 * La funzione prende in ingresso il nodo critico
 */

static struct node* sd( struct node* root ){
    printf("Rotazione sd\n");

    root->left = dd( root->left );

    return ss( root );
}

static struct node* ds( struct node* root ){
    printf("Rotazione sd\n");

    root->right = ss( root->right );

    return dd( root );
}

Le 4 funzioni prendono in ingresso il “nodo critico”, ovvero il più profondo nodo con fattore di sbilanciamento in modulo uguale a 2. Le funzioni mantengono pure i “rapporti di paternità” tra i nodi e aggiornano l’altezza dei nodi coivolti nella rotazione.La funzione di inserimento è semplice da implementare. Il criterio su cui si basa è lo stesso su cui si basa la funzione di inserimento ricorsivo per un normale albero binario; l’unica aggiunta sta nel controllo dello sbilanciamento dopo l’inserimento.Ecco il codice:

inline struct node* avl_insert(struct node* root, void *data, size_t s,compare_function compare){

    if(!root){

        root=node_alloc(data,s);
        root->father=0;

    }else if(compare(data,root->data)<0){
        root->left=avl_insert(root->left,data,s,compare);
        root->left->father=root;

        /*
         * Dopo l'inserimento del nuovo nodo controlliamo se si verifica uno sbilanciamento.
         * Se si, controlliamo le caratteristiche dello sbilanciamento per decidere che tipo di rotazione effettuare
         */
        if((HEIGHT(root->left) - HEIGHT(root->right)) == 2){

            if(compare(data,root->left->data)<0){

                //effettuaiamo una rotazione ss
                root=ss(root);

            }else{
                //rotazione sd
                root=sd(root);

            }
        }
    }else if(compare(data,root->data)>0){
        root->right=avl_insert(root->right,data,s,compare);
        root->right->father=root;

        //Stesse considerazioni fatte sopra
        if((HEIGHT(root->right) - HEIGHT(root->left))==2){

            if(compare(data,root->right->data)>0){
                root=dd(root);

            }else{
                root=ds(root);

            }
        }
    }

    //Aggiorniamo l'altezza del nodo dopo l'inserimento
    root->h=MAX(HEIGHT(root->left),HEIGHT(root->right))+1;

    return root;
}

Ho implementato pure la funzione di cancellazione. Il codice di quest’ultima funzione e di tutte le altre lo potrete scaricare dal mio repository a questo link.Occorre sottolineare che, mentre nel caso degli inserimenti al più si effettua una sola rotazione, nel caso della cancellazione di un nodo si posson fare rotazioni consecutive.Per approfondimenti, cliccate qui.