[Click here for a PDF of this post with nicer formatting]

Motivation.

In my recent post about memory barriers and atomics(or the PDF version here), I stated a number of times that x86 (or x86-64) was among the platforms that provided strongly ordered memory access.

I have an old friend, we’ll call him Joe here, who uses a Dekker’s algorithm program to demonstrate to his computer architecture class that this is in fact not the case. This flies in the face of all my experience with DB2, so it becomes immediately interesting from both a theoretical point of view and a correctness issue for the mutual exclusion code that I maintain for our product, much of which I have written or rewritten myself over the years.

When talking to Joe, I had stated that I thought the MFENCE, LFENCE, and SFENCE instructions were only required for SSE. I said the Intel memory model already provided an ordering guarantee, and that no fence instructions were required. My mistaken recollection was that these were effectively no-ops for non-SSE code and special memory configurations used in privileged mode.

The reasons for my misunderstanding is explored here, including why DB2 mutex implementation does not require these fences. I knew this once back in the days when the P6 was new, but I had forgotten all this processor-ordering business, demonstrated by an implementation of the Dekker algorithm on a recent intel multiprocessor machine.

The fact that DB2 mutex and atomic code does not use memory fence instructions can likely be extrapolated to any other intel targeting mutex implementation regardless of the products or library that providing the mutex. We all are restricted to the same tricks and toolbox, the Intel instruction set specification itself, and we will see that the easiest and fastest way to implement a mutex does not require these fence instructions and why.

Dekker’s algorithm uses a pair of flags and a turn variable to control access to a critical section. While this algorithm is of historical interest, and the subject of classroom discussions, I doubt that anybody uses it for anything real. There are a number of flaws, the most obvious of which is that it allows only the synchronization of a pair of threads. A less obvious limitation is that it assumes strongly ordered memory access. These strongly ordered memory access assumptions can be subdivided into two cases. The first of these ordering assumptions is that we require acquire and release fences to ensure the data guarded by the critical section is not accessed outside of the critical section. The second of the ordering assumptions is the trickier part, since there are memory ordering access assumptions in the algorithm itself. I would expect that these have been studied in depth by others, but even without such study, running the code with and without a few types of barrier instructions demonstrates conclusively that the algorithm is flawed as is, at least on intel hardware.

Trolling the internet I found references stating that Dekker’s algorithm can be faster than mutex methods build using the more modern hardware specific atomic instructions. Experiment shows that this is not necessarily true. To demonstrate this requires some low level knowledge. One cannot compare Dekker’s to pre-canned blackbox library methods like pthread_mutex_lock, where the implementation details are unknown. With Dekker’s algorithm doing brute force spinning, a good comparison requires that the mutex implementation also do so. Results comparing the Dekker’s method and some alternatives will be supplied below. These are of general interest since the question of how expensive is an atomic compared to a mutex occurs frequently (at least in DB2 development).

What the intel documentation says about ordering.

The intel architecture documentation and instruction set reference do detail memory fence instructions. There are actually three such instructions in current CPUs, very much like powerpc, which is intrinsically weakly ordered, and (except for the Power6 blip) always has been. Intel provides a bidirectional fence instruction MFENCE, an acquire fence LFENCE, and a release fence SFENCE. The MFENCE instruction dates way back to the P6 where intels first weakened their memory ordering (defining their unique processor ordering).

Here is what the instruction set reference says about these fence instructions

• SFENCE [1]

Performs a serializing operation on all store-to-memory instructions that were issued prior the SFENCE instruction. This serializing operation guarantees that every store instruction that precedes in program order the SFENCE instruction is globally visible before any store instruction that follows the SFENCE instruction is globally visible. The SFENCE instruction is ordered with respect store instructions, other SFENCE instructions, any MFENCE instructions, and any serializing instructions (such as the CPUID instruction). It is not ordered with respect to load instructions or the LFENCE instruction.

• LFENCE

Performs a serializing operation on all load-from-memory instructions that were issued prior the LFENCE instruction. This serializing operation guarantees that every load instruction that precedes in program order the LFENCE instruction is globally visible before any load instruction that follows the LFENCE instruction is globally visible. The LFENCE instruction is ordered with respect to load instructions, other LFENCE instructions, any MFENCE instructions, and any serializing instructions (such as the CPUID instruction). It is not ordered with respect to store instructions or the SFENCE instruction.

• MFENCE

Performs a serializing operation on all load-from-memory and store-to-memory instructions that were issued prior the MFENCE instruction. This serializing operation guarantees that every load and store instruction that precedes in program order the MFENCE instruction is globally visible before any load or store instruction that follows the MFENCE instruction is globally visible. The MFENCE instruction is ordered with respect to all load and store instructions, other MFENCE instructions, any SFENCE and LFENCE instructions, and any serializing instructions (such as the CPUID instruction).

A condensed and somewhat more human friendly version of these statements can be found in the system programming guide.

The functions of these instructions are as follows:

• SFENCE

Serializes all store (write) operations that occurred prior to the SFENCE instruction in the program instruction stream, but does not affect load operations.

• LFENCE

Serializes all load (read) operations that occurred prior to the LFENCE instruction in the program instruction stream, but does not affect store operations.

• MFENCE

Serializes all store and load operations that occurred prior to the MFENCE instruction in the program instruction stream.

Now, we do not use MFENCE anywhere in DB2 code. It was once tried with the hope of trying to fix some code that had a mysterious bug, but it did not help. We replaced the problematic code with simpler more trustworthy stuff. Reading the intel docs where there is no immediate sign that one would not have to use fence instructions for mutual exclusion code, one may think we avoid MFENCE since we prereq later hardware that supplies the LFENCE and SFENCE instructions and we use those instead. While the prereq statement is now probably true, but that is not why we do not use MFENCE. I will return to this later. For now lets instead explore some code.

The Dekker algorithm, implementation, and a non-correctness test case.

The Wikipedia Dekker’s algorithm page describes the algorithm. An implementation of the code described in that article, pretty much line for line except for some comment addition, is the following

struct FLAG_STATE
{
   volatile int flag ;

   char dummy[120] ; // cacheline separator.
} ;

FLAG_STATE g[2] ;
volatile int turn = 0 ;

void my_lock( int tid )
{
   int other = 1 - tid  ;

   g[tid].flag = true ; // I want in the CS.

   FENCE ;

   while ( true == g[other].flag )
   {
      //
      // The other guy is in the CS or wants in.
      //
      if ( turn != tid )
      {
         // he said he wants it, but hasn't given me my turn yet.
         //
         // let him have a chance.
         //
         g[tid].flag = false ;

         // wait till he gives me my turn
         while ( turn != tid )
         {
         }

         // he's about to give up his turn or has, so I can say I want it again.
         //
         // I'll now spin on his flag changing (though I could miss it if
         // unlucky and have to start all over if he tries again fast).
         //
         g[tid].flag = true ;
      }
   }
}

void my_unlock( int tid )
{
   int other = 1 - tid  ;

   turn = other ;

   g[tid].flag = false ;
}

This is called with tid values of 0 or 1 depending on which of the two contenders are trying to get into the mutual exclusion battle. Only two such contenders are allowed.

This algorithm is costly. Compared to a serially executing the increment instructions without locking or contention this took 76 times more time on the system I tried it on. This isn’t exactly a fair comparision since executing the dekker code without contention would also be faster.

Observe that there is no release barrier and the beginning of my_unlock and no acquire barrier at the end of my_lock, so one would expect this to fail on a weakly ordered platform like PowerPC, even if the algorithm itself guarantees mutual exclusion, and it is implemented properly. Failure here means allowing access of the data protected by the mutex outside of the mutual exclusion region.

If one presumes that the compiler has dumbly emitted the code in program order, and if one also presumes that this algorithm and the Wikipedia description of it is sound, and that it was implemented correctly, then one would expect this to work on a platform with strong memory ordering. On a platform with weak ordering acquire and release fences should be required for correctness, and possibly more.

Does this code work? This code was compiled without optimization, hoping that the compiler will not reorder anything, but I did not check the assembly listings to be sure. Unoptimized code is usually spectularily bad, dumb and straightforward, so it is not entirely unreasonable to presume that the instructions are all generated in program order. Of a set of three runs with FENCE=MFENCE,SFENCE,LFENCE,NOFENCE where two threads looped incrementing an integer variable ten million times each, this code failed the consistency checking with LFENCE and NOFENCE. With SFENCE this code ran correctly twice, but in one of the iterations produced a count 7 less than the required target value for a mutex protected counter incremented a well known number of times. This does not demonstrate that this algorithm is correct with FENCE=MFENCE, but it is decisively busted otherwise.

Now, Joe had a different implementation of Dekker’s that did not do the turn hand off. His looked like so (after tweaking it a bit to make it look like mine for consistency)

void my_lock( int tid )
{
   g[tid].flag = 1 ;
   turn = 1-tid ;

   FENCE ;

   while( g[1-tid].flag )
   {
      if ( turn == tid )
      {
         break ;
      }
   }
}

void my_unlock( int tid )
{
   g[tid].flag = 0 ;
}

This code was also run with FENCE=MFENCE,SFENCE,LFENCE,NOFENCE. It passes with MFENCE and SFENCE, and fails with LFENCE and NOFENCE like the implementation of the Wikipedia listing.

Does this mean that his code is correct with SFENCE or even correct with MFENCE or the implementation of the wikipedia listing is correct with MFENCE? That is harder to say. The correctness of either set of code even with FENCE=MFENCE is not prove that the code is correct by a few successful test runs. What is interesting about all this is that Joe’s assertion that memory ordering is weakened enough to require fences for correctness now seems very plausible.

Note that this code is also not cheap. It is better than the listing originating with the vanilla wikipedia article, but was also clocked at 47 to 67 times slower than uncontended serial access. Again not entirely a fair comparison since this code would also do better serially.

We still have to understand why this is true here and not for atomic instruction based mutual exclusion. Additionally in many years of running DB2 on all varieties of intel and amd64 hardware we haven’t seen signs of requiring fence instructions for our non-mutex atomic code. That doesn’t mean we do not have such problems, since it could be very hard to identify this sort of error. Keeping suspense high, the reasons for this a bit more, and instead show what is required to implement a mutex a more natural way on intel.

Using an lock xchg mutex

Here is a dumb busy spinlock implementation on intel using the GCC compiler

volatile char lock_word = 0 ;

void my_lock( int tid_ignored )
{
   char old_value ;
   char new_value = 1 ;

   do {
      __asm__ __volatile__( "lock; xchgb  %0,%1\n\t" :
                            "=r" (old_value), "+m" (lock_word) :
                            "0" (new_value) :
                            "cc" ) ;

   } while ( 1 == old_value ) ;
}

void my_unlock( int tid_ignored )
{
   lock_word = 0 ;
}

Compared to unlocked serial uncontended increment, using this as a replacement for the Dekker mutex (ie. variations of that with enough fencing inserted to make the code appear functional), this runs about 16 times slower. This is already much better than the Dekker code (either the wikipedia based version or Joe’s), but it would again be interesting to check how much worse that gets without contention. How much of this bus locked atomic exchange cost is aggrivated by the contention itself.

Using an lock xadd for increment

For this simple task of doing thread safe increment, we can do much better easily. Here’s some code to do the increment without using a mutex

volatile unsigned foo = 0 ;

inline void my_increment()
{
   unsigned old_value ;
   unsigned add_value = 1 ;

   __asm__ __volatile__( "lock ; xaddl %0,%1\n\t" :
                         "=r"(old_value), "+m" (foo) :
                         "0" (add_value) :
                         "cc" ) ;
}

void my_lock( int tid_ignored )
{
}

void my_unlock( int tid_ignored )
{
}

Executed serially a loop of such increments was measured at about 3 times slower than the plain old increment. That baseline timing actually used this code but with the bus locking signal flag removed. It costs us to do the bus locking even if we do not need it. Adding contention into the mix, the loop of these increments now clocks in at about 7-9 times slower than the unlocked uncontended code. This provides some rough idea of how much the correct code minimally costs us. All of these times will be summarized below.

Performance comparision.

Here’s a summary of the costs of the various increment methods in order of added cost. This is with a plain old xadd as a baseline (no lock prefix).

• xadd

This was the baseline time.

• lock xadd (ie. atomic add. No fences. No contention).

3x baseline time.

• lock xadd (ie. atomic add. w/ contention)

7-9x baseline time.

• lock xchg (spinlock mutex w/contention)

16x baseline time.

• Joe’s implementation of dekker w/ [sm]fence (mutex w/ contention)

47-67x baseline time.

• my implementation of wikipedia’s dekker algorithm w/ mfence (mutex w/ contention)

76x baseline time.

Why no fences are required for the lock prefixed atomic or spinlock mutex code.

I have held off talking about what makes the lock xchg based code work without requiring fences. The reason for this can be found in volume 3, and is somewhat anticlimatic

8.2.2 “Reads or writes cannot be reordered with I/O instructions, locked instructions, or serializing instructions.”

The DB2 mutex code (and likely many other similar implementations) is fine as is. Our “atomic” library which also uses lock flagged atomic instructions of various sorts (xadd, cmpxchg, …) also requires no additional or specific barriers as it does on ia64 or ppc. This atomic code MUST also use the lock prefix or else it is demonstratably busted when there is contention, so it imposes a minimum costs of approximately 3x in order to be safe when there is contention.

Conclusions.

I think that it is fair to say that the processor ordering model of modern intel cpus is demonstratably unordered requiring barriers in some cases. It would be interesting to identify exactly where in the Dekker algorithm there are ordering dependencies. I suspect such a study is already in the literature, although it is of little practical value since bus locked intel instructions are superior.

While some timing differences were presented, these may just provide a ballpark performance comparision. Doing a performance comparision without optimization is of dubious value. It may possible that since only relative times were compared there may be some validity to the comparision otherwise, but that really requires doing it to be sure. It would be interesting to redo at least the xchg vs xadd vs baseline cases with optimization and see the differences.

The weaker aspects of intel memory ordering are hidden from code that exploits thier bus locked instructions for atomic and mutex library implementations. Requiring that any dependence on ordering is explicitly guarded by critical sections bounded by bus locked instructions is the key to correctness. As I write this it occurs to me that since not all of the DB2 mutex types use an atomic store for the lock release we may actually have an issue. That implicitly depends on ordering, so there may actually be a correctness issue to be investigated and this will have to be explored. Yet another reason not to write this sort of code oneself if it can be avoided. It is so easy to get it wrong and finding a demonstration that this one time in 20 million has blown up can be very hard.

References

[1] Intel. Intel architectures software developer’s manuals.htm [online]. 2009. [Online; accessed 4-December-2009]. http://www.intel.com/products/processor/index.htm.

[2] Wikipedia. Dekker’s algorithm — wikipedia, the free encyclopedia [online]. 2009. [Online; accessed 4-December-2009]. http://en.wikipedia.org/w/index.php?title=Dekker%27s_algorithm&oldid=32%9540788.

[Click here for a PDF alternative to this post]

Abstract.

Discussion of problems and solutions associated with correct atomic and memory barrier usage has been a recurrent theme of many cubicle chats within DB2 development, and I will attempt to describe here part of what I’ve learned in some of those chats. As a software developer, not a hardware implementer, I cannot pretend to understand more than a fragment of this topic, but it is enough to be dangerous and perhaps help others be dangerous too.

Introduction

Use of atomic instructions for manipulating “word” (or smaller) size quantities, avoiding the use of mutual exclusion library functions is becoming increasingly easy for developers. This hasn’t made the using atomic instructions or library methods correctly any less difficult or error prone.

Information on how to use these written for an average Joe developer is hard to come by. What can be easily found is good detailed low level information targeted at or written by operating system kernel extension writers or compiler developers [1], or language implementers and designers [2] [3] [4] [5] [6] [7] [8].

The driving reason for most atomic usage is lock avoidance, and a desire for performance drives most use of atomic methods. When atomic operations are used to replace lock protected updates, this can change the semantics of the program in subtle ways that are difficult to understand or get right. In particular, additional use of memory barrier instructions or library methods may be required to correctly replace lock protected updates.

It is assumed here that most use of atomics is done for performance based lock (mutex) avoidance. An attempt to cheat, writing atomic based code that avoids a lock, unfortunately requires some understanding of what that lock provided in the first place. Here the meaning of, requirements for, and implementation of, a lock are discussed. The implementation of an atomic operation is then discussed, and we then move on to an enumeration of the types of memory barrier instructions and their effects. Finally, unless the assembly samples provided have not scared aways the reader, we cover a simple non-mutex example of barrier usage. This example uses an atomic to signal completion of a previous update and requires memory barriers for correctness on weakly ordered systems.

Why do I need locking or atomics?

Why would you want to use an atomic? If you are reading because you want to know a bit about memory barrier usage, you probably already know. However, for illustrative purposes, consider the very simplest example of non-thread-safe updates of an integer in shared memory.

   pShared->x++ ;

Any such operation requires three steps. Read the value, change the value, update the value in memory. On some platforms this is explicit and obvious since the value will have to be read into a register before making the update. Here’s a PowerPC example that illustrates this

    lwz      r4=(sharedMem)(r3,0)    ; r3 == pShared ; (r3,0) == *pShared
    addi     r0=r4,1
    stw      (sharedMem)(r3,0)=r0

The value in the memory address contained in the register r3 (offset by zero bytes) is loaded into a register (r4). One is added to this (into r0), and the value is stored back into the original memory location. Running on a CISC system this may all appear as one instruction, but it still requires the same number of steps internally.

Suppose you had a threaded application where this counter variable is updated routinely by many threads as they complete some routine and commonplace action. What can now happen, interleaved (with time downwards) across threads may now look like this

T0: pShared->x = 0

T1: R1 = pShared->x (R1 = 0)
    R1 = R1 + 1
T1: 

T2: R2 = pShared->x
    R2 = R2 + 1
    pShared->x = R2

T3: R3 = pShared->x (assume T3 "sees" T2's update, a value of 1)
    R3 = R3 + 1
    pShared->x = R3

T4: R4 = pShared->x (assume T4 "sees" T3's update, a value of 2)
    R4 = R4 + 1
    pShared->x = R4

T1:
    pShared->x = R1 (T2 and T3's updates are now clobbered)

We had four updates to the counter after the initialization, but the counter value only goes up by one. Uncontrolled updates like this can oscillate in peculiar fashions, and something that may be expected to be monotonically increase perhaps only trends upwards on average. If the counter is not heavily contended you may even fluke out, running for a long time before getting unlucky with an inconsistent update of some sort.

This example leads us nicely to requirements for locking or atomic methods to protect such updates.

Use of locking to make the read, change update cycle safe.

Mutual exclusion mechanisms go by many names, locks, mutexes, critical sections, and latches to name a few. System libraries usually provide implementations, a common example of which are the Unix pthread_mutex methods. The unsafe increment above can be made thread safe, protecting it with such a guard. With error handling omitted for clarity, a modified example could be as follows

   pthread_mutex_lock( &m ) ;

   pShared->x++ ;

   pthread_mutex_unlock( &m ) ;

If all threads only ever updates this value we can use a similar bit of code to read and do something with the read value

   pthread_mutex_lock( &m ) ;

   v = pShared->x ;

   pthread_mutex_unlock( &m ) ;

   doSomethingWithIt( v ) ;

No two sequential samplings of the shared variable x would see the value go down was possible in the uncontrolled update case.

Now, you may ask why a mutex is required for the read. If that is just a single operation, why would it matter? That’s a good question, but not easy to answer portably. If your variable is appropriately aligned and of an appropriate size (practically speaking this means less than or equal to the native register size) and you aren’t compiling with funky compiler options that turn your accesses into byte accesses (such options do exist), then you may be able to do just that. However, if that variable, for example, is a 64-bit integer and you are running on a 32-bit platform, then this read may take two instructions and you have a risk of reading the two halves at different points in time. Similarly, suppose you were doing a 32-bit integer read, but that 32-bit integer was aligned improperly on a 16-bit boundary (on a platform that allows unaligned access), then your apparent single read may internally require multiple memory accesses (perhaps on different cache lines). This could cause the same sort of split read scenario.

You probably also need to declare your variable volatile and also have to be prepared to deal with a few other subtleties if lock avoidance on read is going to be attempted. By the end of these notes some of those subtleties will have been touched on.

To make the story short, it should be assumed that if you want portable correct results you need to take and release the mutex for both read and write.

What does correctness cost me?

Now, having corrected the increment code, what does that cost us. Timing the following very simple single threaded program with an without the mutex code

#include 

int x = 0 ;
pthread_mutex_t m = PTHREAD_MUTEX_INITIALIZER ;

int inc() ;

int main()
{
   for ( int i = 0 ; i < 1000000000 ; i++ )
   {
      inc() ;
   }
}

int inc()
{
#if defined USE_THE_MUTEX
   pthread_mutex_lock( &m ) ;
#endif

   x++ ;

   int v = x ;

#if defined USE_THE_MUTEX
   pthread_mutex_unlock( &m ) ;
#endif

   return x ;
}

I find that this billion sets of call a function, increment a variable and return it without the mutex takes about 2.5 seconds. With the mutex calls enabled, the total elapsed time required goes up to 8.5 seconds. It costs over three times the time to do it with a mutex, and that’s without any contention on the mutex whatsoever! With threads all fighting over the mutex and blocking on kernel resources when they cannot get it, this is as good as it gets. Correctness doesn’t come cheaply.

How does the lock work?

Any mutex implementation requires at least one platform specific method of performing a read, update, change cycle as if it is done without interruption. Your hardware will likely provide a Compare and swap or Load Link/Store Conditional (which can be used to construct higher level atomics like compare and swap). Implementing a mutex is intrinsically unportable territory. Older intel cpus did not provide a compare and swap, but one did have a bus locked exchange available (LOCK XCHG).

HP’s PARSIC, also easily argued to not be a modern architecture, only provides an atomic load and clear word instruction, also requiring 16 BYTE a aligned word. What can be assumed is that any multiple cpu machine provides some sort of low level atomic instruction suitable for building a mutual exclusion interface. If one were to imagine what a pthread_mutex_lock protected increment expands to, it would likely have the following form

   T valueRead ;

   while ( ATOMIC_OP_WAS_SUCCESSFUL != SOME_INSTRUCTIONS_FOR_LOCK_ACQUISION( &m ) )
   {
      // instructions (like PAUSE()) for hyper threaded cpus.

      // code for wasting a bit of time hoping to get the lock
      //   (when not running on a uniprocessor).

      // sleep or block using an OS primitive
   }

   valueRead = pShared->myLockProtectedData ;
   pShared->myLockProtectedData = valueRead + 1 ;

   SOME_INSTRUCTIONS_FOR_LOCK_RELEASE( &m ) ;

Even before talking about the hardware, one has to assume that there are mechanisms available to the library writer that prevent the generated code from being reordered. Any instructions containing accesses of myLockProtectedData must NOT occur BEFORE the ACQUISION instructions, and must NOT occur AFTER the RELEASE instructions. If the compiler were to generate code that had the following effect

   T valueRead = pShared->myLockProtectedData ;
   pShared->myLockProtectedData++ ;

   pthread_mutex_lock( &m ) ;
   pthread_mutex_unlock( &m ) ;

Things would obviously be busted. In the case of the pthread functions we have non-inline method with unknown side effects requiring a call to an external shared library. This appears to be enough that the lock protected data does not have to be declared volatile and can be used while the mutex is held as if in a serial programming context. Understanding the murky language rules that give us the desired thread safe behavior is difficult (for an average Joe programmer like me). I came to the conclusion this is not a completely well defined area, motivating a lot of the recent C++ standards work into memory consistency and threaded behavior.

Presuming one assumes that the compiler is laying down the instructions for this code in program order (or something sufficiently close), or that there are mechanism available to ensure this, is this enough to guarantee that we have correct program behavior?

It may come as a rude surprise that, depending on the instructions used to acquire and release the mutex, having the all the instructions scheduled in program order is NOT actually sufficient. Not all hardware executes instructions in the order the compiler specifies. We also need mechanisms to prevent the hardware from starting a memory access too early or letting a memory access complete too late.

This requirement for memory ordering instructions is really the whole point of this discussion. We must have additional mechanism on top of the raw atomic instructions that only ensure read,change,update accesses on an isolated piece of memory (the lock word) can be made as if uninterpretable.

For illustrative purposes, suppose that one was using the gcc Atomic Builtins to attempt to implement a mutex. One (generally) wrong way to use these would be

   // the mutex data
   volatile int m = 0 ;
   #define HELD_BIT 1
   #define WAITER_BIT 2

   // Get the mutex.
   while ( true )
   {
      int prev = __sync_fetch_and_or( &m, HELD_BIT ) ;

      if ( 0 == ( prev & HELD_BIT ) )
      {
         break ;
      }
      // sleep and other stuff, possibly blocking and setting WAITER_BIT.
   }

   // Got the mutex, access the protected data.
   pShared->myLockProtectedData++ ;

   // Release the mutex.
   int preLockState = __sync_fetch_and_and( &m, ~HELD_BIT ) ;
   assert( preLockState & HELD_BIT ) ;
   if ( preLockState & WAITER_BIT )
   {
      WakeUpAnyWaiters() ;
   }

Reading the GCC docs one sees that this has desired compiler behaviour for instruction scheduling. Namely, “these builtins are considered a full barrier. That is, no memory operand will be moved across the operation, either forward or backward”. The compiler will not move instructions that access memory and move them into or out of the critical section bounded by the pair of atomic instructions.

Will the hardware retain that ordering? NO, not necessarily.

These compiler builtins are partially patterned after ones provided by intel, and intel designed them to match functionality provided by their ia64 architecture, a weakly ordered instruction set. While ia64 was effectively killed by AMD64 and Windows (now living on only in the guise of HP’s IPF systems), there are other weakly ordered instruction sets predating ia64. Some PowerPC chips that you will find on big AIX systems, or in older macs, or in your childrens ps3 will be weakly ordered. The Sparc architecture allows for weak ordering, but the chips you find in sun machines implement their TSO (total store order) model which is mostly ordered.

This allowance for unordered memory is why, way down at the end of the GCC page, one finds a couple special _lock methods. The point of these is to ensure that the hardware does not reorder the relative order of memory accesses of the lock word and the lock data, even when the compiler lays down the code in “program order”.

We can do a mutex implementation less wrongly using a memory barrier for release as in the following fragment.

   // the mutex data
   volatile int m = 0 ;
   #define HELD_BIT 1

   // Get the mutex.
   while ( true )
   {
      int prev = __sync_fetch_and_or( &m, HELD_BIT ) ;

      if ( 0 == ( prev & HELD_BIT ) )
      {
         break ;
      }
      // sleep and other stuff if you didn't get it
   }

   // Got the mutex, access the protected data.
   pShared->myLockProtectedData++ ;

   __sync_syncronize() ; 

   m = 0 ;

Observe that the use of the word barrier is overloaded on the GCC page, with some usage associated with compiler instruction scheduling, and other usage referring to memory ordering. Their syncronize function appears to be both. Looking at the generated assembly for a weakly ordered system would verify this interpretation, and one would likely find an lwsync or sync instruction on PowerPC for example. This modification of the locking code ensures the compiler lays down the code with the clearing of the lock word after the protected data access, and the hardware memory barrier instruction should ensure that reads or writes to the protected data are complete before the store to the lock word is allowed to occur. Depending on the type of instruction that is emitted for the synchronize builtin, this may not actually prevent loads and stores that occur after the lock release from being started before the lock is released. Generally if that must be prevented a full barrier is required, and it would not surprise me if most implementations of library methods like pthread_mutex_unlock do not prevent memory accesses that occur after the lock release from being initiated before the lock release occurs. This sort of subtlety is not likely to be found in documentation for the mutex library functions.

With the sample code above we are still left with the possibility that the hardware will execute the memory access instructions for the lock protected data before the lock is acquired, making the lock useless. We can fix this with the _lock test and set and release builtins, and also remove the likely overzealous full memory barrier that synchronize provides (allowing post lock release memory access into the critical section). Sample code for this would may look like

   // the mutex data
   volatile int m = 0 ;
   #define HELD_VALUE 1

   // Get the mutex.
   while ( true )
   {
      int wasHeld = __sync_lock_test_and_set( &m, HELD_VALUE ) ;

      if ( !wasHeld )
      {
         break ;
      }
      // sleep and other stuff if you didn't get it
   }

   // Got the mutex, access the protected data.
   pShared->myLockProtectedData++ ;

   __sync_lock_release( &m ) ;

This now implements functional mutual exclusion code, even on weakly ordered systems. The compiler has provided methods that ensure it does not move the lock protected data accesses out of the critical section, we are using atomic instructions that guarantee other threads cannot also be modifying the data (presuming they all use the same mutex), and also have instructions that ensure the hardware does not inappropriately reorder the memory accesses. Inappropriate reordering means that accesses to the lock protected data remain after the lock acquisition and before the lock release. It may not mean that memory accesses from before the lock acquisition are done by the time the lock is acquired nor that memory accesses that occur after the lock is released occur after the lock is released (the compiler says it does that, but the hardware may be a sneaky bastard behind your back).

So, that’s a mutex. We can use something like this to avoid some of the excessive cost observed when using the pthread library mutex functions, and would expect that the uncontended cost of the back to back increment code could be lessened.

Presuming you do not mind the unportability of such code, and manage to get it right, and want to live with maintaining your own mutex implementation, if you do a poor job dealing with contention, you may very well get worse performance than the system provided methods in the contended codepath.

You really really have to want the performance to go down the path of implementing your own mutual exclusion code. DB2 is such a product. DB2 is also a product where first born sacrifices in the name of performance are not considered abhorrent. In our defense our mutex implementation predates the availability of widely available library methods. We had SMP exploitation before pthreads existed using mutiple processes and shared memory. Our own implementation also has the advantage of providing framework to build problem determination functionality that cannot be found in generic library implementations. Performance driven coding is not the only reason we do it ourself even in this more modern age where library methods do exist.

What is an atomic, and how does an atomic method work?

At the core of every atomic library method or compiler intrinsic is one or more special purpose instructions. In many cases these are the same types of instructions that can be used to implement mutual exclusion code (when properly augmented by memory barrier instructions).

If you don’t have an instruction for exactly what you want, it can often be built from some other lower level operation. Suppose for example you want to safely do the contended increment without using a lock, but only have a compare and swap. Something like the following can be used

int fetch_and_add( volatile int *ptr, int value )
{
   int v = *ptr ;
   while ( true )
   {
      int prev = __sync_val_compare_and_swap( ptr, v, v + value ) ;
      if ( prev == v )
      {
         break ;
      }

      v = prev ;
   }

   return prev ;
}

Generally, one can assume at least a compare and swap operation, but this may also be simulated, built up out of more direct and primitive operations. On PowerPC for example, this is done with the very flexible load-linked/store-condition instructions (load with reservation and store conditional in PowerPC lingo). A fetch and add can be built with a small loop

fadd32_spin:
   lwarx     r0,0,r3       ; load word and acquire a registration
   add       r0,r0,r4      ; add in the desired amount to the possibly current value
   stwcx.    r0,0,r3       ; try to store it
   bne-      fadd32_spin   ; if loaded value old or somebody beat us repeat.
   sub       r3,r0,r4      ; fake a return using normal function return register r3

Building a portable dependence on a load-linked/store-condition primitive can be tricky, but possible. In DB2’s reader-writer mutex implementation, a data structure employing some nifty lock free algorithms ([9] [10] [11] [12] [13] [14]) we use this load-linked/store-conditional code to implement a slick compare-against-bitmask-and-increment-or-set-bit atomic operation. We can do the same thing with compare and swap, but the generated assembly code is particular pretty on AIX, where PowerPC gives us this more powerful primitive.

A historical reflection aside.

Even only a couple of years ago, it was much harder to exploit atomic instructions. Part of the reason is that we have tossed or stopped caring about older systems. Nobody cares anymore about older ia32 systems that did not have cmpxchg. Pre-PowerPC AIX implementations that did not have lwarx and stwcx. instructions are now dead and buried in landfill somewhere. PARISC systems that did not provide any sort of reasonable atomic instructions are now being replaced by HP IPF systems. We can now generally assume that some basic atomic building blocks are available and no longer have to tiptoe around avoiding breaking the old guys.

Another reason that makes atomic usage easier these days is the wide availability of compiler builtins and library infrastructure, rendering this previous area of horrendous unportability accessible. Use of the GCC builtins was illustrated above, showing some incorrect and correct lock implementations. On the same page can be found instructions for compare and swap, atomic read and increment, and others. Similar builtins are available from many other compilers. The boost library implementation of atomic types are a good example. The next revision of the C++ standard will likely include boost-like methods. It appears that C# implements Interchange methods very much like the msdev intrinsics, so this is even becoming a cross language portability area in a limited fashion.

It is fun to illustrate just how murky the portability waters were before the prevalance of compiler intrinsics. Atomic exploitation generally required writing assembly code, and that is never particularly fun. Here is a sample code showing how one would code an “inline assembly” language implementation of fetch and add using the IBM compiler targeting PowerPC

int PowerPcFetchAndAdd32Internal ( volatile int * pAtomic, int value ) ;

#pragma mc_func PowerPcFetchAndAdd32Internal { "7c001828" "7c002214" \
   "7c00192d" "40a2fff4" "7c640050" }

#pragma reg_killed_by PowerPcFetchAndAdd32Internal cr0, gr0, gr3

What we have here is literally a mechanism to instruct the compiler to shove the raw instructions, specified by hexadecimal opcodes, directly into the instruction stream, with the writer of the inline assembly routine providing information about all the registers clobbered and side effects of the code. This inline assembly code is actually generated from the listing in PowerPC assembly fragment above, which was carefully constructed to have the same structure as a real function call on AIX (input params in r3,r4, and output in r3)

Before the GCC intrinsics were available, one could use the powerful GCC inline assembly syntax to instruct the compiler what to shove into the code. Here’s an example for fetch and add

#define IA32_XADD32(pAtomic, addValue, oldValue)                    \
__asm__ __volatile__( "lock; xaddl
 /* outputs */     : "=r"(oldValue), /*
                     "+m" (*pAtomic) /*
 /* inputs */      : "0" (addValue)  /*
 /* clobbers */    : "cc"            /* condition registers (ZF) */ \
                       )

Looks kind of like ASCII barf, but does the job nicely enough if you can get it right.

On platforms that provided no good inline assembly methods was not available (which includes those platforms and compilers for which inline assembly makes the code worse than using out of line assembly) one had the option of coding standalone assembler files to create the function call methods desired. That can be just as difficult since one is then forced to learn the assembler syntax in more detail and understand the idiosyncrasies of all the platform calling conventions.

Types of barriers.

There are in general a few different types of barriers (or fences). Not all architectures that allow for unordered instructions necessarily implement all of these. Some of these may also only be available as variations of specific instructions, so when standalone use is required a more expensive instruction may be required.

To describe the barrier types I will borrow ia64 nomenclature and assembly fragments, which is nicely descriptive despite death of this platform. The ia64 architecture provides acquire, release and full barriers. Some of these are built into the load and store operations on these platforms.

Acquire barrier

If one sees assembly code containing

ld4      r3   = [r30]   ; "BEFORE"
st1      [r4] = r40     ; "BEFORE"

ld4.acq  r5   = [r42]   ; LOAD with ACQUIRE barrier

ld4      r6   = [r43]   ; AFTER
st1      [r7] = r44     ; AFTER

One has no guarantee about the relative ordering of the BEFORE-labeled memory accesses. Similarly one has no guarantee about the ordering of the pair of AFTER-labeled memory accesses, but because of the acquire barrier the load and acquire barrier must be complete before the AFTER accesses.

Note that the acquire barrier is one directional. It does not prevent the pair of BEFORE memory accesses from occurring after the barrier. Illustrating by example, the instructions could be executed as if the compiler had generated the following sequence of instructions

ld4.acq  r5   = [r42]

st1      [r7] = r44
ld4      r3   = [r30]
ld4      r6   = [r43]
st1      [r4] = r40

There may be additional architecture rules about dependent loads and stores effecting the types of code motion that is allowed by the compiler, or the way that the CPU implements the architecture, but I will ignore those here.

This sort of effective instruction reordering is often described saying that one instruction has passed another.

This acquire barrier, perhaps unsurprising, is exactly the type of barrier required when implementing code for mutex acquisition. In addition to ia64, the PowerPC platform requires such barriers, and provides the isync instruction to the programmer. An example in a higher level language that may not have the desired effect without an such an acquire barrier is

if ( pSharedMem->atomic.bitTest() ) // use a bit to flag that somethingElse is "ready"
{
   foo( pSharedMem->somethingElse ) ;
}

If correct function of your program depends on somethingElse only being read after the atomic update flagging a condition is complete, then this code is not correct without an isync or other similar platform specific acquire barrier.

Your code should really look something like this

if ( pSharedMem->atomic.bitTestAcquire() )
{
   foo( pSharedMem->somethingElse ) ;
}

Here it is assumed the atomic library provides acquire and release variations of all atomic operations. For PowerPC this could be as simple as adding an isync to the unordered atomic operation, but on a platform like ia64 one would probably use a different instruction completely (ie. cmpxchg.acq for example). With such an acquire barrier the somethingElse access or another other memory operation that follows the load associated with the bitTest necessarily follows the load itself. For example, this can mean that a non-dependent load that has been executed ahead of the acquire barrier will have to be thrown out and reexecuted.

Release barrier

A release barrier is perhaps a bit easier to understand. Illustrating again by example, suppose that we have the following set of load and store instructions, separated by a release barrier.

ld4      r3    = [r30]   ; BEFORE
st1      [r4]  = r40     ; BEFORE

st4.rel  [r42] = r5      ; STORE with release barrier.

ld4      r6    = [r43]   ; "AFTER"
st1      [r7]  = r44     ; "AFTER"

The release barrier ensures that the pair of BEFORE-labeled reads and write memory accesses are complete before the barrier instruction. There is nothing that prevents the BEFORE instructions from executing in an alternate order, but all of the effects of these instructions visible by any other CPU must be complete before the effects of the store barrier instruction is observable by that other CPU. Like the acquire barrier, the release barrier is a one way fence. Neither of the instructions labeled “AFTER” necessarily have to be executed after the release barrier.

On PowerPC this release barrier is implemented with a light weight sync (different than the isync that implements the acquire barrier). As a general rule of thumb, code that requires a release barrier will require an acquire barrier and vice-versa. These instructions are almost always required to be used in complementary pairs.

Similar but opposite to the acquire barrier, a release barrier has the semantics required for implementing the release portion of a mutex.

Full barrier.

We have seen how use of release and acquire barriers can ensure that that memory access before or after the barrier can be forced to maintain that order relative to the barrier. There are however, some types of memory ordering that neither of these can ensure.

Using ia64 code to illustrate, suppose one has memory accesses before a release barrier, and memory accesses after an acquire barrier.

ld4      r3    = [r30]   ; BEFORE
st1      [r4]  = r40     ; BEFORE

st4.rel  [r42] = r5      ; STORE with release barrier.
ld4.acq  r8    = [r45]   ; LOAD with acquire barrier.

ld4      r6    = [r43]   ; AFTER
st1      [r7]  = r44     ; AFTER

The barriers here ensure that the BEFORE-labeled instructions all complete before the release barrier effects become visible to other cpus, and also ensure that none of the AFTER-labeled instructions may start before the acquire barrier instruction is complete. This barrier cannot prevent the acquire barrier LOAD instruction from completing before the STORE barrier. Illustrating by example, the CPU could execute things like so

ld4.acq  r8    = [r45]   ; LOAD with acquire barrier.

ld4      r3    = [r30]   ; BEFORE
st1      [r4]  = r40     ; BEFORE

ld4      r6    = [r43]   ; AFTER
st1      [r7]  = r44     ; AFTER

st4.rel  [r42] = r5      ; STORE with release barrier.

This alternate ordering also meets the requirements of the original code. All of the BEFORE-labeled instructions are still before the release barrier and all the AFTER-labeled instructions occur after the acquire barrier. In order to enforce this type of ordering we require a special standalone ordering construct. On ia64 this is the mf (memory fence) instruction, and on that platform it acts as both a standalone acquire and release barrier, but also prevents store load reordering. Of the weakly ordered platforms that are widely available (PowerPC, Sparc, ia64), all of these architectures require a special standalone instruction to enforce this special ordering.

The Sparc architecture has a number of possible ordering models, but the one implemented in all the CPUs that you would find in a Solaris Sparc based system is designated total store order (TSO). For all practical purposes this means that all instruction pairs are ordered except a store load sequence, and in order to enforce such an ordering one must interject a ‘membar #storeload’ instruction.

On PowerPC, in order to guarantee store load instructions maintain their order, one must interject a heavy-weight sync instruction between the two (sync).

One gets the impression that whatever hardware black magic is required to implement store load ordering, it has the appearance of being harder to enforce this particular type of instruction ordering than any other.

One of the interesting things about these store load barrier instructions, is not that they do enforce this ordering, but that the release and acquire barrier instructions do not. In particular a mutex implemented with acquire and release barriers and not with a full barrier instruction can let exterior memory accesses into the critical section.

Suppose for example we have the following code

pSharedMem->x = 3 ;  // BEFORE
v = pSharedMem->y ;  // BEFORE

mutex.acquire() ; // assumed to contain an acquire barrier.

pSharedMem->z = 4 ;
foo( pSharedMem->w ) ;

mutex.release() ; // assumed to contain a release barrier.

pSharedMem->a = 2 ;  // AFTER
q = pSharedMem->b ;  // AFTER

Unless the mutex uses very heavy handed memory barrier instructions one has the possibility that either the AFTER or BEFORE labeled memory accesses may sneak into the critical section that the mutex provides.

This subtlety may not be documented by the mutex implementation, but one should not assume that a mutex does any more than keep stuff within the critical section from getting out.

The flag pattern illustrated.

The mutex is by far the most important example of barrier use that is required for correctness in a weakly ordered system. It is however, perhaps not the easiest to understand. A slightly simpler pattern is the use of an atomic to flag that another memory update is complete.

As mentioned it is common for correct memory barrier usage to require paired conugate acquire and release barriers. This is very much like what is in the implementation of the critical section that you are probably trying to avoid by using an atomic and the associated barriers. It will however, be split, with the release and acquire barrier usage divided into portions for the producer and consumer. As an example your critical section implementation would typically be of the form:

while (!pShared->lock.testAndSet_Acquire()) ;
// (this loop should include all the normal critical section stuff like
// spin, waste,
// pause() instructions, and last-resort-give-up-and-blocking on a resource
// until the lock is made available.)

// Access to shared memory.

pShared->foo = 1
v = pShared-> goo

pShared->lock.clear_Release()

Acquire memory barrier above makes sure that any loads (pShared->goo) that may have been started before the successful lock modification are tossed, to be restarted if necessary.

The release memory barrier ensures that the load from goo into the (local say) variable v is complete before the lock word protecting the shared memory is cleared.

You have a similar pattern in the typical producer and consumer atomic flag scenario (it is difficult to tell by your sample if that is what you are doing but should illustrate the idea).

Suppose your producer used an atomic variable to indicate that some other state is ready to use. You’ll want something like this:

pShared->goo = 14

pShared->atomic.setBit_Release()

Without a release (“write”) barrier here in the producer you have no guarantee that the hardware isn’t going to get to the atomic store before the goo store has made it through the CPU store queues, and up through the memory hierarchy where it is visible (even if you have a mechanism that ensures the compiler orders things the way you want).

In the consumer

if ( pShared->atomic.compareAndSwap_Acquire(1,1) )
{
   v = pShared->goo
}

Without a “read” barrier here you won’t know that the hardware hasn’t gone and fetched goo for you before the atomic access is complete. The atomic (ie: memory manipulated with the Interlocked functions doing stuff like lock cmpxchg), is only “atomic” with respect to itself, not other memory.

That’s really all there is to this example, so it is perhaps anticlimatic, as well as including a redundant discussion of a mutex implementation. However, it is a good first step and these notes are already too long. A more and better example will perhaps follow on a different day.

Conclusion.

Some of the issues associated with using atomics and barriers to replace the simpler lock mechanism have been discussed. It is unfortunate to have to provide examples that utilize assembly code to illustrate these ideas, but it is harder to discuss these topics without doing so. Hopefully the inclusion of some of the low level details and assembly listings has not rendered this on paper discussion too incomprehensible.

The fact that some low level systems details must be understood for correct use of atomics to avoid locking should perhaps be a warning to the programmer. It can be very hard to implement lock avoidance algorithms correctly, harder still to test them. There is also a significant maintenance impact to going down this road, so be good and sure that you really need the performance provided before taking this path. Premature optimization in this area can be particularly dangerous and costly.

Do you really have a good reason for going down the atomic path instead of just using a lock? Doing all this correctly can be very difficult. Be prepared for a lot of self doubt and insecurity in code reviews and make sure you have a lot of high concurrency testing with all sorts of random timing scenarios. Use a critical section unless you have a very very good reason to avoid it, and don’t write that critical section yourself.

The correctness aspects of memory barrier utilization makes your code intrinsically unportable. Because many correct uses of atomics also require barrier instructions, this can in turn imply that the “innocent and simple” atomic class library implementation you have access to can be correspondingly unportable. This may be surprising to the developer, but then again, many aspects of weakly ordered system behavior can be surprising.

On a weakly ordered system, your compiler probably provides _acquire and _release variations for most of the atomic manipulation methods. On a strongly ordered platform like ia32 or amd64, _acquire() or _release() suffixed atomic intrinsics will not be any different than the vanilla operations. With ia64 effectively dead (except on HP where its still twitching slightly) PowerPC is probably the most pervasive surviving weakly ordered architecture around, and even it has toyed with non-weak ordering (power5 was weakly ordered, power6 was not, and power7 is again weakly ordered). It will be interesting to what path the hardware folks take, and whether these details are of any importance in 10 or 15 years.

References

[1] M. Lyons, E. Silha, and B. Hay. PowerPC storage model and AIX programming, 2002. http://www.ibm.com/developerworks/eserver/articles/powerpc.html.

[2] D. Lea. The JSR-133 cookbook for compiler writers, 2005.

[3] L.C.H.J. Boehm. Threads and memory model for C++ [online]. http://www.hpl.hp.com/personal/Hans_Boehm/c++mm/.

[4] J. Manson and B. Goetz. Jsr 133 (java memory model) faq [online]. 2004. http://www.cs.umd.edu/ pugh/java/memoryModel/jsr-133-faq.html.

[5] L.C.H.J. Boehm. C++ atomic types and operations. mailings N2427. C++ standards committee, 2007. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2427.html.

[6] H. Sutter. {Prism-A Principle-Based Sequential Memory Model for Microsoft Native Code Platforms Draft Version 0.9. 1}, 2006. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2197.pdf.

[7] R. Silvera, M. Wong, P. McKenney, and B. Blainey. {A simple and efficient memory model for weakly-ordered architectures}, 2007. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2153.pdf.

[8] S.V. Adve and K. Gharachorloo. Shared memory consistency models: A tutorial. Computer, 29(12):66–76, 1996. http://www.hpl.hp.com/techreports/Compaq-DEC/WRL-95-7.pdf.

[9] J.D. Valois. Lock-free linked lists using compare-and-swap.

[10] T.L. Harris, K. Fraser, and I.A. Pratt. A practical multi-word compare-and-swap operation. Lecture Notes in Computer Science, 2508:265–279, 2002. http://www.cl.cam.ac.uk/research/srg/netos/papers/2002-casn.pdf.

[11] T.L. Harris. A pragmatic implementation of non-blocking linked-lists. Lecture Notes in Computer Science, 2180:300–314, 2001. http://www.cl.cam.ac.uk/research/srg/netos/papers/2001-caslists.pdf.

[12] T.L. Harris. Harris Papers [online]. http://www.cl.cam.ac.uk/research/srg/netos/lock-free/.

[13] R. Bencina. lock free papers and notes. [online]. 2006. http://www.audiomulch.com/ rossb/code/lockfree/.

[14] M.M. Michael. {Safe memory reclamation for dynamic lock-free objects using atomic reads and writes}. In {Proceedings of the twenty-first annual symposium on Principles of distributed computing}, pages 21–30. ACM New York, NY, USA, 2002. http://www.research.ibm.com/people/m/michael/podc-2002.pdf.

Continue from part 1. This is a wrapper class for Mutex.

Header File:
#ifndef _MUTEX_H_
#define _MUTEX_H_
#include <pthread.h>
class Mutex
{
public:
    friend class Condition;

    Mutex();
    virtual ~Mutex();
    Mutex(const Mutex& copy);

    bool lock();
    bool unlock();
    bool tryLock();
private:
    pthread_mutex_t      *m_pmutex;
    bool                  m_isOwner;

};

#endif // _MUTEX_H_
 Source File:

#include <cerrno>
#include “mutex.h”

Mutex::Mutex()
      :m_pmutex(0),
       m_isOwner(true)
{
    m_pmutex = new pthread_mutex_t;
    if (pthread_mutex_init(m_pmutex, 0) == 0)
    {
        //ok
    }
}

Mutex::~Mutex()
{
    if (m_isOwner)
    {
        if (pthread_mutex_destroy(m_pmutex) == EBUSY)
        {
            unlock();
            pthread_mutex_destroy(m_pmutex);
        }
        delete m_pmutex;
    }
}

Mutex::Mutex(const Mutex& copy)
{
    m_pmutex = copy.m_pmutex;
    m_isOwner = false;
}

bool Mutex::lock()
{
    if (pthread_mutex_lock(m_pmutex) == 0)
        return true;
    return false;
}

bool Mutex::unlock()
{
    if (pthread_mutex_unlock(m_pmutex) == 0)
        return true;
    return false;
}

bool Mutex::tryLock()
{
    if (pthread_mutex_trylock(m_pmutex) == 0)
        return true;
    return false;
}

This exercise poses the questions in a relatively simple manner so i will answer in the same way.

Finding Definitions for Eight Terms and Concepts used in Threaded Programming.

The term or concept will be followed by the definition.

Thread Synchronization

Thread synchronization refers to the simultaneous execution of multiple threads (defined further down) and/or processes, this allows a program to provide a wider range of services than using singular processes.

Locks

Basically a lock is a synchronization mechanism put in place to control access to resources between threads and processes. These locks are put in place to ensure the integrity of the data resources and that no 2 threads can access one data source at the same time.

Dead Locks

Wikipedia defines a deadlock as:

In computer science, deadlock refers to a specific condition when two or more processes are each waiting for each other to release a resource, or more than two processes are waiting for resources in a circular chain. Deadlock is a common problem in multiprocessing where many processes share a specific type of mutually exclusive resource known as a software lock or soft lock. Computers intended for the time-sharing and/or real-time markets are often equipped with a hardware lock (or hard lock) which guarantees exclusive access to processes, forcing serialized access. Deadlocks are particularly troubling because there is no general solution to avoid (soft) deadlocks.

Semaphores

Wikipedia defines a semaphore as:

In computer science, a semaphore is a protected variable or abstract data type which constitutes the classic method for restricting access to shared resources such as shared memory in a parallel programming environment. A counting semaphore is a counter for a set of available resources, rather than a locked/unlocked flag of a single resource. It was invented by Edsger Dijkstra^[1]. Semaphores are the classic solution to preventing race conditions in the dining philosophers problem, although they do not prevent resource deadlocks.

Mutual Exclusion (Mutex)

Mutual Exclusion is the concept of locking resources and restricting access.

Wikipedia defines the concept of mutual exclusion as:

Mutual exclusion (often abbreviated to mutex) algorithms are used in concurrent programming to avoid the simultaneous use of a common resource, such as a global variable, by pieces of computer code called critical sections. A critical section is a piece of code where a process or thread accesses a common resource. The critical section by itself is not a mechanism or algorithm for mutual exclusion. A program, process, or thread can have critical section in it without any mechanism or algorithm, which implements mutual exclusion. Examples of such resources are fine-grained flags, counters or queues, used to communicate between code that runs concurrently, such as an application and its interrupt handlers. The synchronization of access to those resources is an acute problem because a thread can be stopped or started at any time. To illustrate: suppose a section of code is altering a piece of data over several program steps, when another thread, perhaps triggered by some unpredictable event, starts executing. If this second thread reads from the same piece of data, the data, which is in the process of being overwritten, is in an inconsistent and unpredictable state. If the second thread tries overwriting that data, the ensuing state will probably be unrecoverable. These shared data being accessed by critical sections of code, must therefore be protected, so that other processes which read from or write to the chunk of data are excluded from running.

Thread

Basically threads are a lightweight processes. Darrel Ince defines a thread as The execution of a chunk of code which can be carried out in parallel with the execution of other chunks of code.

Event

Wikipedia defines an event as:

In computing an event is an action that is usually initiated outside the scope of a program and that is handled by a piece of code inside the program. Typically events are handled synchronous with the program flow, that is, the program has one or more dedicated places where events are handled. Typical sources of events include the user (who presses a key on the keyboard, in other words, through a keystroke). Another source is a hardware devices such as a timer. A computer program that changes its behavior in response to events is said to be event-driven, often with the goal of being interactive.

Waitable Timer

The Microsoft Developer Network defines a waitable timer object as:

A waitable timer object is a synchronization object whose state is set to signaled when the specified due time arrives. There are two types of waitable timers that can be created: manual-reset and synchronization. A timer of either type can also be a periodic timer.

Demonstration of Threading in Python

This program creates 10 tasks and sets a bounded semaphore so no more than 3 of the 10 processes can run at the same time. A mutex lock is also put in place. Each thread acquires the semaphore resource and lock and then releases it and runs until all taks have been executed successfully.

References

http://en.wikipedia.org/wiki/Deadlock

http://en.wikipedia.org/wiki/Semaphore_%28programming%29

http://en.wikipedia.org/wiki/Mutual_exclusion

http://en.wikipedia.org/wiki/Thread_%28computer_science%29

http://en.wikipedia.org/wiki/Event_%28computing%29

http://msdn.microsoft.com/en-us/library/ms687012%28VS.85%29.aspx

For those of us who work in the multi-threaded environment, the concept of mutex (“mutual exclusion”) should be no stranger. There are two common categories of mutex – simple and recursive.

Simple mutex can only be locked once, and if the same thread tries to lock the same mutex again, it will deadlock.

Recursive mutex can be locked by the same thread numerous times, and the same thread must unlock the same number of times before the critical region can be accessed by another thread.

Recursive mutex is common. In fact, mutexes are recursive by default in Windows.

With all that said, I believe recursive mutex should be avoid as much as possible.

Obvious Overhead

Since recursive mutex tracks more information than a simple mutex, it should be no surprise that it is more expensive.

Here is a sample test case with multiple threads accessing a critical region. The mutex implementation is from Boost.Thread library.

Red dotted line indicates the cost of a single simple mutex. Blue line indicates the cost of multiple recursive mutexes.

A single recursive mutex is quite a bit more expensive than a simple mutex. If recursive mutex is used, it is probably going to be locked more than once by the same thread (otherwise, why would you use it?). The graph also demonstrates the cost of recursively locking the mutex again, up to five times. As you can see, it is linear, and it is definitely not free.

The term “recursive mutex” might sound fancy, but they are nothing but redundant locks. After the critical region is locked, each additional recursive mutex adds nothing but overhead.

Some might argue that this cost is negligible. This might be true depending on the application, but this is just the surface of the problem.

The Real Problem

The truth is that locks and mutexes are difficult to use correctly. Until we have STM, there is no silver bullet.

Recursive mutex makes locking easy by taking away the programmer’s responsibility of tracking mutexes. Programmers can add recursive mutex to existing functions and member functions easily without refactoring code. This would not have been possible with simple mutex. However, this advantage is merely an illusion.

Don’t believe it? Here’s the thought of David Butenhof, the person who introduced recursive mutex in POSIX thread.

Here’s some of his powerful arguments:

1. Recursive mutex allows you to lose track of your locking scheme and scope. As your call stack becomes deeper, you will have less sense on how long you’ve been locking the critical region.

2. Recursive mutex are reference counted, and releasing it does not imply that the critical region is freed. So how would you know how to unlock?

I’ve learned a lot from his post. His argument addresses the fundamental design issue in software that use recursive mutex.

To avoid using recursive mutex, we have to consider refactoring.

Refactoring

In functional programming, a typical refactoring technique to break up recursive locking is by splitting a function into two components- outer and inner. The outer function holds the lock, and invoke the inner functions to perform its task. The inner functions is an internal function that is not exposed as an API.

In Object Oriented programming, this technique can also be applied in form of Pimpl idiom or private functions.

Consider this example:

Class A has two public functions – One and Two. One calls Two. Both methods are thread-safe, they both lock the same recursive mutex, which is a private member of Class A. Below demonstrates that the functional programming refactoring technique can be applied in OO programming.

A::One calls A::Two, they both lock on the same mutex member variable.

Refactor the implemention of A::Two into A::Two_Impl. A::One and A::Two nows calls A::Two_Impl. Recursive mutex becomes unnecessary, and can be refactored into a simple mutex.

Refactoring – A More Complicated Scenario

When more than one classes are involved, things gets more complicated. The technique above will not hold.

Here’s a more complicated (yet realistic) example:

Class A has two public functions – One and Three. Both functions are thread-safe, and lock the same recursive mutex, which is a private member variable of A. Class B has one public function – Two.

A::One calls B::Two, and B::Two calls A::Three.

A more complicated scenario that involves more than one class.

Now we have a mess. The possible call paths grows exponentially as the call stack gets deeper.

In the example, there are three possible paths.

1. A::One, B::Two, A::Three

2. B::Two, A::Three

3. A::Three

If we apply the previous technique, we would refactor A::Three into A::Three and A::Three_Impl. That won’t work here because class B expects A::Three_Impl to be a public interface.

There is no fix-all solution here, but here’s some suggestion.

1. Refactor A and B such that they do not circular reference each other. In other word, A can reference B, or B can reference A, but not at the same time.

2. Merge A and B in one class, or move B::Two into class A.

Use Recursive Mutex If It Is Absolutely Positively Necessary

Unfortunately, in the real world, things get even more complicated. The complicated example above only involves two classes. But what if more classes are involved, the number of call paths grows exponentially, and becomes unrealistic to refactor?

As a discipline of computer science, I would argue that for every solution with recursive mutex, there must be a solution for simple mutex.

As a software engineer who is under the gun with a deadline… I say just hack it and use recursive mutex. We are shipping tomorrow!

But as David Butenhof said, recursive mutex is a hack. Instead of relying on recursive mutex up front, we should avoid it until it is absolutely necessary.

Conclusion

Recursive mutex is dangerous because you lose sense of locking scope. It costs more than a simple mutex.

Consider to refactor the code to avoid the use of recursive mutex.

Use recursive mutex if it can’t be refactored economically.

Source

The source and the spreadsheet can be downloaded here.

Tools: Visual Studio 2008, Boost 1.39, Window XP SP3 32bit

Some applications are built to allow users to run single instance of an application at a time. That means we need to get an information from first instance of an application to the second instance and thus we need to prevent the second instance from running.

We can use one of the synchronization objects mutex to ensure two or more threads do not attempt to access a shared memory simultaneously. We need to check it within the startup(WinMain) of the application.

BOOL WinMain()
{
   if(!IsInstanceRunningAlready)
   {
      return FALSE;
   }

   //Do further operations here

   return FALSE;
}

BOOL IsInstanceRunningAlready()
{
   // For handling Single Instance of Application
   HANDLE hSingleInstance = NULL;
   hSingleInstance		= CreateMutex( NULL, FALSE,_T("ApplicationName"));
   DWORD dwLastError	= GetLastError();

   if(dwLastError == ERROR_ALREADY_EXISTS)
   {
      AfxMessageBox(_T("Application already running"));
      CloseHandle(hSingleInstance);
      return 0;
   }

   return 1;
}

Managed Threading

Threading Objects and Features

A process is a collection of virtual memory space, code, data, and system resources. A thread is code that is to be serially executed within a process. A processor executes threads, not processes, so each application has at least one process, and a process always has at least one thread of execution, known as the primary thread. A process can have multiple threads in addition to the primary thread.

Processes communicate with one another through messages, using Microsoft’s Remote Procedure Call (RPC) technology to pass information to one another. There is no difference to the caller between a call coming from a process on a remote machine and a call coming from another process on the same machine.

When a thread begins to execute, it continues until it is killed or until it is interrupted by a thread with higher priority (by a user action or the kernel’s thread scheduler). Each thread can run separate sections of code, or multiple threads can execute the same section of code. Threads executing the same block of code maintain separate stacks. Each thread in a process shares that process’s global variables and resources.

Threads are the basic unit to which an operating system allocates processor time, and more than one thread can be executing code inside that process. Each thread maintains

• exception handlers,
• a scheduling priority, and
• a set of structures the system uses to save the thread context until it is scheduled.

The thread context includes all the information the thread needs to seamlessly resume execution, including

• the thread’s set of CPU registers and
• stack, in the address space of the thread’s host process.

The .NET Framework further subdivides an operating system process into lightweight managed subprocesses, called application domains, represented by System..::.AppDomain. One or more managed threads (represented by System.Threading..::.Thread) can run in one or any number of application domains within the same managed process. Although each application domain is started with a single thread, code in that application domain can create additional application domains and additional threads. The result is that a managed thread can move freely between application domains inside the same managed process; you might have only one thread moving among several application domains.

If your application uses only one thread of execution, you can combine asynchronous programming with .NET Framework remoting or XML Web services created using ASP.NET to use the processing time of other computers in addition to that of your own to increase responsiveness to the user and decrease the data processing time of your application.

Disadvantages of Multiple Threads

It is recommended that you use as few threads as possible, thereby minimizing the use of operating-system resources and improving performance. Threading also has resource requirements and potential conflicts to be considered when designing your application. The resource requirements are as follows:

• The system consumes memory for the context information required by processes, AppDomain objects, and threads. Therefore, the number of processes, AppDomain objects, and threads that can be created is limited by available memory.
• Keeping track of a large number of threads consumes significant processor time. If there are too many threads, most of them will not make significant progress. If most of the current threads are in one process, threads in other processes are scheduled less frequently.
• Controlling code execution with many threads is complex, and can be a source of many bugs.
• Destroying threads requires knowing what could happen and handling those issues.

Resources that require synchronization include:

• System resources (such as communications ports).
• Resources shared by multiple processes (such as file handles).
• The resources of a single application domain (such as global, static, and instance fields) accessed by multiple threads.

Synchronizing Data for Multithreading

When multiple threads can make calls to the properties and methods of a single object, it is critical that those calls be synchronized. Otherwise one thread might interrupt what another thread is doing, and the object could be left in an invalid state. A class whose members are protected from such interruptions is called thread-safe.

The Common Language Infrastructure provides several strategies to synchronize access to instance and static members:

• Synchronized code regions. You can use the Monitor class or compiler support for this class to synchronize only the code block that needs it, improving performance.
• Manual synchronization.You can use the synchronization objects provided by the .NET Framework class library. See Overview of Synchronization Primitives, which includes a discussion of the Monitor class.
• Synchronized contexts. You can use the SynchronizationAttribute to enable simple, automatic synchronization for ContextBoundObject objects.
• Synchronized property. A few classes, such as Hashtable and Queue, provide a Synchronized property that returns a thread-safe wrapper for an instance of the class. See Collections and Synchronization (Thread Safety).

The common language runtime provides a thread model in which classes fall into a number of categories that can be synchronized in a variety of different ways depending on the requirements. The following table shows what synchronization support is provided for fields and methods with a given synchronization category.

Category	Global fields	Static fields	Static methods	Instance fields	Instance methods	Specific code blocks
No Synchronization	No	No	No	No	No	No
Synchronized Context	No	No	No	Yes	Yes	No
Synchronized Code Regions	No	No	Only if marked	No	Only if marked	Only if marked
Manual Synchronization	Manual	Manual	Manual	Manual	Manual	Manual

Threading and Application Design

In general, using the ThreadPool class is the easiest way to handle multiple threads for relatively short tasks that will not block other threads and when you do not expect any particular scheduling of the tasks. However, there are a number of reasons to create your own threads:

• If you need a task to have a particular priority.
• If you have a task that might run a long time (and therefore block other tasks).
• If you need to place threads into a single-threaded apartment (all ThreadPool threads are in the multithreaded apartment).
• If you need a stable identity associated with the thread. For example, you should use a dedicated thread to abort that thread, suspend it, or discover it by name.
• If you need to run background threads that interact with the user interface, the .NET Framework version 2.0 provides a BackgroundWorker component that communicates using events, with cross-thread marshaling to the user-interface thread.

Apartment Threading (Single Threaded Apartment)

Apartment threaded means there are multiple threads within the application. In single threaded apartment (STA) each thread is isolated in a separate apartment underneath the process. The process can have any number of apartments that share data through a proxy. The application defines when and for how long the thread in each apartment should execute. All requests are serialized through the Windows message queue such that only a single apartment is accessed at a time and thus only a single thread will be executing at any one time. STA is the threading model that most Visual Basic developers are familiar with because this is the threading model available to VB applications prior to VB.NET. You can think of it like an apartment building full of a row of one room apartments that are accessible one at a time through a single hallway. The advantage this provides over single threaded is that multiple commands can be issued at one time instead of just a single command, but the commands are still sequentially executed.

Free Threading (Multi Threaded Apartment)

Free threaded applications were limited to programming languages such as C++ until the release of Microsoft .NET. The free threaded/Multi Threaded Apartment (MTA) model has a single apartment created underneath the process rather than multiple apartments. This single apartment holds multiple threads rather than just a single thread. No message queue is required because all of the threads are a part of the same apartment and can share data without a proxy. You can think of it like a building with multiple rooms that are all accessible once you are inside the building. These applications typically execute faster than single threaded and STA because there is less system overhead and can be optimized to eliminate system idle time.

These types of applications are more complex to program. The developer must provide thread synchronization as part of the code to ensure that threads do not simultaneously access the same resources. A condition known as a race condition can occur when a thread accesses a shared resource and modifies the resource to an invalid state and then another thread accesses the shared resource and uses it in the invalid state before the other thread can return the resource to a valid state. Therefore it is necessary to place a lock on a resource to prevent other threads from accessing the resource until the lock has been removed. However, this can lead to a deadlock situation where two threads are competing for resources and neither can proceed.

The only way to avoid situations like these is through good design and testing.

Rules for single-threaded apartments are simple, but it is important to follow them carefully:

• Every object should live on only one thread (within a single-threaded apartment).
• Initialize the COM library for each thread.
• Marshal all pointers to objects when passing them between apartments.
• Each single-threaded apartment must have a message loop to handle calls from other processes and apartments within the same process. Single-threaded apartments without objects (client only) also need a message loop to dispatch the broadcast messages that some applications use.
• DLL-based or in-process objects do not call the COM initialization functions; instead, they register their threading model with the ThreadingModel named-value under the InprocServer32 key in the registry. Apartment-aware objects must also write DLL entry points carefully. There are special considerations that apply to threading in-process servers. For more information, see In-Process Server Threading Issues.

Following are some important considerations regarding synchronization for multithreaded apartments:

• COM provides call synchronization for single-threaded apartments only.
• Multithreaded apartments do not receive calls while making calls (on the same thread).
• Multithreaded apartments cannot make input-synchronized calls.
• Asynchronous calls are converted to synchronous calls in multithreaded apartments.
• The message filter is not called for any thread in a multithreaded apartment.

Important Note:

Do not lock the type — that is, typeof(MyType) in C#, GetType(MyType) in Visual Basic, or MyType::typeid in C++ — in order to protect static methods (Shared methods in Visual Basic). Use a private static object instead. Similarly, do not use this in C# (Me in Visual Basic) to lock instance methods. Use a private object instead. A class or instance can be locked by code other than your own, potentially causing deadlocks or performance problems.

Synchronized Context

You can use the SynchronizationAttribute on any ContextBoundObject to synchronize all instance methods and fields. All objects in the same context domain share the same lock. Multiple threads are allowed to access the methods and fields, but only a single thread is allowed at any one time.

Setting Up for a Background Operation

To set up for a background operation, add an event handler for the DoWork event. Call your time-consuming operation in this event handler.To start the operation, call RunWorkerAsync. To receive notifications of progress updates, handle the ProgressChanged event. To receive a notification when the operation is completed, handle the RunWorkerCompleted event.The methods that handle the ProgressChanged and RunWorkerCompleted events can access the application’s user interface, because those events are raised on the thread that called the RunWorkerAsync method. However, the DoWork event handler cannot work with any user-interface objects because it runs on the background thread.

Safe Points

Multithreaded Applications.

You can provide parameters and return values to thread-pool threads by using the optional ByVal state-object variable of the QueueUserWorkItem method. Thread-timer threads also support a state object for this purpose. For information on thread pooling and thread timers, see Thread Pooling and Thread Timers.

Sub DoWork()
    ' Queue a task
    System.Threading.ThreadPool.QueueUserWorkItem( _
        New System.Threading.WaitCallback(AddressOf SomeLongTask))
    ' Queue another task
    System.Threading.ThreadPool.QueueUserWorkItem( _
        New System.Threading.WaitCallback(AddressOf AnotherLongTask))
End Sub
Sub SomeLongTask(ByVal state As Object)
    ' Insert code to perform a long task.
End Sub
Sub AnotherLongTask(ByVal state As Object)
    ' Insert code to perform another long task.
End Sub

Application Domains and Threads.

Events (C#).

Events can be used to synchronize threads.

Delegate is a creatable type. Programming with delegates requires creating delegate objects from delegate types.

Delegates seamlessly support binding a notification source to multiple handler methods through a feature known as multicasting.

Delegate binding, see Delegates in the Common Type System and CreateDelegate(Type,Object,MethodInfo,Boolean).

Delegates are referred to as

• multicast, or
• combinable,

because a delegate can invoke one or more methods and can be used in combining operations.A delegate can represent a

• static method or
• an instance method.

When the delegate represents an instance method, the delegate stores not only a reference to the method’s entry point, but also a reference to the class instance. Unlike function pointers,

• delegates are object oriented and
• type safe.

How to: Subscribe to and Unsubscribe from Events (C# Programming Guide)

The line of code that is needed to subscribe to the event is also automatically generated in the InitializeComponent method in the Form1.Designer.cs file in your project. It looks like this:

  this.Load += new System.EventHandler(this.Form1_Load);

To subscribe to events programmatically

1. Define an event handler method whose signature matches the delegate signature for the event. For example, if the event is based on the EventHandler delegate type, then the following code represents the method stub:

     void HandleCustomEvent(object sender, CustomEventArgs a)
     {
        // Do something useful here.
     }

2. Use the addition assignment operator (+=) to attach your event handler to the event. In the following example, assume that an object named publisher has an event named RaiseCustomEvent. Note that the subscriber class needs a reference to the publisher class in order to subscribe to its events.

   publisher.RaiseCustomEvent += HandleCustomEvent;

Note that the above syntax is new in C# 2.0. It is exactly equivalent to the C# 1.0 syntax in which the encapsulating delegate must be explicitly created using the new keyword:

publisher.RaiseCustomEvent += new CustomEventHandler(HandleCustomEvent);

To subscribe to events by using an anonymous method

• Use the addition assignment operator (+=) to attach your anonymous method to the event. In the following example, assume that an object named publisher has an event named RaiseCustomEvent and that a CustomEventArgs class has also been defined to carry some kind of specialized event information. Note that the subscriber class needs a reference to publisher in order to subscribe to its events.

    publisher.RaiseCustomEvent += delegate(object o, CustomEventArgs e)
    {
      string s = o.ToString() + " " + e.ToString();
      Console.WriteLine(s);
    };

It is important to note that you cannot easily unsubscribe from an event if you used an anonymous method to subscribe to it. To unsubscribe in this scenario, go back to the code where you subscribe to the event, store the anonymous method in a delegate variable, and then add the delegate to the event.

When an event has multiple subscribers, the event handlers are invoked synchronously when an event is raised. To invoke events asynchronously, see Calling Synchronous Methods Asynchronously.

four common ways to use BeginInvoke and EndInvoke to make asynchronous calls. After calling BeginInvoke you can do the following:

• Do some work and then call EndInvoke to block until the call completes.
• Obtain a WaitHandle using the System.IAsyncResult.AsyncWaitHandle property, use its WaitOne method to block execution until the WaitHandle is signaled, and then call EndInvoke.
• Poll the IAsyncResult returned by BeginInvoke to determine when the asynchronous call has completed, and then call EndInvoke.
• Pass a delegate for a callback method to BeginInvoke. The method is executed on a ThreadPool thread when the asynchronous call completes. The callback method calls EndInvoke.

  Important
  Always call EndInvoke to complete your asynchronous call.

you should know that events in the .NET Framework are layered on top of delegates. When you use an event-driven application framework such as Windows® Forms or ASP.NET, your knowledge of delegates will make you a much stronger developer.

Delegates provide the primary means in .NET for executing a method on a secondary thread in an asynchronous fashion. Therefore, delegates open the door to multithreading.

the compiler generates a class definition for each delegate type. The class that is generated for each delegate type is a creatable class that inherits from the System.Multicast delegate.

compiler has also added a public constructor and three public methods named

• Invoke,
• BeginInvoke, and
• EndInvoke.

Asynchronous Programming Design Patterns

Asynchronous operations are typically used to perform tasks that might take a long time to complete, such as opening large files, connecting to remote computers, or querying a database. An asynchronous operation executes in a thread separate from the main application thread. When an application calls methods to perform an operation asynchronously, the application can continue executing while the asynchronous method performs its task.

Asynchronous programming is a feature supported by many areas of the .NET Framework, including:

• File IO, Stream IO, Socket IO.
• Networking.
• Remoting channels (HTTP, TCP) and proxies.
• XML Web services created using ASP.NET.
• ASP.NET Web Forms.
• Message Queuing using the MessageQueue class.

The .NET Framework provides two design patterns for asynchronous operations:

• Asynchronous operations that use IAsyncResult objects.
• Asynchronous operations that use events. Event-based Asynchronous Pattern Overview.

For relatively simple multithreaded applications, the BackgroundWorker component provides a straightforward solution. For more sophisticated asynchronous applications, consider implementing a class that adheres to the Event-based Asynchronous Pattern.

Asynchronous Programming Overview

Discusses the IAsyncResult based asynchronous design pattern, which provides a flexible programming model to deal with asynchronous operations.

The correct choice depends on whether the application has instructions that can execute while the operation completes. If an application cannot perform any additional work until it receives the results of the asynchronous operation, the application must block until the results are available. To block until an asynchronous operation completes, you can use one of the following approaches:

• Call EndOperationName from the application’s main thread, blocking application execution until the operation is complete. For an example that illustrates this technique, see Blocking Application Execution by Ending an Asynchronous Operation.
• Use the AsyncWaitHandle to block application execution until one or more operations are complete. For an example that illustrates this technique, see Blocking Application Execution Using an AsyncWaitHandle.

Applications that do not need to block while the asynchronous operation completes can use one of the following approaches:

• Poll for operation completion status by checking the IsCompleted property periodically and calling EndOperationName when the operation is complete. For an example that illustrates this technique, see Polling for the Status of an Asynchronous Operation.
• Use an AsyncCallback delegate to specify a method to be invoked when the operation is complete. For an example that illustrates this technique, see Using an AsyncCallback Delegate to End an Asynchronous Operation.

Creating Event Handlers in Windows Forms

• How to: Subscribe to and Unsubscribe from Events (C# Programming Guide)
• How to: Publish Events that Conform to .NET Framework Guidelines (C# Programming Guide)
• How to: Raise Base Class Events in Derived Classes (C# Programming Guide)
• How to: Implement Interface Events (C# Programming Guide)
• Thread Synchronization (C# Programming Guide)
• How to: Use a Dictionary to Store Event Instances (C# Programming Guide)
• Events Sample
• Event Design

Multithreading in Components

Managed Threading Best Practices

Implementing the Event-based Asynchronous Pattern

Describes the standardized way to package a class that has asynchronous features.

features of the Event-based Asynchronous Pattern discussed in this topic.

• Opportunities for Implementing the Event-based Asynchronous Pattern
• Naming Asynchronous Methods
• Optionally Support Cancellation
• Optionally Support the IsBusy Property
• Optionally Provide Support for Progress Reporting
• Optionally Provide Support for Returning Incremental Results
• Handling Out and Ref Parameters in Methods

Deciding When to Implement the Event-based Asynchronous Pattern

Describes how to determine when you should choose to implement the Event-based Asynchronous Pattern instead of the IAsyncResult pattern.

Event-based Asynchronous Pattern Technology Sample

Shows how to use the Event-based Asynchronous Pattern to perform common asynchronous operations.

Events and Delegates

How to: Run an Operation in the Background

use the BackgroundWorker class to run the operation on another thread.

Robust Programming

Caution
When using multithreading of any sort, you potentially expose yourself to very serious and complex bugs. Consult the Managed Threading Best Practices before implementing any solution that uses multithreading.

Deadlocks

A deadlock occurs when each of two threads tries to lock a resource the other has already locked. Neither thread can make any further progress.

Many methods of the managed threading classes provide time-outs to help you detect deadlocks. For example, the following code attempts to acquire a lock on the current instance. If the lock is not obtained in 300 milliseconds, Monitor.TryEnter returns false.

Race Conditions

A race condition is a bug that occurs when the outcome of a program depends on which of two or more threads reaches a particular block of code first. Running the program many times produces different results, and the result of any given run cannot be predicted.

A simple example of a race condition is incrementing a field. Suppose a class has a private static field (Shared in Visual Basic) that is incremented every time an instance of the class is created, using code such as objCt++; (C#) or objCt += 1 (Visual Basic). This operation requires loading the value from objCt into a register, incrementing the value, and storing it in objCt.

In a multithreaded application, a thread that has loaded and incremented the value might be preempted by another thread which performs all three steps; when the first thread resumes execution and stores its value, it overwrites objCt without taking into account the fact that the value has changed in the interim.

This particular race condition is easily avoided by using methods of the Interlocked class, such as Interlocked.Increment. To read about other techniques for synchronizing data among multiple threads, see Synchronizing Data for Multithreading.

Race conditions can also occur when you synchronize the activities of multiple threads. Whenever you write a line of code, you must consider what might happen if a thread were preempted before executing the line (or before any of the individual machine instructions that make up the line), and another thread overtook it.

Overview of Synchronization Primitives

The .NET Framework provides a range of synchronization primitives for controlling the interactions of threads and avoiding race conditions. These can be roughly divided into three categories:

• locking,
• signaling, and
• interlocked operations.

The categories are not tidy and clearly defined: some synchronization mechanisms have characteristics of multiple categories; events that release a single thread at a time are functionally like locks; the release of any lock can be thought of as a signal; and interlocked operations can be used to construct locks. However, the categories are still useful.

Signaling

The simplest way to wait for a signal from another thread is to call the Join method, which blocks until the other thread completes. Join has two overloads that allow the blocked thread to break out of the wait after a specified interval has elapsed.

Wait handles provide a much richer set of waiting and signaling capabilities.

Wait Handles

Wait handles derive from the WaitHandle class, which in turn derives from MarshalByRefObject. Thus, wait handles can be used to synchronize the activities of threads across application domain boundaries.

Threads block on wait handles by calling the instance method WaitOne or one of the static methods WaitAll, WaitAny, or SignalAndWait. How they are released depends on which method was called, and on the kind of wait handles.

For a conceptual overview, see Wait Handles. Also EventWaitHandle, AutoResetEvent, and ManualResetEvent.

Interlocked Operations

Interlocked operations are simple atomic operations performed on a memory location by static methods of the Interlocked class. Those atomic operations include addition, increment and decrement, exchange, conditional exchange depending on a comparison, and read operations for 64-bit values on 32-bit platforms.

Note:
The guarantee of atomicity is limited to individual operations; when multiple operations must be performed as a unit, a more coarse-grained synchronization mechanism must be used.

Although none of these operations are locks or signals, they can be used to construct locks and signals. Because they are native to the Windows operating system, interlocked operations are extremely fast.

Interlocked operations can be used with volatile memory guarantees to write applications which exhibit powerful non-blocking concurrency, however they require sophisticated, low level programming, and for most purposes simple locks are a better choice.

For a conceptual overview, see Interlocked Operations.

SynchronizationAttribute

When this attribute is applied to an object, only one thread can be executing in all contexts that share an instance of this property. This is done by contributing sinks that intercept and serialize incoming calls for the respective contexts. If the property is marked for reentry, then callouts are intercepted too. The callout interception allows other waiting threads to enter the synchronization domain for maximal throughput.

Note:

There are two classes named SynchronizationAttribute : one in the System.Runtime.Remoting.Contexts namespace, and the other in the System.EnterpriseServices namespace. The System.EnterpriseServices..::.SynchronizationAttribute class supports only synchronous calls, and can be used only with serviced components. (For more information on serviced components, see [<topic://cpconservicedcomponentoverview>].) The System.Runtime.Remoting.Contexts..::.SynchronizationAttribute supports both synchronous and asynchronous calls, and can be used only with context bound objects. (For more information on context bound objects, see the ContextBoundObject class.)

Recommendations for Class Libraries

Consider the following guidelines when designing class libraries for multithreading:

• Avoid the need for synchronization, if possible. This is especially true for heavily used code. For example, an algorithm might be adjusted to tolerate a race condition rather than eliminate it. Unnecessary synchronization decreases performance and creates the possibility of deadlocks and race conditions.
• Make static data (Shared in Visual Basic) thread safe by default.
• Do not make instance data thread safe by default. Adding locks to create thread-safe code decreases performance, increases lock contention, and creates the possibility for deadlocks to occur. In common application models, only one thread at a time executes user code, which minimizes the need for thread safety. For this reason, the .NET Framework class libraries are not thread safe by default.
• Avoid providing static methods that alter static state. In common server scenarios, static state is shared across requests, which means multiple threads can execute that code at the same time. This opens up the possibility of threading bugs. Consider using a design pattern that encapsulates data into instances that are not shared across requests. Furthermore, if static data are synchronized, calls between static methods that alter state can result in deadlocks or redundant synchronization, adversely affecting performance.

General Recommendations

Consider the following guidelines when using multiple threads:

• Don’t use Thread.Abort to terminate other threads. Calling Abort on another thread is akin to throwing an exception on that thread, without knowing what point that thread has reached in its processing.
• Don’t use Thread.Suspend and Thread.Resume to synchronize the activities of multiple threads. Do use
• • Mutex,
  • ManualResetEvent,
  • AutoResetEvent,
  • Monitor.
• Don’t control the execution of worker threads from your main program (using events, for example). Instead, design your program so that worker threads are responsible for waiting until work is available, executing it, and notifying other parts of your program when finished. If your worker threads do not block, consider using thread pool threads. Monitor.PulseAll is useful in situations where worker threads block.
• The Birth of the Thread Pool
  Capability 1: Calling a Method Asynchronously
  Capability 2: Calling a Method at Timed Intervals
  Capability 3: Calling a Method When a Single Kernel Object Becomes Signaled
• Don’t use types as lock objects. That is, avoid code such as lock(typeof(X)) in C# or SyncLock(GetType(X)) in Visual Basic, or the use of System.Threading.Monitor.Enter(System.Object) with Type objects. For a given type, there is only one instance of System.Type per application domain. If the type you take a lock on is public, code other than your own can take locks on it, leading to deadlocks. For additional issues, see Reliability Best Practices.
• Use caution when locking on instances, for example lock(this) in C# or SyncLock(Me) in Visual Basic. If other code in your application, external to the type, takes a lock on the object, deadlocks could occur.
• Do ensure that a thread that has entered a monitor always leaves that monitor, even if an exception occurs while the thread is in the monitor. The C# lock statement and the Visual Basic SyncLock statement provide this behavior automatically, employing a finally block to ensure that Monitor.Exit is called. If you cannot ensure that Exit will be called, consider changing your design to use Mutex. A mutex is automatically released when the thread that currently owns it terminates.
• Do use multiple threads for tasks that require different resources, and avoid assigning multiple threads to a single resource. For example, any task involving I/O benefits from having its own thread, because that thread will block during I/O operations and thus allow other threads to execute. User input is another resource that benefits from a dedicated thread. On a single-processor computer, a task that involves intensive computation coexists with user input and with tasks that involve I/O, but multiple computation-intensive tasks contend with each other.
• Consider using methods of the Interlocked class for simple state changes, instead of using the lock statement (SyncLock in Visual Basic). The lock statement is a good general-purpose tool, but the Interlocked class provides better performance for updates that must be atomic. Internally, it executes a single lock prefix if there is no contention. In code reviews, watch for code like that shown in the following examples. In the first example, a state variable is incremented:

Performance Considerations with Locks, Threads, Semaphores

You can improve performance by using the Increment method instead of the lock statement, as follows:

C#

System.Threading.Interlocked.Increment(myField);

In the second example, a reference type variable is updated only if it is a null reference (Nothing in Visual Basic).

C#

if (x == null)
{
    lock (lockObject)
    {
        if (x == null)
        {
            x = y;
        }
    }
}

Performance can be improved by using the CompareExchange method instead, as follows:

C#

System.Threading.Interlocked.CompareExchange(ref x, y, null);

The methods of this class help protect against errors that can occur when the scheduler switches contexts while a thread is updating a variable that can be accessed by other threads, or when two threads are executing concurrently on separate processors. The members of this class do not throw exceptions.The Increment and Decrement methods increment or decrement a variable and store the resulting value in a single operation. On most computers, incrementing a variable is not an atomic operation, requiring the following steps:

1. Load a value from an instance variable into a register.
2. Increment or decrement the value.
3. Store the value in the instance variable.

If you do not use Increment and Decrement, a thread can be preempted after executing the first two steps. Another thread can then execute all three steps. When the first thread resumes execution, it overwrites the value in the instance variable, and the effect of the increment or decrement performed by the second thread is lost.

The Exchange method atomically exchanges the values of the specified variables. The CompareExchange method combines two operations: comparing two values and storing a third value in one of the variables, based on the outcome of the comparison. The compare and exchange operations are performed as an atomic operation.

//A simple method that denies reentrancy.
        static bool UseResource()
        {
            //0 indicates that the method is not in use.
            if(0 == Interlocked.Exchange(ref usingResource, 1))
            {
                Console.WriteLine("{0} acquired the lock", Thread.CurrentThread.Name);

                //Code to access a resource that is not thread safe would go here.

                //Simulate some work
                Thread.Sleep(500);

                Console.WriteLine("{0} exiting lock", Thread.CurrentThread.Name);

                //Release the lock
                Interlocked.Exchange(ref usingResource, 0);
                return true;
            }
            else
            {
                Console.WriteLine("   {0} was denied the lock", Thread.CurrentThread.Name);
                return false;
            }
        }

Source Code Examples

• Part 1 – Creating a thread in Windows Forms -  Some basic ideas about firing a background and how to synchronize it with the UI.
• Part 2 – Using BackgroundWorker -  the .NET 2.0 way using a BackgroundWorker component

• Part 3 – Automatic Marshalling – creating your own component with auto marshalling

When Not to Use Thread Pool Threads

There are several scenarios in which it is appropriate to create and manage your own threads instead of using thread pool threads:

• You require a foreground thread.
• You require a thread to have a particular priority.
• You have tasks that cause the thread to block for long periods of time. The thread pool has a maximum number of threads, so a large number of blocked thread pool threads might prevent tasks from starting.
• You need to place threads into a single-threaded apartment. All ThreadPool threads are in the multithreaded apartment.
• You need to have a stable identity associated with the thread, or to dedicate a thread to a task.

BackgroundWorker Component Overview

Describes the BackgroundWorker component, which gives you the ability to execute time-consuming operations asynchronously (“in the background”), on a thread different from your application’s main UI thread.

There are many commonly performed operations that can take a long time to execute. For example:

• Image downloads
• Web service invocations
• File downloads and uploads (including for peer-to-peer applications)
• Complex local computations
• Database transactions
• Local disk access, given its slow speed relative to memory access

System.MarshalByRefObject
System.ComponentModel.Component
System.ComponentModel.BackgroundWorker

BackgroundWorker

Describes this class and has links to all its members.

To execute a time-consuming operation in the background, create a BackgroundWorker and listen for events that report the progress of your operation and signal when your operation is finished. You can create the BackgroundWorker programmatically or you can drag it onto your form from the Components tab of the Toolbox. If you create the BackgroundWorker in the Windows Forms Designer, it will appear in the Component Tray, and its properties will be displayed in the Properties window.

To set up for a background operation, add an event handler for the DoWork event. Call your time-consuming operation in this event handler. To start the operation, call RunWorkerAsync. To receive notifications of progress updates, handle the ProgressChanged event. To receive a notification when the operation is completed, handle the RunWorkerCompleted event.

Note:
You must be careful not to manipulate any user-interface objects in your DoWork event handler. Instead, communicate to the user interface through the ProgressChanged and RunWorkerCompleted events.BackgroundWorker events are not marshaled across AppDomain boundaries. Do not use a BackgroundWorker component to perform multithreaded operations in more than one AppDomain.

If your background operation requires a parameter, call RunWorkerAsync with your parameter. Inside the DoWork event handler, you can extract the parameter from the DoWorkEventArgs.Argument property.

System.Threading..::.ThreadPool

Remarks

Note:

The HostProtectionAttribute attribute applied to this type or member has the following Resources property value: Synchronization | ExternalThreading. The HostProtectionAttribute does not affect desktop applications (which are typically started by double-clicking an icon, typing a command, or entering a URL in a browser). For more information, see the HostProtectionAttribute class or SQL Server Programming and Host Protection Attributes.

Many applications create threads that spend a great deal of time in the sleeping state, waiting for an event to occur. Other threads might enter a sleeping state only to be awakened periodically to poll for a change or update status information. Thread pooling enables you to use threads more efficiently by providing your application with a pool of worker threads that are managed by the system. One thread monitors the status of several wait operations queued to the thread pool. When a wait operation completes, a worker thread from the thread pool executes the corresponding callback function.

Note:
The threads in the managed thread pool are background threads. That is, their IsBackground properties are true. This means that a ThreadPool thread will not keep an application running after all foreground threads have exited.

You can also queue work items that are not related to a wait operation to the thread pool. To request that a work item be handled by a thread in the thread pool, call the QueueUserWorkItem method. This method takes as a parameter a reference to the method or delegate that will be called by the thread selected from the thread pool. There is no way to cancel a work item after it has been queued.

Timer-queue timers and registered wait operations also use the thread pool. Their callback functions are queued to the thread pool.

There is one thread pool per process. The thread pool has a default size of 250 worker threads per available processor, and 1000 I/O completion threads. The number of threads in the thread pool can be changed by using the SetMaxThreads method. Each thread uses the default stack size and runs at the default priority.

The thread pool maintains a minimum number of idle threads. For worker threads, the default value of this minimum is the number of processors. The GetMinThreads method obtains the minimum numbers of idle worker and I/O completion threads.

When all thread pool threads have been assigned to tasks, the thread pool does not immediately begin creating new idle threads. To avoid unnecessarily allocating stack space for threads, it creates new idle threads at intervals. The interval is currently half a second, although it could change in future versions of the .NET Framework.

If an application is subject to bursts of activity in which large numbers of thread pool tasks are queued, use the SetMinThreads method to increase the minimum number of idle threads. Otherwise, the built-in delay in creating new idle threads could cause a bottleneck.

Note:
Unnecessarily increasing the number of idle threads can also cause performance problems. Stack space must be allocated for each thread. If too many tasks start at the same time, all of them might appear to be slow. Finding the right balance is a performance-tuning issue.

When the thread pool reuses a thread, it does not clear the data in thread local storage or in fields that are marked with the ThreadStaticAttribute attribute. Therefore, data that is placed in thread local storage by one method can be exposed to any other method that is executed by the same thread pool thread. A method that accesses a field that is marked with the ThreadStaticAttribute attribute could encounter different data depending on which thread pool thread executes it.

Starting with the .NET Framework version 2.0 Service Pack 1, the throughput of the thread pool is significantly improved for applications that make heavy use of small thread pool tasks. These applications will see improvements in three areas:

• queuing tasks,
• dispatching thread pool threads, and
• dispatching I/O completion threads.

To use this functionality, your application should target the .NET Framework version 3.5. For more information, see .NET Framework 3.5 Architecture.

More Samples, .NET Snippets

use the Invoke Method of the ISynchronizeInvoke Interface:

CODE

#region UpdateStatus
private static void UpdateStatus(string msg, string status)
{
//create our Object Array
object[] items = new object[2];
//populate our array with the parameters
//passed to our method
items[0] = msg;
items[1] = status;
//call the delegate
_synch.Invoke(_status, items);
}
#endregion

Using this we can update any UI component in the main thread without worrying if the call is being made from the proper thread. So as you can see, there is a thread-safe way to share data between threads. Though we aren’t using data, we’re simply updating a component on the UI with the status of our background thread. This process can now rune while leaving the UI in a usable state.

Basic Instincts: Implementing Callbacks with a Multicast Delegate.

is a follow-up to the December 2002 installment in which I introduced the basic concepts and programming techniques associated with delegates. I am going to assume you have already read that column and that you are familiar with the fundamentals of programming delegates. For example, you should know how to define a delegate type, how to create a delegate object that’s bound to a handler method, and how to execute the handler method by calling the delegate object’s Invoke method.

How to: Combine Delegates (Multicast Delegates)(C#).

This example demonstrates how to compose multicast delegates. A useful property of delegate objects is that they can be assigned to one delegate instance to be multicast using the + operator. A composed delegate calls the two delegates it was composed from. Only delegates of the same type can be composed.

The - operator can be used to remove a component delegate from a composed delegate.

Example

C#

delegate void Del(string s);

class TestClass
{
    static void Hello(string s)
    {
        System.Console.WriteLine("  Hello, {0}!", s);
    }

    static void Goodbye(string s)
    {
        System.Console.WriteLine("  Goodbye, {0}!", s);
    }

    static void Main()
    {
        Del a, b, c, d;

        // Create the delegate object a that references 
        // the method Hello:
        a = Hello;

        // Create the delegate object b that references 
        // the method Goodbye:
        b = Goodbye;

        // The two delegates, a and b, are composed to form c: 
        c = a + b;

        // Remove a from the composed delegate, leaving d, 
        // which calls only the method Goodbye:
        d = c - a;

        System.Console.WriteLine("Invoking delegate a:");
        a("A");
        System.Console.WriteLine("Invoking delegate b:");
        b("B");
        System.Console.WriteLine("Invoking delegate c:");
        c("C");
        System.Console.WriteLine("Invoking delegate d:");
        d("D");
    }
}

Output

Invoking delegate a:
  Hello, A!
Invoking delegate b:
  Goodbye, B!
Invoking delegate c:
  Hello, C!
  Goodbye, C!
Invoking delegate d:
  Goodbye, D!

See Also

Reference

MulticastDelegate

Concepts

C# Programming Guide
Events (C# Programming Guide)

Quick factories using delegates

Delegates are very useful to act as object factories. In the example below, it is used to allow the function to build another object based on it, while keeping the function unaware about types.

namespace Microsoft.MSDN.Wiki.Samples
{
    class Vehicle
    {
        int Price;
    }
    class Car : Vehicle
    {
        protected Car(string model, int passengers)
        {
        }
        public static Car Create(string model, int passengers)
        {
            // Perform some check and create a new instance
            return new Car(model, passengers);
        }
        public Vehicle FakeClone(Vehicle src)
        {
            // Cars are special in our example, and will not be cloned
            // return the original object instead
            return src;
        }
    }
    class Bus : Vehicle
    {
        private string model;
        private int passengers;
        private int cargo;
        protected Bus(string model, int passengers, int cargo)
        {
            // Assign variables
            this.model = model;
            this.passengers = passengers;
            this.cargo = cargo;
        }
        public static Bus Create(string model, int passengers, int cargo)
        {
            // TODO: Perform some check and create a new instance
            return new Bus(model, passengers, cargo);
        }
        public Vehicle CloneSpecial(Vehicle src)
        {
            // TODO: Perform some check and clone the instance
            Bus bus = (Bus)src;
            return new Bus(bus.model, bus.passengers, bus.cargo);
        }
    }
    class Program
    {
        public delegate Vehicle CreateNewVehicle(Vehicle src);
        public void ProcessBus(Bus bus)
        {
            ProcessVehicle(bus, bus.CloneSpecial);
        }
        public void ProcessCar(Car car)
        {
            ProcessVehicle(car, car.FakeClone);
        }
        public void ProcessVehicle(Vehicle source, CreateNewVehicle factory)
        {
            // This function might need to clone the vehicle. Use the delegate
            // so the caller can control the creation
            bool needToCloneObject = true;
            if (needToCloneObject)
            {
                Vehicle newObj = factory(source);
                if (newObj != null)
                {
                    // Now we have a new object to work with
                }
            }
        }
        [STAThread]
        public static void Main(string[] args)
        {
        }
    }
}

As you can see we can take the knowledge about the creation logic outside the function when pure inheritance does not work (e.g. having a generic Clone() function in Vehicle) or when the logic changing depending on the context. For example, you might need to clone in a specific way on one condition and clone it differently on another condition.

Anonymous delegates can be used to simplify context changes

you can also use an anonymous delegate to simplify the cloning process. But remember that this technique is not constrained to cloning. That was just the motivation for the example.

namespace Microsoft.MSDN.Wiki.Samples
{
    class Vehicle
    {
        int Price;
    }
    class Car : Vehicle
    {
        protected Car(string model, int passengers)
        {
        }
        public static Car Create(string model, int passengers)
        {
            // Perform some check and create a new instance
            return new Car(model, passengers);
        }
    }
    class Bus : Vehicle
    {
        private string model;
        private int passengers;
        private int cargo;
        protected Bus(string model, int passengers, int cargo)
        {
            // Assign variables
            this.model = model;
            this.passengers = passengers;
            this.cargo = cargo;
        }
        public static Bus Create(string model, int passengers, int cargo)
        {
            // TODO: Perform some check and create a new instance
            return new Bus(model, passengers, cargo);
        }
        public Bus Clone()
        {
            return new Bus(this.model, this.passengers, this.cargo);
        }
    }
    class Program
    {
        public delegate Vehicle CreateNewVehicle(Vehicle src);
        public void ProcessBus(Bus bus)
        {
            ProcessVehicle(bus, delegate(Vehicle src)
            {
                // Perform some special logic
                return ((Bus)src).Clone();
            });
        }
        public void ProcessCar(Car car)
        {
            ProcessVehicle(car, delegate(Vehicle src)
            {
                // Cars are not cloned in this situation
                return src;
            });
        }
        public void ProcessVehicle(Vehicle source, CreateNewVehicle factory)
        {
            // This function might need to clone the vehicle. Use the delegate
            // so the caller can control the creation
            bool needToCloneObject = true;
            if (needToCloneObject)
            {
                Vehicle newObj = factory(source);
                if (newObj != null)
                {
                    // Now we have a new object to work with
                }
            }
        }
        [STAThread]
        public static void Main(string[] args)
        {
        }
    }
}

Please see my previous post

Mutual Exclusion / Critical Section aka Mutex

• “Mutual exclusion” — scheduling threads so that only one thread at a time executes a critical section of code. Essentially, keeping the threads from interfering with each other. Avoid reader/writer and writer/writer conflicts on shared memory between threads.
• Critical section — a section of code that manipulates shared memory, and so must not be run by multiple threads at the same time.
• “Race condition” — a piece of code with unaddressed concurrency problems, and so depending on how the threads happen to schedule each time, you may get different output.

Coordination

• “Coordination” — managing the schedules of multiple threads so they can start/pause/etc. to mesh their schedules.
• Typically this centers on handing information from one thread to another, or signaling one thread that another thread has finished doing something.
• We will use join() and Semaphores for this

Example:

class Pair {
   private int a, b;
   public Pair()
   {
      a = 0;
      b = 0;
   }
   // Returns the sum of a and b. (reader)
   public int sum()
   {
      return(a+b);
   }
   // Increments both a and b. (writer)
   public void inc()
   {
      a++;
      b++;
   }
}

Reader/Writer conflict:

• thread1 runs inc() while thread2 runs sum() on the same object.
• In that case, the sum() thread could get an incorrect value if the inc() is halfway done.
• In part, this happens because the lines of inc() and sum() interleave
• Note that this is only a problem if the threads are executing against the same object so they both touch the same memory.

Writer/Writer conflict:

• thread1 runs inc() while thread2 runs inc() on the same object.
• The two inc()’s can interleave to mess up the object state

Object Lock + Synchronized Method

• In Java, every object has a “lock”
• A “synchronized” method respects the lock of the receiver object.
• For a thread to execute a synchronized method against a receiver, it first obtains the lock of the receiver. The lock is released when the method exits
• If the lock is already held by another thread, the calling thread blocks (efficiently) waiting for the other thread to exit and so make the lock available.
• In this way, multiple threads, in effect, take turns on who can execute against the receiver — mutual exclusion.
• If a method is not synchronized, it will ignore the lock and just go ahead.

Example:

In our example above, we solve the race conditions by declaring the critical section as synchronized — they respect the lock on the receiver object.

public synchronized int sum()
{
   return(a+b);
}
// Increments both a and b. (writer)
public synchronized void inc()
{
   a++;
   b++;
}

Synchronized(obj) {…} Block

• A variant of the synchronized method.
• Acquire/Release lock for a specific object.
• Allows us to specify exactly where to lock, and with what object
• A little slower, a little less readable

Below is an implementation of a File based mutex in Java

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
import java.nio.channels.OverlappingFileLockException;
import java.util.Date;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

/**
 * A file based mutual exclusion lock that allows to acquire and release the underlying
 * resource. On acquiring the lock the file is created with the given contents. The
 * lock is free on construction.
 *
 * The lock is also associated with a maximum unused period. If the lock is not
 * updated during the given period, any other thread waiting to acquire the lock
 * will be given the lock.
 *
 * The lock is said to be acquired only when the file exists and file last used
 * timestamp is within unusedMsec. permitted
 *
 * The read and write operation to the mutex file are synchronized across
 * different process by encapsulating that with a create & lock operation
 * on a separate file. So before a process can acquire/read mutex (update mutex
 * content & timestamp) , it needs to create & lock another predefined
 * file and also delete the lock file after the operation on mutex are done. This is
 * to ensure that multiple processes/threads dont acquire the mutex at the same time
 */
public class FileMutex {
    /** The lock file name **/
    private String mutex = System.getProperty("java.io.tmpdir") + "app.lck";

    /** Msecs. file can be left unused before another thread acquires the lock File **/
    private long unusedMSec;

    /** file contents telling which component/application is holding the lock **/
    private String mutexContent;

    /** FileName of the lock for mutex **/
    private String lockOnMutex = mutex + ".lck";

    public static final long ONE_SECOND = 1000;
    public static final long ONE_MINUTE = 60 * ONE_SECOND;

    /**
     * @param lockFileContent
     * @param unusedMSec
     */
    public FileMutex(String lockFileContent, long unusedMSec) {
        this.mutexContent = lockFileContent;
        this.unusedMSec = unusedMSec;
    }

    public void acquire() throws InterruptedException {
        try {
            while (isInuseMutex())
                Thread.sleep(ONE_SECOND);
            createMutex();
        } catch (Exception e) {
            throw new InterruptedException("Error acquiring lock: " + e.getMessage());
        }
    }

    public synchronized void release() {
        releaseMutex();
    }

    public boolean tryAcquire() throws InterruptedException {
        try {
            if (!isInuseMutex()) {
                boolean created = createMutex();
                return created;
            }
        } catch (Exception e) {
            throw new InterruptedException("Error acquiring lock: " + e.getMessage());
        }
        return false;
    }

    public boolean update() {
        createMutex();
        return true;
    }

    /**
     * @return
     */
    private boolean isInuseMutex() {
        File lockOnMutexFile = new File(lockOnMutex);
        FileChannel channel = null;
        FileLock lock = null;
        try {
            channel = new RandomAccessFile(lockOnMutexFile, "rw").getChannel();
            lockOnMutexFile.createNewFile();
            try {
                lock = channel.tryLock();
            } catch (OverlappingFileLockException e) {
                // File is already locked in this thread or virtual machine
                return true;
            }

            File mutexFile = new File(mutex);
            if (!mutexFile.exists()) {
                return false;
            }

            Date lockLastUsedDate = new Date(mutexFile.lastModified());
            Date oldestAllowedLockUsedDate =
                         new Date(System.currentTimeMillis() - unusedMSec);
            if (lockLastUsedDate.before(oldestAllowedLockUsedDate)) {
                return false;
            }
        } catch (Exception e) {
            System.out.println("Error acquiring lock: " + e.getMessage());
        } finally {
            try {
                if (lock != null)
                    lock.release();
            } catch (IOException e) {
            }
            try {
                channel.close();
            } catch (IOException e) {
            }
            lockOnMutexFile.delete();
        }
        return true;
    }

    /**
     * Creates the lock file with the given contents.
     *
     */
    private boolean createMutex() {
        File lockOnMutexFile = new File(lockOnMutex);
        FileChannel channel = null;
        FileLock lock = null;
        try {
            channel = new RandomAccessFile(lockOnMutexFile, "rw").getChannel();
            lockOnMutexFile.createNewFile();
            try {
                lock = channel.tryLock();
            } catch (OverlappingFileLockException e) {
                // File is already locked in this thread or virtual machine
                return false;
            }

            File mutexFile = new File(mutex);
            mutexFile.createNewFile();
            FileWriter writer = new FileWriter(mutexFile);
            writer.append(mutexContent);
            writer.flush();
            writer.close();
        } catch (IOException e) {
            System.out.println("Error acquiring lock: " + e.getMessage());
        } finally {
            try {
                if (lock != null)
                    lock.release();
            } catch (IOException e) {
            }
            try {
                channel.close();
            } catch (IOException e) {
            }
            lockOnMutexFile.delete();
        }
        return true;
    }

    /**
     * Release the mutex .. that is Delete the underlying lock file, if it exists
     */
    private void releaseMutex() {
        File mutexFile = new File(mutex);
        mutexFile.delete();
    }
}

First of all…you can see an example here which may clarify your concept!

Their synchronization semantics are very different:

* mutexes allows serialization of access to a given resource i.e. multiple threads wait for a lock, one at a time, the thread owns the lock until it is done, most importantly only this particular thread can unlocks it.

* a binary semaphore is a counter with value 0 and 1, a task blocking on it until any task does a sem_post. The semaphore advertise that a resource is available, and it provides the mechanism to wait until it is signaled as being available.

As such one can see a mutex as a token passed from task to tasks and a semaphore as traffic red-light (it signals someone that it can proceed).
There is a lot of confusion around these two concepts. They permit the same kind of feature i.e. synchronization but as I explained with different semantics.

Semaphores are usually more expensive to use than mutex, but they provide you sometimes more reliability. For example, let’s say you have two threads, thread A and thread B trying to serialize access to resource R.

The mutex way would be;

step 00 – create a lock object
step 01 – A request the lock
step 02 – B request the lock
step 03 – A grab the lock
step 04 – A uses R
step 05 – A release the lock
step 06 – B grab the lock
step 07 – B uses R
step 08 – A request the lock
step 09 – B release the lock
step 10 – A grab the lock

Q: What happens if thread B returns between step 7 and step 9 ?
A: step 10 may never occurs …

In this scenario there is no way to recover, the mutex is an opaque object and you are stuck in a deadlock situation.

With semaphores things are a little bit different, you can make step 10 happen even if your thread leaves early.

Since people are confused by semaphores or misuse them more frequently, here are a few examples on how to use them.

Serializing access using a semaphore

Note that when using a binary semaphore to serialize access to a resource, you need to initialize it with the value 1, meaning resource available.

Requesting the lock is done by calling sem_wait() on it, and releasing the lock is done by calling sem_post():

// thirst get a semaphore and initialize it with the value 1
sem_init( … )
…

// lock the resource
sem_wait( …)

… do some stuff …

// release the resource
sem_post( … )

The consumer / producer model

if you initialize the semaphore withe the value 0 and call sem_wait on it, the semantic is different, you are waiting for some one to signal you that there is some job to do.
In this scenario, the semaphore is not used to serialize access to a particular resource, it is used as signaling mechanism.
A consumer is waiting for a consumer to signal him that some data is available or that there is work to do.

very simple and effective, that’s why semaphore are used so frequently for producer/consumer work flows.

// thirst get a semaphore and initialize it with the value 0
sem_init( … )
…

while( true )
{
// wait for some job to do
sem_wait( …)

… work hard on it …
}

// yep there is no sem_post here: the producer, that gives this task
// work to do is supposed to be the one calling sem_post()

check your platform semaphore implementation, sem_post(), sem_wait() and sem_init() are the API to use POSIX semaphores.

Your platform API may be different, but the concepts are the same.

Find definitions for eight terms and concepts used in threaded programming:

1. Thread Synchronisation

Thread synchronisation is also referred to as process synchronisation. “Process synchronization refers to the coordination of simultaneous threads or processes to complete a task in order to get correct runtime order and avoid unexpected race conditions” (Wikipedia – Synchronization, n.d.).

2. Locks

“a lock is a synchronization mechanism for enforcing limits on access to a resource in an environment where there are many threads of execution. Locks are one way of enforcing concurrency control policies” (Wikipedia – Lock, n.d.).

3. Deadlocks

The best definition I could find for a deadlock is an example when programming in Java. “Deadlocks can occur in Java because the synchronized keyword causes the executing thread to block while waiting for the lock, or monitor, associated with the specified object. Since the thread might already hold locks associated with other objects, two threads could each be waiting for the other to release a lock; in such a case, they will end up waiting forever” (Goetz, 2001).

4. Semaphores

“a semaphore is a protected variable or abstract data type which constitutes the classic method for restricting access to shared resources such as shared memory in a multiprogramming environment. A counting semaphore is a counter for a set of available resources, rather than a locked/unlocked flag of a single resource. It was invented by Edsger Dijkstra. Semaphores are the classic solution to preventing race conditions in the dining philosophers problem, although they do not prevent resource deadlocks” (Wikipedia – Semaphore, n.d.).

5. Mutex (mutual exclusion)

“Mutual exclusion (often abbreviated to mutex) algorithms are used in concurrent programming to avoid the simultaneous use of a common resource, such as a global variable, by pieces of computer code called critical sections.

Examples of such resources are fine-grained flags, counters or queues, used to communicate between code that runs concurrently, such as an application and its interrupt handlers. The problem is acute because a thread can be stopped or started at any time.

To illustrate: suppose a section of code is altering a piece of data over several program steps, when another thread, perhaps triggered by some unpredictable event, starts executing. If this second thread reads from the same piece of data, the data, which is in the process of being overwritten, is in an inconsistent and unpredictable state. If the second thread tries overwriting that data, the ensuing state will probably be unrecoverable. These shared data being accessed by critical sections of code, must therefore be protected, so that other processes which read from or write to the chunk of data are excluded from running.

A mutex is also a common name for a program object that negotiates mutual exclusion among threads, also called a lock” (Wikipedia – Mutual exclusion, n.d.).

6. Thread

“a thread of execution results from a fork of a computer program into two or more concurrentlytasks. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources” (Wikipedia – Thread, n.d.).

7. Event

An event is a sensor output, user action, or a message from other programs or threads (Wikipedia – Event-driven programming, n.d.).

8. Waitable timer.

Waitable timers were introduced with Windows 98 and Windows NT 4.0. and they were designed to work with threads that need to block for some times. These timers are kernel objects which are signaled in the specified time, or in regular time intervals. They can specify the callback function (actually, an asynchronous procedure call, or APC) which gets called when timer gets signaled. This callback function is usually called a completion routine.

In order to enable execution of the completion routine, the thread must be in alertable state (executing SleepEx(), WaitForSingleObjectEx() , WaitForMultipleObjectsEx(), MsgWaitForMultipleObjectsEx() , SignalObjectAndWait() functions). In practice, this means that our main thread will be blocked while we are using waitable timers.

To start working with a waitable timer, we must open an existing timer, or create the new one (Trifunovic, 2001).

References

Wikipedia – Synchronization (n.d.). Retrieved 11th May 2009 from http://en.wikipedia.org/wiki/Synchronization_(computer_science)

Wikipedia – Lock (n.d.). Retrieved 11th May 2009 from http://en.wikipedia.org/wiki/Lock_(computer_science)

Goetz, B. (2001) Avoid Synchronization Deadlocks. Retrieved 11th May 2009 from http://www.javaworld.com/javaworld/jw-10-2001/jw-1012-deadlock.html

Wikipedia – Semaphore (n.d.). Retrieved 11th May 2009 from http://en.wikipedia.org/wiki/Semaphore_(programming)

Wikipedia – Mutual exclusion (n.d.). Retrieved 11th May 2009 from http://en.wikipedia.org/wiki/Mutex

Wikipedia – Thread (n.d.). Retrieved 11th May 2009 from http://en.wikipedia.org/wiki/Thread_join

Wikipedia – Event-driven programming (n.d.). Retrieved 12th May 2009 from http://en.wikipedia.org/wiki/Event-driven_programming

Trifunovic (2001). CodeProject: Timers Tutorial. Retrieved 12th May 2009 from http://www.codeproject.com/KB/system/timers_intro.aspx#WaitableTimers

Problemas de acessores que lêem e escrevem em um dado recurso são bastante comuns no dia-dia, se considerarmos a importância de bancos de dados na vida moderna. O programa abaixo implementa um algoritmo para sincronizar operações de leitura e escrita de maneira segura. Esse algoritmo, entretanto, privilegia os acessores leitores, que passam a dominar o recurso assim que um esteja utilizando.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include
<pthread.h>
#include <semaphore.h>

#define ESCRITORES 1
#define LEITORES 5
#define VEL_CHEGADA 2

pthread_mutex_t mutex_variavel_ql;
sem_t banco;
int quer_ler = 0;

void ler_banco_dados()
{

	printf("Lendo banco de dados...n");
	sleep(rand()%5 + 1);
}

void processar_dados()
{
	printf("Processando dados...n");
	sleep(rand()%5 + 1);

}

void produzir_dados()
{
	printf("Produzindo dados...n");
	sleep(rand()%5 + 1);
}

void escrever_no_banco()
{
	printf("Escrevendo no banco...n");
	sleep(rand()%5 + 1);
}

void* leitor(void *args)
{
		sleep(2);
		pthread_mutex_lock(&mutex_variavel_ql);
		quer_ler++;
		printf("%d threads querem ler do banco.n", quer_ler);
		if(quer_ler == 1)
			sem_wait(&banco);
		pthread_mutex_unlock(&mutex_variavel_ql);
		ler_banco_dados();
		pthread_mutex_lock(&mutex_variavel_ql);
		quer_ler--;
		if(quer_ler == 0)
			sem_post(&banco);
		pthread_mutex_unlock(&mutex_variavel_ql);
		processar_dados();
		pthread_exit(NULL);
}

void* escritor(void *args)
{
		sleep(2);
		produzir_dados();
		sem_wait(&banco);
		escrever_no_banco();
		sem_post(&banco);

} 

int main()
{

	int i,res;
	pthread_t leitor_thread[LEITORES];
	pthread_t escritor_thread[ESCRITORES];
	srand(time(NULL));

	res = pthread_mutex_init(&mutex_variavel_ql,NULL);
	if(res != 0)
	{
		perror("Não foi possível criar mutex!");
		exit(EXIT_FAILURE);
	}

	res = sem_init(&banco,0,1);
	if(res != 0)
	{
		perror("Não foi possível criar semáforo!");
		exit(EXIT_FAILURE);
	}

	for(i = 0; i < ESCRITORES; i++)
	{
		res = pthread_create(&escritor_thread[i],NULL,escritor,NULL);
		if(res != 0)
		{
			perror("Não foi possível criar thread!");
			exit(EXIT_FAILURE);
		}
	}

	for(i = 0; i < LEITORES; i++)
	{
		res = pthread_create(&leitor_thread[i],NULL,leitor,NULL);
		if(res != 0)
		{
			perror("Não foi possível criar thread!");
			exit(EXIT_FAILURE);
		}
	}

	for(i = 0; i < ESCRITORES; i++)
	{
		pthread_join(escritor_thread[i],NULL);
		if(res != 0)
		{
			perror("Thread join failed");
			exit(EXIT_FAILURE);
		}
	}

	for(i = 0; i < LEITORES; i++)
	{
		pthread_join(leitor_thread[i],NULL);
		if(res != 0)
		{
			perror("Thread join failed");
			exit(EXIT_FAILURE);
		}
	}

	sem_destroy(&banco);
	pthread_mutex_destroy(&mutex_variavel_ql);

	exit(EXIT_SUCCESS);
}

Resultado do programa:

Hi,

.Net framework’s System.Threading namespace provides Mutex class which is a synchronization primitive that can also be used for interprocess synchronization. Using this mutex object we can also control running of multiple instances of a same application. Here is the sample code to do that. The code should be wriiten on the starting point of the application obviously for a windows or console application it would be the Main() function.

static class Program
{
///<summary>
/// The main entry point for the application.
///</summary>
[STAThread]
static void Main()
{
bool instanceCountOne = false;
using (Mutex mtex = new Mutex(true, “MyRunningApp”, out instanceCountOne))
{
if (instanceCountOne)
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new Form1());
mtex.ReleaseMutex();
}
else
{
MessageBox.Show(“An application instance is already running”);
}
}
}
}

Note: Replace the application name with your corresponding application name. Include the namespace System.Threading.

Happy coding…

yenkay

The difference between a mutex and a semaphore is something I often forget. So while this is fresh in my mind…

Mutex: Locks a critcal section of code when in use by a thread, and unlocks it only when that thread is complete. It works on a one out, one in system but only ever allows one thread at any time. If a thread tries to access the mutex when it is locked, it is forced to wait in a queue until it is unlocked.

Semaphore: A semaphore is an extension of a mutex and also locks a critical section of code when in use by a thread. However with a semaphore you can specify how many threads are allowed simultaneous access. For example, your semaphore allows 4 simultaneous threads. Each time a thread uses the semaphore, the semaphore decreases its counter by 1 until it is zero, at which point no more threads are allowed access and are forced to queue. When a thread finishes using the semaphore, the semaphore increaes it’s counter by one to allow the next thread in the queue access.

This site provided me with the best explanation: http://geekswithblogs.net/shahed/archive/2006/06/09/81268.aspx

Also, put even simpler by this site http://www.cs.tau.ac.il/~hayim/courses/os/lecture_notes/lecture5/lecture.html :

“A mutex allows one thread inside the critical section, a semaphore allows n threads in a critical section (when the number n is given as a parameter on the initialization).”

Si les da este error cuando tratan de navegar un site hecho en ASP.net 2, con Visual Studio 2005, sigan estos pasos para solucionarlo:

1. Si tenes el VS abierto, cerralo
2. Anda a la carpeta de archivos temporales de ASP.NET:  <Windows dir>\Microsoft.Net\Framework\v2.0<nro. del framework>\Temporary ASPNET pages
3. Borra la carpeta de tu aplicacion (o todas, si tenes ganas — son temporales)
4. Corre iisreset para reiniciar el IIS
5. Abri un browser y navega tu sitio
6. Ahora abri el Visual Studio.

Parece ser un problema con los permisos en algunas carpetas internas.

Решился написать серию микро-статей о программировании под Win32. Статья про Мьютексы будет первая.

Частенько сталкивался с вопросом новичков в программировании – «Как сделать так , чтобы запускалась только одна копия программы?». Все очень просто – как один из способов, нужно использовать Мьютексы (англ. Mutex – объект взаимного исключения). Вообще-то Мьютексы используются для синхронизации потоков, но они очень хорошо подходят и для много другого (когда человек пишет троянца или кейлогер, то нужно быть стопроцентно уверенным, что запустится только одна копия программы, иначе могут быть конфликты).

Так как основным языком программирования для меня является C++, то и пример того, как надо использовать Мьютексы я буду реализовывать на этом языке. Пример очень простой. Все объяснения находятся внутри кода. Алгоритм работы: создаем мьютекс с определенным именем; если мьютекс создался (не вернул код ошибки), то это значит, что запущена только одна копия программы и поэтому работа программы продолжается; если мьютекс не был создан (вернул код ошибки), то это значит, что уже запущена одна копия программы и работу второй копии программы нужно завершить (выход из программы через “..return 0;”).

// Mutex.cpp : Defines the entry point for the console application.
//

#include “stdafx.h”
#include <iostream>
#include <tchar.h>
#include <windows.h>

bool Alone()
{
HANDLE hMutex = CreateMutex(0,TRUE,”UniqueMutex!”);

if(GetLastError() == ERROR_ALREADY_EXISTS)
{
CloseHandle(hMutex);
return false;
}
return true;
}

int _tmain(int argc, _TCHAR* argv[])
{
if(!Alone())
return 0;
else
{
for(int i=0;i<10;i++)
{
std::cout << “I am alone ” << std::endl;

Sleep(2000);
}
}
return 0;
}

I always get cranky when reading discussions about Threads. For some reason a large group of programmers think coding threads is hard. And with every news article that touches the issue, the discussion reignites.

Yes, threads can be hard and complicated at times, especially when dealing with locks, mutex’s, semaphores, etc. but when you rewind your way of thinking there’s some easy guidelines that will make threading a second nature.

First one; think object-oriented. Thinking in a straight serialized way like the way for example the C-language is seen, limits your actual options. You need to think in separate units. When dealing with classes it’s easier to separate functionalities and make them function on their own without depending on others.

Second one; every application that has a user-interface is an opportunity to use threads. Everything the user clicks on can potentially spawn a new thread or reuse one.

The slight downside to a GUI application, is that some GUI’s don’t really like multithreading. GTK is one of them. The Win32 API is generally really cool about it, and Cocoa can do it as well. In environments like GTK, you have to think of a messaging system that hands over feedback in-between GUI thread iterations.

A simple system I have come up with for GTK, is to message and buffer function-pointers, so an entire custom function can handle all of the GUI changes.

Third; don’t over-think locking mechanisms nor mutexes. Think of ways to avoid communication between multiple threads that are running by giving the data they need in the beginning. If you simply have to, find a way to just use a limited amount of functions and centralize them so you can encase them with locks.

If you want to share an object filled with data, the easiest thing you can do is to make a copy with one synchronized function.

Fourth; think generic. If you’ve written a piece of code before, you can separate it into a new function or even a new object given specific variables. If you can do that, then you’re a step closer to figuring out if that piece of code can be threaded.

Fifth; Don’t write something you’re not yet done with. You see it all the time, even I succumb to the temptation quite often – functions that hint at something that’s not yet ready. Functions like newRecord() that append an empty object to an array, while you still have to fill in the values. Or public record counts that are incremented before the record has been added.

If you see a code snip like this;

void addNewValue( string myvalue ) {
   myValueCount++;
   resizeMyArrayTo( myValueCount );
   myValues[myValueCount-1] = myvalue;
   myvalue = myvalue + " my appendix";
}

You’re in serious need of a rewrite to this;

void addNewValue( string myvalue ) {
   myvalue = myvalue + " my appendix";
   resizeMyArrayTo( myValueCount + 1 );
   myValues[myValueCount] = myvalue;
   myValueCount++;
}

If you’re careful enough to make your code depend on variable assignments that have the least chance of being split into separate opcodes, then you don’t even have to use locks in most threaded environments. So sure, myValueCount++ will probably be split into 2 opcodes, one that increments a register, and one that assigns the value to the actual variable. So eventhough in an extremely rare case that those two opcodes get split, and at the same time another item wants to change the same variable, your variable will be off by 1, but at least your application won’t crash. In the first example your application has the potential to crash, as the variable that’s incremented before the actual value has been appended will give other threads the opportunity to access a non-existing object.

The same thing happens when you use things like this;

function doSomething( int i ) {
   CMyValue *obj = myTable->record[i]->newValue();
   obj->datatype = STR;
   obj->value = "this is my value"
}

Make your code prepare the object before you give it to someone else;

function doSomething( int i ) {
   CMyValue *obj = new CMyValue();
   obj->datatype = STR;
   obj->value = "this is my value"

   myTable->record[i]->appendValue( obj );
}

Threading is way of thinking, and once you’re able to think in that direction, it won’t seem so hard anymore.