Earlier in the year I announced the shortlist for this year’s EGaming Review Awards, for the category of ‘Mobile Operator of the Year’ – well last night was the night in question, and the winner was Probability.

Special mention was also given to Betfair, for their relatively new mobile offerings.

The full list of winners is as follows: -

• Operator of the Year – Bwin
• Innovation of the Year – Gamesys (Jackpotjoy)
• Online Marketing Campaign of the Year – PKR
• Affiliate Program of the Year – Virgin Games
• Offline Marketing Campaign – Betclick
• Mobile Operator of the Year – Probability (Ladylucks)
• Socially Responsbile Operator of the Year – Betsson
• Asian Operator of the Year – SBO Bet
• Latin American Operator of the Year – Betboo
• Bingo Operator of the Year – Cashcade (Foxy Bingo)
• European Sports Betting Operator of the Year – Unibet
• UK Sports Betting Operator of the Year – Betfair
• Casino Operator of the Year – 888
• Poker Operator of the Year – Party Gaming
• Financial Betting Operator of the Year – Bet on Markets
• Rising Star of the Year – Gaming Media Group
• European Live Gaming Operator of the Year – Unibet
• Asian Live Gaming Operator of the Year – VCBetAsia

What do you think of the decision? Would you have nominated the same companies for the awards? Would love to hear your views on who should have (and shouldn’t have) won last night!

This is a further discussion on the chi-square distribution. The gamma density function is . The parameter is the scale parameter and is the shape parameter. Thus the chi-square density function is of the form where is a positive integer. This is the chi-square distribution with degrees of freedom, denoted by .

The goal is to establish these basic facts. If follows a normal distribution with zero mean and variance , that is , then the density of is the gamma density . If , then has the chi-square density function (chi-square distribution with one degree of freedom). As a result, we have some corollaries about sum of squares of normal variables.

Theorem 1. If , then has the gamma density function .

Proof. The density function of is . We derive the distribution function and we obtain the density function .

With a change of variable using , we have the following:

We then differentiate to get the density function:

Note that . Thus is the gamma density .

Corollary 1. If , then has the chi-square density function . That is, has a chi-square distribution of one degree of freedom, .

Corollary 2. If are independent standard normal random variables, then the sum has the chi-square density function . That is, has a chi-square distribution with degrees of freedom, .

Corollary 3. If are independent normal random variables with the common distribution , then has a chi-square distribution with degrees of freedom.

Comment. Corollary 2 and Corollary 3 are possible because the independent sum of chi-square distributions is a chi-square distribution. This can be seen using convolution of gamma densities (see this post).

Exercise 7.12: Let be the sum of independent rolls of a fair dice. Show that, for any , .

Solution: Let be a Markov chain on state space consist of states: , where the chain reaches state if and only if .
The transition matrix is for and . The claim is equivalent to

We know that if a Markov chain has an irreducible, aperiodic transition matrix and an invariant distribution , then for all of the state space.

We will prove the defined above is irreducible, aperiodic and then find the invariant distribution of (it will turn out that as we need).

Exercise 7.21: Consider a Markov chain on the states , where for we have and . Also, and . This process can be viewed as a random walk on a directed graph with vertices , where each vertex has two directed edges: one that returns to and one that moves to the vertex with the next higher number(with a self-loop at vertex ). Find the stationary distribution of this chain.(This example shows that random walks on directed graphs are very different than random walks on undirected graph).

Solution:

Complex decision making is a vital project management skill; required not only by the project manager but also by the project’s sponsor and client / customer among others.

Some of the key areas involving complex decisions include risk management, many aspects of planning (particularly optimising choices) and dealing effectively with issues and problems in a range of areas from scope and quality to cost and performance.

There is an underlaying assumption in project management (derived from traditional scientific management) that decisions will be based on a rational assessment of the situation to optimise outcomes. Unfortunately this is not true! As complexity increases assuming a ‘rational decision making paradigm’ becomes increasingly unrealistic. Human decision makers become ‘predictably irrational’.

Understanding the built in biases and ‘predictable irrational’ decision making processes used by people confronted with complex decisions can help managers requiring optimised decisions to craft strategies to minimise suboptimal outcomes. But where can busy project managers access this information?

I have just finished reading the most amazing paper on the subject that canvases the whole spectrum from risk aversion to behavioural economics in a practical, easy to read format; and it is free!

Behavioural economics and complex decision making: implications for the Australian tax and transfer system has been written by Andrew Reeson and Simon Dunsttall of the Australian national science agency, CSIRO. The report was commissioned by the ‘Henry Review’ into the Australian taxation system and is published on their web site. Whilst you can safely skip the last section which focuses on applying the knowledge to our tax system. The preceding 7 sections are focused on how people make complex decisions in any sphere and are just as relevant to complex project decisions as to complex investment and taxation decisions.

You can download this free resource from the review panel’s website: download the paper (a copy is also on the Mosaic web site on the assumption the Government site is temporary and will close once the Henry Review has reported: download from Mosaic).

If you find the report useful and you don’t live in Australia, you can buy the next Australian you meet a beer; it was his or her taxes that paid for this amazingly useful report. I know I will be keeping my copy handy for a very long time to come.

This week, I will be presenting live from Australia the final session of the Fall PMI College Of Scheduling (COS) Wednesday Webinar Series: Scheduling in the Age of Complexity. This hour-long event will provide key insights for better scheduling from a personal level: What is the role of the scheduler and what is our future?

The PMI-COS Fall series is designed to bring highlights from the 6th Annual Scheduling Conference held in Boston, MA earlier this year.  Archived presentations are available at http://www.pmicos.org/ondemandlearning.asp if you find them of interest, why not sign up for the College?

The Featured Presentation:   Scheduling in the Age of Complexity

Scheduling was developed as a computer based modelling process at a time when ‘command and control’ was the dominant management paradigm. The mathematical precision of the early scheduling calculations were somehow translated into certain project outcomes. Today, the certainties are no longer so apparent. Most projects run late and uncertainty and complexity are starting to take center stage.

This paper identifies the key elements in Complexity Theory to suggest the real role of a schedule in ‘the age of complexity’. It concludes by recommending a way to re-establish the role of the scheduler in the successful delivery of projects in the 21st Century.

DATE:  Wednesday, December 2, 2009
TIME:   5:00pm EST (US Eastern Daylight Savings Time); Doors open at 4:45pm

LOCATION: http://pmi.acrobat.com/r31077016/

There is no dial-in telephone option for the presentation. All voice will be through the classroom platform.

The purpose of this post is to show that the gamma density functions are closed under the operation of convolutions. As a result of this fact, we observe that (1) the sum of two independent gamma variables with the same scale parameter has a gamma distribution, (2) the independent sum of exponential variables with identical mean has an Erlang distribution, and (3) the independent sum of chi-square distributions is a chi-square distribution. See this post for a basic discussion of the chi-square distribution.

Two parameters determine the gamma density – the scale parameter and the shape parameter . The following is the gamma density function:

.

The convolution relationship we want to establish is: Note that if the shape parameter is a positive integer, then we have an Erlang distribution. In this case, we can think of the distribution as a model for the total waiting time for changes to occur in a Poisson process and is the mean rate of arrival of changes. If , we have an expoential distribution. If and where is a positive integer, then is the chi-square density (the chi-square distribution with degrees of freedom,).

Convolutions

Let and be two continuous random variables with densities and , respectively. The convolution of the two densities is defined as:

If the two random variables and are independent, then is the density function of the sum .

If the random variables and only take on positive real numbers, then the convolution of the two densities is:

Convolutions of Gamma Densities

Given two gamma densities and , the convolution is again a gamma density: The following shows the derivation:

In the above integral, perform a change of variable using and we have the following:

Note that in the above derivation, we utilize the beta integral

Observations

If and are independent gamma variables with densities and , respectively, then the sum has density function . If are independent exponential random variables having common density , then the sum has density function , which is an Erlang density function.

With respect to the chi-square distribution, if has density function and has density function and if and are independent, then the sum has density function . In other words, the independent sum of two chi-square distributions and is .

This is Part 1 of a basic discussion on chi-square distribution. The purpose is to lay the ground work for future discussion. The chi-square distribution is a special case of the gamma distribution. Looking at it in the context of the Poisson-Exponential-Erlang-Gamma distributions helps put chi-square in prespectives (Erlang distribution is a special case of the gamma distribution). So this is the objective of this discussion. This is a very light discussion. The goal is to establish a good handle on the probability density function (pdf) of the chi-square distribution through the angle of Poisson-Exponential-Erlang-Gamma. This discussion may be helpful to someone who is studying for a test on this topic. In subsequent posts I will discuss chi-square in more details including applications.

The Poisson-Exponential-Erlang-Gamma Angle

Think of a Poisson process where the mean rate of arrival of changes is . Then is the mean time until the next change in this Poisson process. For a more detailed discussion on the Poisson process, see [1]. Now think of the following three random variables derived from the same Poisson process.

1. Poisson : the number of changes in the time interval . The mean number of arrivals is .
2. Exponential : the waiting time until the arrival of the first (or next) change. The mean waiting time is .
3. Erlang : the waiting time until the arrival of the change where is a positive integer. The mean waiting time is .

Here’s are the probability function and density functions:

1. Poisson : where with and .
2. Exponential : , , with and .
3. Erlang : , , with and .

Two parameters determine the Erlang distribution – (the number of arrivals of changes) and (the mean arrival rate in the Poisson process). The parameter is called the shape parameter since it determines the shape of the distribution (e.g. compare the shape of the density curves for and ). The parameter is called the scale parameter.

We now generalize Erlang to the gamma distribution. The pdf has the same form as the Erlang pdf, except that can now take on any positive real number and the factorial is replaced by the gamma function . Thus the gamma pdf is:

where and and .

As with Erlang, the gamma distribution is determined by two parameters and . The parameter is the shape parameter (the one in the exponent of in the pdf). The parameter is the scale parameter (the one in the exponent of in the pdf).

The Chi-square Distribution

The chi-square distribution is a gamma distribution where the shape parameter is where is a positive integer and the scale parameter is . When is an even integer, we get an Erlang distribution and we can think of the shape parameter as the number of arrivals in a Poisson process. The scale parameter would be the mean arrival rate in the Poisson process (so the mean time until the next change is 2). Familiarity of the Poisson-Erlang-Gamma relationship gives a good handle on the chi-square density function. Thus the following is the pdf of the chi-square distribution along with its mean and variance.

where and and .

If the random variable is distributed according to the above pdf, is said to have the chi-square distribution with degrees of freedom. The distribution is abbreviated by .

Consider an example based a Poisson process. Suppose the arrivals of cars at a toll booth is modeled by a Poisson process. Suppose on average cars arrive at the booth at the rate of 30 cars per hour. What is the probability that the toll booth worker will have to wait 16 minutes for the next 5 cars to arrive? Here, the mean rate of arrival per minute is . This is the scale parameter. The shape parameter is . Thus, if is the waiting time until the fifth car, this is . Using a Chi-square table in a probability and statistics text (e.g. [1]), . Thus .

One quick note about the Erlang distribution. The Erlang distribution plays an important role in queuing theory and is a member of a wider class of distributions called the phase-type distributions. Distributions in this class break down the total time of a process into phases, each having an exponential distribution. In an Erlang distribution, all of the phases are in series and have the same exponential parameter.

Reference

1. Hogg, R. V., Tanis, E. A., Probability and Statistical Inference, Second Edition, Macmillan, New York, 1983.

In the 17th century French salons, one of the favourite pastimes was gambling. One of the essayists of those times, Chevalier de Méré was playing the following game: roll a single die 4 times and bet of getting a 6. His reasoning was the following: if the chance of getting 6 for one roll is 1/6, then for 4 rolls it would be 4/6, so in the long run he would win more than he would lose.

When nobody wanted to play with him anymore because he was winning, he changed his game a little: roll 2 dice 24 times and bet on getting a double 6. I this case his reasoning was: the chance of getting a double 6 for one roll (of 2 dice) is 1/36. So by throwing 24 time the probability would be 24/36 = 4/6 so he would still win (although it would take a longer time) but the game being different he could convince people to play again with him. To his total surprise he started to lose in this game. Since he could not explain this he asked one of his friends, Blaise Pascal to help him with the mathematics.

So let’s see the correct way to estimate the probabilities in this (simple) case.

Game 1

First of all it is not correct to add probabilities in this case. If we have 2 events A and B then:

P(A or B) = P(A) + P(B) - P(A and B)
P(A and B) is 0 only if the events are mutually exclusive, that is if A happens then B will not happen. In our case that means if we get 6 on the first roll of die then it is not possible not get 6 on the second roll, which is obviously wrong.

The correct way to calculate the probability is the following:

P(to get at least one 6 in 4 rolls) =
1 - P(to not get 6 in any roll)
Now P(to not get 6 in any roll) is just P(not get 6 on the first roll AND not get 6 on the second roll AND ... so on). For 2 events:

P(A and B) = P(A)P(B)
only if the 2 events are independent that is A happening has no influence over B happening. In our case that means that getting 6 on the first roll has no influence over getting 6 on the second roll which is true.

So the correct probability for this game is:

P(win) = 1 - (5/6)(5/6)(5/6)(5/6) = 1 - (5/6)4 = 0.51775
Thus, Chevalier de Méré had almost 52% chance of winning (instead of his estimate of 66.6%) so we can say he was lucky in his probability estimation.

The expected value is a useful measure of the winnings/losses in the long run. Assuming that each participant is paying 1 dollar per game the expected value is:

E = 2·P(win) + (-2)·P(lose) = 0.071

Game 2

As seen from the previous game, in this case the correct probability is:

P(win) = 1 - (35/36)24 = 0.4914
and the expected value:

E = 2·P(win) + (-2)·P(lose) = -0.035

As indicated by practice (presumably) Chevalier de Méré would lose on this second game. If there’s a lesson we can learn from this is don’t estimate probability if you don’t know the rules, you will get it wrong even for simple 2 dice games.

To forecast the future, marketing leaders often look to the past. But the past isn’t always a very reliable gauge of future conditions. For proof, we need to look back to a day-in-the-life of a turkey, and implications of not preparing for possible “extreme” events around the corner.

First, let’s start with a fun exercise courtesy of Wilmott Magazine. Let’s look at damage estimates of earthquakes in California from 1970 to 1993 in the table below. Can you make an educated calculation of losses due to earthquakes in 1994?

Taking a look at the distribution of data, notice the low end is “0” and at high end, the most damage caused was “129”. So what’s your guess?

If you’re like me, you probably guessed wrong. Using the above numbers as an “anchor”, most people would reasonably assume that 1994’s earthquake was either an average of the above numbers or perhaps a bit higher than 129. Maybe you even threw out “129” as an outlier in the dataset. To be honest, I guessed around “200”.

The correct answer is “2217.2”! FEMA estimates that every year earthquake losses in the United States add up to $4.4 billion a year. But then, some extreme outliers can really skew that number, especially years like 1994 where just the Northridge Earthquake in California alone tallied $20B in damage!

Let’s get back to talking turkeys via a parable from Nassim Taleb, author of the “Black Swan”. Dr. Taleb reminds us that fat, dumb and happy is probably the best way to describe the life of a turkey. They’re fed and nurtured for three years straight. Day after day, they expect the same thing. But then, one fateful day arrives and the “life” of a turkey ends quite abruptly.

Can we accurately predict the future based on reviewing and analyzing historical data? Sometimes, but we have to make assumptions of similar conditions, a normal distribution, and event independence. Complex systems will have none of these characteristics. Dr. Taleb says as much; “Real life isn’t a casino.”

Indeed, the parable of the turkey and the earthquake loss estimation exercise show us that predicting the future in complex systems can be a futile exercise because there are so many unknowns, changing conditions, and inter-connecting relationships. Extreme events that carry a huge impact happen, and some would argue they’re happening a whole lot more often as interlocking financial markets and globalization become commonplace.

Should prediction exercises be avoided? Nassim Taleb would argue otherwise; “We need to start thinking of the inconceivable,” he says. And while we cannot determine the exact probability of tomorrow’s events, we can “get a general idea about the possibility of their occurrence.”

And that’s where scenario planning comes into play. Bill Ziemba, author of the aforementioned Wilmott Magazine article says, “Getting all the scenarios and their probabilities right is impossible and doesn’t matter much anyway. What is important is to cover the board of possible occurrences. Then you will make sound decisions with risk under control.”

The fact is, like the turkey, we just don’t know what tomorrow will bring. So, plan for the five to seven most likely occurrences and then develop contingencies based on those scenarios. French microbiologist Louis Pasteur says it best, “In the fields of observation chance favors only the prepared mind.”

For a turkey, today may appear like any other “normal” day. However, tomorrow could be the chopping block.

Questions:

• Nassim Taleb says, “It is only in very rare circumstances that probability (by itself) is a guide to decision making.” Does this mean that historical data analysis isn’t worth the effort?
• If chance favors the prepared mind, what’s the “next unexpected twist” that marketers should be looking for?

The Premier of South Australia, Mike Rann, is embroiled in something of a sex scandal at the moment.  I won’t go into all the sordid details, but, briefly, a woman by the name of Michelle Chantelois is claiming that Rann had sex with her several years ago (he was single but she was married at the time), a claim he is emphatically denying.  The political vultures are circling because, given the denial, if a “smoking gun” (or perhaps more aptly a “blue dress”) is produced then Rann is cactus.

Anyway, none of this is really interesting to me.  Actually, it’s hardly anybody’s business.  The shenanigans these two people did, or did not, get up to in their private lives years ago is entirely between them and their families as far as I’m concerned.  But today Chantelois has come out and volunteered to take a lie detector test to determine who is telling the truth.  So I wonder, statistically what’s the probability that she will pass the test?

To answer this question we’ll use Bayes’ Theorem and some rather dodgy data points I picked up around the internet.  It’s on the internet, you see, so it must be true.

The first bit of information we need is the probability that Chantelois is actually telling the truth.  In a recent, and utterly meaningless straw poll conducted by the Adelaide Advertiser, this probability is precisely 53%.

Shortly before 3pm, 53 per cent of respondents to an AdelaideNow poll believed former Parliament House waitress Ms Chantelois was telling the truth about claims of a sexual relationship with Mr Rann, while 47 per cent believed the Premier’s rejection of the allegations.

Therefore P(Chantelois is telling the truth) = P(T) = 0.53;

and P(Chantelois is not telling the truth) = P(N) = 1-P(T) = 0.47.

The next bit of information we need concerns the reliability of polygraph tests themselves.  Personally I’ve always been more than a little sceptical of the infernal things.  Polygraphs smell like voodoo science to me, and according to Wikipedia,

Polygraph testing has little credibility among scientists. Despite claims of 90-95% validity by polygraph advocates, critics maintain that rather than a “test”, the method amounts to an inherently unstandardizable interrogation technique whose accuracy cannot be established.  A 1997 survey of 421 psychologists estimated the test’s average accuracy at about 61%, a little better than chance.

Therefore P(polygraph says you’re telling the truth, given that you’re telling the truth) = P(+|T) = 0.61; and

P(polygraph says you’re telling the truth, given that you’re lying) = P(+|N) = 1-P(+|T) = 0.39.

Now using Bayes’ Theorem, we can calculate Chantelois’ chance of evading the lie detector test.

P(Chantelois is not telling the truth, given that the polygraph says she is)

= P(N|+)

= P(+|N) x P(N) / P(+)

= [ P(+|N) x P(N) ] / [ P(+|T)xP(T) + P(+|N)xP(N) ]

= [ 0.39 x 0.47 ] / [ 0.61 x 0.53 + 0.39 x 0.47 ]

= 0.362 (i.e. 36.2%)

Too high to put any kind of faith in the results of the test.

The calculations above were all done with tongue planted firmly in cheek and are not to be taken seriously.  Whether it’s Rann or Chantelois really telling the truth I don’t know or care.  What is important is that Bayes’ Theorem shows us that, even with accurate tests, there is a good chance of a misclassification.  A single test is usually not enough.

——

One of the simple example graphs I’ve used in some of my research on gossip algorithms has been the 2-dimensional torus with vertices, which looks like a grid with the top and left edges wrapped around to connect with the bottom and right edges. Every vertex has 4 neighbors. Now imagine a very lazy random walk on this graph in which a random walker moves from vertex to one of its neighbors with probability . It’s “well known” that this random walk takes around steps to mix. That is, if is the matrix of transition probabilities then

Here is the second largest eigenvalue of and the relaxation time is the inverse of the spectral gap of the matrix. One way of characterizing for reversible Markov chains is via the Dirichlet form. For a function on the states of the chain define the Dirichlet form by

In our example the stationary distribution and for all edges in the graph. We write if

Define a norm associated with via

Then the characterization is

One question I asked myself today was whether it was “easy” to see what you should choose in the grid example to get the scaling of . Here’s one choice that gives the correct scaling. We’ll set to be constant on each column. Assume without loss of generality that is divisible by 4 and set . The values for on the columns will be like two triangles:

Now we can evaluate the norm, noting that there are vertices per column:

This is because the sum of the first squares scales like and . Now turning to the Dirichlet form, note that each difference between columns is at most and there are fewer than edges for which . Thus:

Taking the ratio gives the lower bound of as desired.

The first I tried was just equal to +1 on the first half of the columns and -1 on the second half of the columns. This ends up giving a suboptimal bound, because the norm but in the denominator we get positive terms. The key is to make all the differences in the denominator small while keeping the average of large enough. Even though you sum over small differences in the denominator, it stays small enough to pay for the in the numerator.

While doing this calculation, I noticed that the book Markov Chains and Mixing Times is also online — it makes a handy reference and is a little easier to use than my old go-to, the Aldous-Fill book.

All week long the sports media has refused to use simple math to analyze Bill Belichick’s decision last week to go for it on 4th down versus the Colts.  Even though the media assumes (correctly?) that the viewer is an intellectual slouch, football media machine always projects an attitude of analysis and discussion.

I did read one article that offered a proper analysis, but its tone gives the impression that it involves extreme math.  In fact, the analysis is simple conditional probability.  Using the probability from that article, this diagram shows how simple the decision is.

I have been reading a previous issue of New Scientist and came across an article by Angela Saini which you can find here on how probability is used in the court room. Also if you goto the page you can take an online test to determine if the article is relevant for you next time you get stuck on jury duty.

It appears that to be a good jury member you need to have a good understanding of conditional and Bayesian Probability. So firstly what do I mean by conditional probability?

Let us consider two events and then the conditional probability of occurring given that has already happened is denoted by . Let us consider a fairly simple example. Consider rolling two unique 6 sided dice, let be the even where the sum of the dice is and let be the event where one dice landed on 4. So here we have that and , however here we have that

because we already know that we have one dice being a 4 so the other dice also needs to be a 4 and that only happens one time in six.

Technical aside: To those probability nerds out there I know I should have labeled these dice, die 1 and die 2 to avoid any complications in finding however I am just trying to give a simple overview and will just skim over this.

The actual definition of conditional probability is given as follows. If , the conditional probability of given is

where we read as the probability of both and occurring at the same time. You can check that we could have used this expression in the above example.

The key point here is that the probability of an even occurring can change if we are given some extra information. Also note that if the two events are independent (i.e. are not related to each other) then .

Now what has this got to do with court cases? It has all to do with how some evidence will be presented in court, normally you should consider it as a conditional probability. We have all watched a some tv show that has involved some court case where an expert witness has said that “only 3% of the population has a AB blood type and so does the defendant” So what would we make of this in terms of probability?

Let be the even the defendant is innocent and let be the event that some evidence is being used for or against the defendant. So what out expert witness has given us is . We view it this way any person from that 3% of the population could have left that blood and secondly we assume innocence and have to prove guilt.

What we want to know is so we need some way to relate these two probabilities. This is where Bayes formula comes into the picture.

Let and be the same events as above then

Here you the jury would have a gut feel for what should be, for instance motive, past record and even the way he looks. The probability would be given to you by the person that gives the evidence and can be calculated as

where stands for not i.e. means not innocent. Also note that , as here the defendant actually committed the crime. Lets have a look at the example

Suppose that you are 80% certain that the defendant is innocent. So we have . A Forensics expert gives some evidence that states that some blood of type AB was found at the scene and only 3% of the population have that type of blood and that the defendant has type AB blood. This means that we take . To find we substitute in to the equation

So once this evidence is given to you the defendants probable innocents plummets from 80% down to 22.4%. In this example you could still not convict using just this pice of evidence. What you can do is keep on adjusting during the entire trial (where relevant) using the above reasoning.

I will finish this post with four problems in understanding probabilities in court cases are (in no real order):

• 1) Prosecutor’s or Defendant’s Fallacy
• 2) Ultimate Issue Error Explicitly taking a small with the defendants likelihood of innocence.
• 3) Base-Rate Neglect
• 4) Dependent Evidence Fallacy This is related to the independence or dependence of events. In terms of court cases this would pop up in genetic effects. For instance we all know that certain physiological problems run in the family be it breast cancer or disease. If two events are independent from each other then the probability of both of these events happening can be found by multiplying both of the probabilities together.
  However using the breast cancer example there is a 1/8 chance of a women having breast cancer during her life. So what is the probability of a mother and daughter both developing cancer during there lives. Well if mother has breast cancer then it is more likely that the daughter could develop breast cancer sometime during her life. I don’t know what that chance is so for sake of argument lets just say that it is twice as likely then the average person, that is a 1/4 chance. So we would find that there is a (1/8)(1/4)=1/32 chance that both mother and daughter will have cancer some time during there lives.

So this begs the question, if every one can be called up for jury duty should we be teaching more probability in schools so every one can understand trials that can include many confusing probabilities?

Math can be really fun. Seriously.

This post is the 2nd in a series of posts I’m planning to have about why math is such a beautiful, useful, and awe-inspiring subject, and that a lot of us can do math (advanced/seemingly difficult math even). Math is such an integral part of humanity since our cave dwelling days, and much more so now in most of our technology driven lives. Previously I wrote about how even advanced math, particularly advanced geometry, can be easily tackled with just your imagination. This time it’s about probability. I can just imagine some of you cringe at the thought of math, let alone probability. But I’ll try to show you that often times, logical reasoning is all that it takes to wrap your head around probabilities, even the ones that confound a lot of brilliant people, even some mathematicians themselves. In fact, we’ll end this article with a simulation of a game/game show. Not bad huh?

Probability and people

In a nutshell, probability is the area of math which deals with the likelihood of an event happening. It is usually expressed as a number, whether a fraction or a decimal, between 0 and 1, with a probability of 1 meaning the event will surely happen and a probability of 0 meaning the event won’t happen.

Now, don’t be too hard on yourself thinking that probability is too hard for you, unlike most of the human population. In fact, probability is one really confounding area of math and problems in it that seem to be easy in hindsight, turn out to be deceptively difficult or tricky, even for  mathematicians, teachers, and other brilliant men and women around the globe. In fact a lot of us have trouble wrapping our heads around probabilities. You mix that with human hopefulness and also the difficulty of grasping very large numbers and what you get is the staggering number of people around the world falling in line to get their lotto tickets so they could win the multi-million prize money.

In fact, if we do the math, in a typical 6/49 game of lotto (6 unique numbers chosen out of 49 numbers, where the order of the 6 numbers is not important) we find that your chances of winning today after buying that lotto ticket is 1 in about 14,000,000. So if Lucy (one of the earliest hominids/proto-humans known to us) or her people, or perhaps even Neanderthals started betting on the lottery at the beginning of their lives, some of them should be millionaires by now. That’s how bad we are at assessing odds, especially coupled with large numbers. So when you go buy that lotto ticket later, I’m afraid the odds are so much against you.

However, I’ll discuss next a particularly perplexing probability problem pondered by people, even brilliant ones, and found the solution to be deceptively trivial after all. Actually, even after you get the explanation, from a practical standpoint it doesn’t seem like so. But the logical reasoning will quite surely buy you out. But don’t fret, all you need again is imagination and logical reasoning.

Game time

Some of you may have heard/read about the American game show Let’s Make a Deal. The Monty Hall problem (MHP) was named after the show’s host. Simply stated, the rules of the game are as follows:

The game master (GM), has 3 doors: 2 with goats behind them and one with a car behind it. The GM lets you choose one door, which you think holds the prize car behind it. Since the GM’s job is to make you and the audience excited and enjoy the game, the GM opens another door. But since the GM knows the placement of the goats and the car i.e. which door has which item behind it, the GM opens a door which has a goat behind it. Now, the GM poses a question to you: Do you or do you not want to change the door you initially picked i.e. the GM gives you an opportunity to stay with the door you originally picked, or to choose the other door, knowing that one of the doors, which the GM opened, has a goat behind it.

The GM in the show is of course Monty Hall (MH). Now, you’d most probably think that since there are only 2 doors left unopened, that the probability of getting either a goat or a car is now 50/50 or 50% right i.e. it doesn’t matter whether you switch doors or not?

Nope.

In fact, however counterintuitive this may seem, your chances of getting the car at this point of the game doubles if you decide to change the door you initially picked. How? Let’s find out shall we?

Goat, Car, Goat

Now let’s strap on our imagination and logical reasoning caps to find out how the probability of getting the car increases two-fold if you switch your chosen door, and that it’s not a 50/50 chance of getting the car once a door with a goat has been opened by the GM.

Monty Hall problem

One way of looking at how this counterintuitive probability problem is correctly tackled is by taking the possibility of the events one at a time (refer to the figure above please). In this scenario we show that when you switch doors, you always double your chances of winning. Here’s how:

1. First event, say you picked a door and it happened to have the prize car behind it.  Regardless of which door the GM opens, switching in this case either gives you goat A or goat B i.e. you lose the prize car. Out of the 3 possible scenarios (2 of which are listed right after this one), in this one event/case do you lose the prize car.

2. Second event, you choose a door with a goat (goat A) behind it. The GM opens a door again with a goat (goat B) behind it. If you switch in this case, you get the car. This event, wherein you get the car by switching, is one event which you get the prize car. Score one for you.

3. Third event, you choose the 3rd door with a goat(goat B) behind it. The GM again opens a door with a goat (this time, goat A). So when you switch, you get the car again. Yay. This event, wherein you again get the car by switching, is another event which lets you take home the prize.

So what did we get from all this? We saw that out of 3 events/cases of picking either of the 3 doors, you always get 2 events (event 2. and 3.) which favor switching and which lets you walk away with the prize (or in this case, drive away with the prize). So the odds of getting the car/prize in the MHP is not 50% as a lot of us would initially assume, but instead, is really 2/3 or approximately 66.7%.

It can take a while to sink in, but the reasoning/explanation is quite logical and sound.

Try it out!

I actually tried this out with my mother and at another time with my younger brother. What I did was I got 3 opaque plastic cups (simulating the doors) and 2 toy cows (no goat toys in our house at that time) and 1 robot toy that transforms into a car (not bad for a prize no?). I made them act as a GM at one time, with me being the game contestant. Of course to prove my point I always switched. We did this about 20 times and I got the prize car (or robot) at around 14 times out of the 20 (roughly 2/3 of 20). Then I acted as a GM and they acted as the contestant. Then their job was not to switch doors (or cups), just to prove my point that you get the prize more often than not (2/3 of the time remember?) by switching instead of staying with your original door/cup.

They even asked me if I was doing a magic trick on them. I told them it was the power of mathematics and of logical thinking. Imagine what much more primitive, let’s say Bronze-aged men, would think of me, with this knowledge, even without modern devices like a cellphone. Perhaps they’d think of me as an oracle or even a god.

Great, great. But what’s the use?

I think one important thing we can get from this (other than to show you that you can do maths you thought were too hard or complicated for you) is that with math, we can make decisions in our lives (sports betting, lottery, game shows and so on) with more clarity, logic, and sound reasoning, instead of just blind optimism.

If you didn’t get the logic on how to win the game at first glance, or if you thought it was 50/50, don’t be ashamed, a lot of people (some brilliant even) fell for it too. In fact, out of 228 subjects in a study, only 13% chose to switch, and that the rest (87%) assumed that the switching didn’t matter since the likelihood of getting the car out of the 2 unopened doors are equal (research by Mueser and Granberg, 1999).

Quoting cognitive psychologist Massimo Piattelli-Palmarini

“… no other statistical puzzle comes so close to fooling all the people all the time”

and

“that even Nobel physicists systematically give the wrong answer, and that they insist on it, and they are ready to berate in print those who propose the right answer.”

So, not bad eh? Still think math (or at least those areas you think are too advanced or complicated for you) isn’t for the average person? If so, then look forward to my next posts about math.

References, resources, and further reading

• Wikipedia article on the mathematics of lottery
• Wikipedia article on the Monty Hall problem

204312     ความน่าจะเป็นและกระบวนการสุ่มสํ าหรับวิศวกรคอมพิวเตอร์     Probability and Random Processes for Computer Engineers

ความน่าจะเป็น ความน่าจะเป็นแบบมีเงื่อนไข ความเป็นอิสระของเหตุการณ์ ตัวแปรสุ่ม ฟังก์ชันการแจกแจงและความหนาแน่น ฟังก์ชันของตัวแปรสุ่มเดียว ตัวแปรสุ่มหลายตัวการดํ าเนินการกับตัวแปรสุ่มตัวเดียวและหลายตัว กฎของจํ านวนเลขขนาดใหญ่ ทฤษฎีจํ ากัดช่วงกลางกระบวนการสุ่มและการประยุกต์ การประยุกต์กับปัญหาทางวิศวกรรมคอมพิวเตอร์

(Probability; conditional probability and independence of events; random variables; distribution and density functions; functions of one random variable; multiple random variables; operations on one and multiple random variables; laws of large numbers; central limit theorem; random processes and their applications; application to computer engineering problems.)

(204312 มหาวิทยาลัยเกษตรศาสตร์)

205215     ความน่าจะเป็นประยุกต์สํ าหรับวิศวกรไฟฟ้า     Applied Probability for Electrical Engineers

ความน่าจะเป็นแบบร่วมและแบบมีเงื่อนไข ความเป็นอิสระทางสถิติ ตัวแปรสุ่มแบบไม่ต่อเนื่องและแบบต่อเนื่อง ฟังก์ชันการแจกแจงและความหนาแน่น การดํ าเนินการบนตัวแปรสุม่ ตัวเดียวและหลายตัว การคาดหมาย โมเมนต์และฟังก์ชันลักษณะเฉพาะ กฎของตัวเลขขนาดใหญ่ ทฤษฎีขีดจํ ากัดกลาง การคํ านวณความเชื่อถือได้เบื้องต้น การทดสอบรูปแบบการกระจายของข้อมูล

(Joint and conditional probability, statistical independence, discrete and continuous random variables, distribution and density functions, operations on one and multiple random variables, expectation, moments and characteristic functions, law of large numbers, central limit theorem, basic reliability calculations, testing the fit of a distribution of data.)

(205215 มหาวิทยาลัยเกษตรศาสตร์)

206221     ความน่าจะเป็นและสถิติประยุกต์สํ าหรับวิศวกร     Applied Probability and Statistics for Engineers

ความน่าจะเป็น ค่าคาดคะเนและการแจกแจงความน่าจะเป็นที่ใช้ทั่วไป การแจกแจงจากการสุ่มตัวอย่าง การอนุมานทางสถิติสํ าหรับปัญหาการสุ่มตัวอย่างหนึ่งและสองชุดการวิเคราะห์การถดถอย การวิเคราะห์ความแปรปรวนและการประยุกต์สถิติกับระบบอุตสาหกรรม

(Probability, expectation and common probability distributions, sampling distributions, statistical inference for one – and – two – sample problems, regression analysis, analysis of variance and their applications to industrial systems.)

(206221 มหาวิทยาลัยเกษตรศาสตร์)

Let and be random variables with joint pdf . Many applications are concerned with the conditional distribution of a random variable (e.g. ) given a realized value of another random variable (e.g. ). Sometimes the mean of the conditional distribution of is linear in , i.e. for some constants and . In such situations, it turns out that the conditional mean can be expressed as follows:

Furthermore, when the conditional variance is constant, it can be expressed as follows:

Both and hold when and are jointly distributed according to the bivariate normal distribution. In this post, I prove both and and give a simple non-bivariate normal example where these two conditions hold. I will discuss bivariate normal distribution in a subsequent post.

DISCUSSION OF EQUATION (1)

Given that , we can take expected value on both sides. In some of the expectation below, the subscript indicates that the expectation is taken with the marginal desnsity of .

is an equation in terms of and . We can find another equation of and by multiplying both sides of by . Then we take expectation with respect to .

Substituting with , we obtain the following equation:

Using equations (4) and (5) to solve for and , we obtain the solutions:

DISCUSSION OF EQUATION (2)

The conditional variance is a function of :

Let’s plug the linear conditional mean into the conditional variance .

Now, take expectation with respect to .

Plugging into , we obtain:

If the conditional variance is constant, then the equation (6) above becomes:
.

EXAMPLE

Consider and with the following joint pdf:

For and :

.

For and :

.

Using the given joint density, we can calculate the following:

Note that the conditional density for is simply the result of shifting of the variable the following density:
for and for .

The mean and variance of are: and . Other than the adjustment due to the shifting, is identically distributed (hence the variance is constant with ). The shifting causes the conditional mean to be linear: .

The unconditional mean of is . The unconditional variance of is . Comparing the conditional mean with equation (1) and the conditional variance with equation (2), we obtain the following:

Last night a friend of our family was at our home and told about how they quit praying because it did not seem to do any good. This person would fervently lobby God, but each time the request would fall on deaf ears. This person was notably discouraged by this lack of response and therefore made the decision that prayer did not work.

Though I held my tongue at the time, I wanted to give her a hearty amen.

I haven’t been a real believer in the supposed power of prayer for quite some time and I must admit cringing a bit when others see fit to invoke it. This doubt is not the product of some selfish want that God didn’t fulfill, it’s not because God didn’t rain money from heaven like Joel Osteen or those of his ilk might claim, rather, it is the product of much time and thought and study.

First of all, I think the idea of successful intercessory prayer comes from a skewed or perhaps even nonexistent understanding of statistical probability. When one realizes that improbable does not equal impossible, the idea of a “miraculous” recovery takes on a much more realistic bend. Just because the chance of an event’s success is at the far end of the bell curve does not mean it cannot happen without divine help. Ask as atheist who survived an improbable health scare or the family of a believer who died and maybe that will put it in perspective. It does seem a bit arbitrary when you consider who makes it and who doesn’t. There was a recent study released on the phenomena of intercessory prayer and you can read about that here.

I wonder why people only pray for things within the realm of probability. Recovery from cancer is possible. In his debate with Rick Warren, atheist Sam Harris puts it this way:

You could prove to the satisfaction of every scientist that intercessory prayer works if you set up a simple experiment. Get a billion Christians to pray for a single amputee. Get them to pray that God regrow that missing limb. This happens to salamanders every day, presumably without prayer, so this is within the capacity of God. I find it interesting that people of faith only tend to pray for conditions that are self-limiting.

Thus you begin to wonder if Christians even believe in the power of prayer.

Now, this does not mean that prayer is not a useful tool. I feel quite certain that the peace of mind provided by having loved ones pray over you or by offering up your own prayer can be a great aid in recovery. While this may not be supernatural and is certainly not a gift solely relegated to Christianity, the belief that one will recover can go a long way. I imagine the psychological benefits are great for both those who receive and say prayers.

On another note, I often wonder if some prayers (not ones for physical recovery) stem from little more than laziness on the part of Christians. We can pray for God to feed the hungry and take care of those in need, but if we are not willing to get out in the world and actually work to make it a better place, what does that say of us?

Again, this is not to discourage anyone from praying or to say that I never will (I do). Instead, I’m just thinking some things through and wanting to put them in perspective.

Our Winter school on Probability and Geometry is approaching. It will take place at the Hebrew University of Jerusalem on December 15-23. The deadline for application is November 25, 2009; by all means, apply! it will be a very nice event. We got additional support from The Clay Mathematics Institute and from Microsoft Research. Financial support (local expences, and, in some cases, partial support for the travel) may be available.

The poster gives the updated list of confirmed speaker. (There may be 1-2 additional speakers that did not confirm yet.)

Here is the link to the site of the school.  I will update this page regarding program and schedule. UPDATE (Preliminary program)

Among the ptlanned opics for the lecture series: Probability on Trees and Graphs; Percolation;  Graph limits;  Random triangulations and KPZ;  Random walks and Brownian motions; SLE  I;  SLE and the Ising Model;  Boolean functions, Noise sensitivity and Fourier analysis; Applications to Physics;  Further connections to Geometry.

219323     ความน่าจะเป็นและสถิติสําหรับวิศวกรซอฟต์แวร์และความรู้     Probability and Statistics for Software and Knowledge Engineer

ความน่าจะเป็น ความน่าจะเป็นแบบมีเงื่อนไข ค่าคาดคะเน และการแจกแจงความน่าจะเป็นทั่วไป การแจกแจง การสุ่มตัวอย่าง การอนุมานทางสถิติ การวิเคราะห์การถดถอย การวิเคราะห์ความแปรปรวนและการประยุกต์

(Probability; conditional probability; expectation and common probability distributions; sample distributions; statistical interference; regression analysis; analysis of variance and their applications.)

(219323 มหาวิทยาลัยเกษตรศาสตร์)

Presentology or futurology?  Which is your pick?

I think most of us think that it is good to defer gratification. Well we know it is.  We all know by now the story of the kids given a marshmallow and told they will get another if they leave it untouched until the experimenter gets back. The 30% of kids who resist the urge to wolf down the marshmallow do better in life.

So it is better to be a futurologist?  Not so?

But what is the future?

Our fascination with the future rests in a great part on a fallacy of prediction.

Since mankind has kept records we have been pretty keen on consulting oracles, reading the tea leaves, listening to the weather reports ~ anything to allow us to know what will happen and to be on the right side of history.

We desperately want to know and we desperately want the world to be as predictable as the sun coming up in the morning.

Some predictability is good

I see nothing wrong with that.  Personally, I like my keys to be where I left them.  And I quite like it if my black dog doesn’t lie in a dark passage way for me to trip over him.

Here in lies the important point.  It is not forecasting the future that is important.  It is understanding how the world works that is important.

If there is no one else in the house it would be jolly strange if my keys moved from where I put them.  If there is anyone else in the house, even a black dog who likes keys, some dogs do, then my keys might not be where I left them.  It is the mechanism not the prediction that is important.

When we know the mechanism, then we can do something about it.  We must know the mechanism and all the mechanisms that are relevant. Keys rarely move by themselves but other people might move them without telling me.  Mechanisms introduce randomness and it is better to allow for randomness than get fixated on certainty.

Let’s take my dear black dog as a second example.  He might lie in the dark passageway quite often, but I can’t predict when he will.  I can only allow for the possibility that he might and either walk more slowly or whistle and hope he moves and makes a noise so I can hear him.  Knowing how the world works and the range of possibilities we might encounter is what matters.

So what is better: presentology or futurology?

Now I have explained this like this, it seems quite obvious but what does this affect the choice of presentology or futurology?  How does this relate to the kids and the marshmallows?

I need to know the mechanisms to know what I  can do now, RIGHT NOW.  Because the future follows from now, I want to know how I can change now.

• I want to change now so that now is better.
• And I want to act now because now is the only time we can act.
• I want to act because I like action. Action makes us feel good.
• And I want to change now because it makes the future more interesting!

Instead of worrying whether or not I will trip over my black dog, I ask myself what mechanisms I can manage to walk safely to my destination.  Calling to my dog is one of them.  If I want to get to the end of the passage safely, I must manage all of the mechanisms, on their own terms, as they come up.

Knowing that I want to get to the end of the passage safely or knowing that I get to the end of the passage safely 90% of the time simply doesn’t help me.

But knowing that a black dog tends to lie there quietly, and knowing that dogs do respond when you call, knowing these mechanisms helps me manage possibilities and helps me rearrange NOW, in this case what is going on in my head.   By understanding now and rearranging it, I allow possibilities to evolve that I might enjoy.

Presentology : the art of now

What needs to be done now?

We are all talking about now. Personal kanbans, productivity, mindfulness, solidarity, happiness. It is all about being master of the present ~ master of what is happening this minute!

Note:  The late Russ Ackhoff used the term presentology to describe his philosophy of management

I have removalists arriving tomorrow. I don’t know when they will arrive.

It’s times like these that I think about the Cable Guy Paradox by Hajek.

Imagine that you’re like me and you’ve called a removalist (or – in the original – a guy to come and fix your cable television). They’ve told you that they will, with 100% certainty, be at your place between 9am and 3pm. So there are three hours in which they could arrive before noon and three hours in which they could arrive after noon. If you ignore the chance that the removalist will arrive at exactly noon, there is a 50% chance that she will arrive in the morning and an equal chance that she will arrive in the afternoon.

Your housemate decides to have a bit of a wager with you as to when the removalist will arrive: morning or afternoon?

‘[W]e may put the reasoning in terms of a plausible diachronic rationality principle somewhat in the spirit of van Fraassen’s ‘Reflection Principle’ (1984 and 1995). The idea is that you should not knowingly frustrate a rational future self of yours. I will call it the ‘Avoid Certain Frustration Principle’:

Suppose you now have a choice between two options. You should not choose one of these options if you are certain that a rational future self of yours will prefer that you had chosen the other one—unless both your options have this property.‘ — Hajek [Source]

So we’re placing this bet and we’re trying to avoid certain frustration. If we are to place our bet on the removalist arriving in the morning, for each second that passses, the odds increase that the removalist will arrive in the afternoon.

For example, if the removalist has not arrived by 10am, then there are only two morning hours left, but three afternoon hours left.

‘The choice to bet on the morning interval falls squarely under the purview of the Avoid Certain Frustration Principle. It is thus ruled out. Rationality, then, requires you to bet on the afternoon interval (the only choice that is not so ruled out). This is paradoxical, because your initial reasoning that there is nothing to favour one interval over the other seemed impeccable.‘ — Hajek [ibid.]

If you’ve got a 50-50 chance of being correct, it doesn’t matter which you choose because you’ve got equal chance of being correct. And yet, here we are, knowing that, in the future, these even odds will change predictably against us if we choose the morning bet.

Isn’t that cool? Yes, yes it is.

Interestingly (perhaps worryingly), Hajek thinks that the Avoid Certain Frustration Principle should be challenged.

‘Sometimes it is rational to knowingly act against your rationally-formed future preferences, even when you know exactly how to avoid doing so.‘ — Hajek [ibid.]

Over on his blog, Consequentially.org, Greg Restall says:

‘Hájek uses this example to motivate a rejection and revision of the Avoid Certain Frustration Principle. I think that rejecting this principle seems sound, but I don’t think that this example shows it. Here’s why:

I don’t think that in this case I am certain to be frustrated. For I’m not certain that there is, in fact, an interval of time where I will regret making the bet. First, the bet might be cancelled for some reason – the guy might arrive early, contrary to his promise. More interestingly, we might decide to go out and leave someone else to mind the house, only to return after 4pm. In that case, at any time after 8am, I don’t know that the guy hasn’t arrived, so I don’t have the same grounds for regret. So, I’m not certain that I’ll have my regret.‘ — Restall [Source]

For me, it seems that Restall bypasses the point. The point is that you’re given 50-50 odds and yet know that these odds are going to shift against you. Within the context of just the bet, these odds are going to change.

Imagine that you’re sitting an introduction to probability class. You’re asked the question: ‘You’ve got a six-sided fair dice and you roll it. What are the odds that you’ll roll a 6.’ The correct answer is not: ‘Well, it would be one in six but it’s slightly less because you’re not accounting for your pet dog eating the dice when you roll it, or the Large Hadron Collider creates a miniature black hole which swallows your dice, or… &c., &c., &c.’

Sure, you’re correct that these are all possible but they’re not really relevant to the problem at hand. Restall’s objection can be easily bypassed with the following.

‘There is 100% certaintly event S will happen between time t0 and time t1. Time t0.5 is directly between t0 and t1. P(S occurs after t0.5) = 0.5. If S has not happened by any time after t0, P(S occurs after to.5) increase.’

The Avoid Certain Frustration Principle still holds but worries about being mugged and the like no longer apply.

It is interesting that both Hajek and Restall suggest rejecting the ACF Principle and not rejecting the unstated Principle of Making Decisions Based Exclusively on Probability. As we know that the probability will change, we should ignore the odds and go with our rational deliberations. Probability is only one tool in our reasoning toolkit. Just as you don’t use a screwdriver to chop down the mightiest oak in the forest, you wouldn’t exclusively use probability to decide how you’ll place your bet.

Don’t get me wrong. That’s really, really weird. The origins of probability go back to when mathematicians would rip off people in ye ancient pubs by keeping the odds in their (the mathematicians’) favour. To say that there is a class of gamble which doesn’t come down to probability is a bit disturbing.