What are Independent and Identically Distributed Random Variables?
A set of variables is independent and identically distributed (IID) if (a) the variables are all mutually independent (see independence) and (b) the variables are all drawn from the same probability distribution. A sequence of results from a d20, for example, is IID -- each roll is independent of the others, and each time you roll, you're drawing from the same probability distribution (i.e. 1/20 chance of each result).
Most random data that you tend to come across in everyday situations is going to be IID. The strongest and most simple example of this is flipping a (fair) coin. Because the coin doesn’t remember the last thing it showed, all of the flips are “independent”. The variables are identically distributed because as long as the coin is fair, there is a 50/50 chance each and every time that you will get heads or tails - thus, identically and even distribution.
In fact, it is more informative to list examples of sets of random variables that are not IID:
- A poker hand, considering each card as a separate random variable, is not IID, because the variables are not mutually independent (if the first card is the ace of hearts, you know that the next card isn't going to be).
The output of a Markov chain is not IID, because the probability distribution of each variable is dependent upon the previous state of the Markov chain.