T-Test

What is a T-Test?

The t-test compares the means (averages) of two populations to determine how different they are from each other. The test generates a T-score and P-value, which quantify exactly how different each population is and the likelihood that this difference can be explained by chance or sampling error.

There are three variations of the t-test used for different scenarios:

  • Independent Samples: compares the averages for two groups.

  • One Sample: tests the averages of a single group against a known average.

  • Paired Sample: compares averages from the same group at different points of time.

How to Interpret a T-test?

The null hypothesis for the Independent and One Sample tests is that the means of both groups are identical. For the Paired Sample, the null hypothesis is that the pairs of differences between both tests are equal.

Regardless of the technique used, the test will output a t-score. This score is simply the ratio between the mean difference across two groups, as well as the difference within the groups. The bigger the t score, the larger the difference between samples, which also means the test results are more likely reproducible. A smaller score means more similarity between groups. For example, a t score of 2 means that the means of both groups are twice as different from each other as they are from data points in their own group.

In addition, a p-value is generated for each t-score, which represents the probability that difference was caused by random chance. Generally, a p-value of less than 5% is required to fail to accept the null hypothesis.