What is P-Value?

I've seen people misinterpret the p-value in a variety of ways. Here are a few examples of common errors:

1. The likelihood that we would incorrectly reject the null hypothesis: Although a low p-value encourages the rejection of the null hypothesis, it says nothing about the likelihood of rejection.

2. The level of statistical significance: We choose the level of significance before we conduct the experiment. Only when the p-value meets our level of significance (p alpha) can we draw conclusions.

3. The magnitude of the intervention effect: The p-value does not represent the magnitude of the intervention in the sample that was introduced during the experiment.

4. The likelihood that the null hypothesis is correct: This comes close and may not cause much harm, but it will cause confusion. It is impossible to discuss the truth of a null hypothesis using a frequentist statistic. A high p-value indicates that our data is highly consistent with our null hypothesis.