Null hypothesis test and Type I / Type II errors

Null hypothesis test

- why do it?  to have some idea whether there is a relationship between an independent variable (IV) and a dependent variable (DV)

- how? 

1. make the hypothesis that there is NO relationship (null hypothesis)
2. calculate the p-value, which is the PROBABILITY OF GETTING THESE DATA GIVEN NULL HYPOTHESIS IS TRUE, rather than the p(null hypothesis is true)
3. If p-value is small enough (by convention under 5%, which is called α (alpha) level), we'll reject the null and claim that there's a statistical significant relationship between the IV and DV

Type I error

- false positive
- how to remember?

1. positive <- hey, there's a relationship!
2. false <- actually there is not... :(
3. that means we have rejected the null while we shouldn't have 

- α (alpha) = P(Type I error | null hypothesis is true)
 

Type II error

- false negative
- i.e. failed to reject the null 
- β (beta) = P(Type II error | null hypothesis is false)

Power

= P(correctly reject the null) = 1- β


Ref: http://onlinestatbook.com/2/logic_of_hypothesis_testing/errors.html