PRINCIPLES OF HYPOTHESIS TESTING
Hypothesis - Probabilities are formalized into hypotheses, which are research predictions that can be tested. A statistical hypothesis (which refers to probability in terms of sampling and measurement error) is stated in the "null" form.
Null Hypothesis - assumes that there is no relationship between 2 or more variables.
The null states that nothing is different about the population means of the two groups. The researcher does an inferential statistical test to determine the probability that the null hypothesis is not true. The researcher wants to prove that there is a relationship and therefore hopes that the null is false (i.e., rejected). If the null is false, which means that it is false that there is no difference between groups, then there is high probability that there is a difference between two groups.
Our criminal legal system is based on this premise of the null: Innocent until proven guilty. We begin with a null hypothesis: There is no relationship between the crime and the person charged with the crime. If the null is proven false then there is a high probability that the person committed the crime. The prosecuting attorney wants to show that there is a high probability of being correct in rejecting the null.
You can never prove a null hypothesis, only retain it as a possibility. It is much better to design the study to reject something. Be wary of conclusions bases on unrejected null hypothesis.
The reason null hypothesis are used with inferential statistics is that we never prove anything, we only fail to disprove. Failure to disprove is consistent with the reality of probability in our lives. In other words, if we cannot find compelling evidence that they are different, the most plausible conclusion is that they are the same. For conceiving and designing research the research hypothesis is far more important than the null. The null is a technical necessity in using inferential statistics.
Level of Significance - is a value selected to indicate the chance that it is wrong to reject the null hypothesis. It is also called "probability," or "p level" (when calculated from the results), or "alpha level" (when set by the researcher before obtaining the results).
The lower the level of significance the more confidence the researcher has in rejecting the null. A level of 0.01 is one chance in hundred. A level of 0.05 is 5 chances in 100. Therefore, a significance level of 0.01 provides us with more confidence that we are safe in rejecting the null. One chance in 1000 (0.001) is safer than one chance in 100 (0.01).
Not to reject the null hypothesis is a weaker position. The lower the significance level selected as the criteria of when to reject the null, the harder it makes it to reject the null. We want to make the criteria as hard as possible, such as .01 or .001, but usually the researcher sets the significance level or criteria at .05. Negative results are less likely to get published. The researcher wants to reject the null with confidence.
True experimental designs utilize randomization of subjects to groups and manipulation of the treatment variable.
Quasi-experimental designs are used when it is impossible to randomly assigned subjects to experimental and control groups, and when a control group is unavailable.
Ways to Achieve High Internal Validity:
1. Reliable measurement - be sure the observation conditions are the same throughout data collection. Observers are trained and checked for reliability and bias. The observation is defined operationally. Report all aspects of data collection so that any threat to validity can be reasonably ruled out.
2. Repeated measurement - repeated measurement controls for normal variation and provides a clear, reliable description of the behavior.
3. Description of conditions - provide a detailed description so other researchers may build on the study
4. Baseline and treatment condition; duration and stability - observations should last about the same length of time and contain about the same number of observations. Observe for long enough periods to establish a stable pattern or baseline.
5. Single-variable rule - change only one variable during the treatment phase and describe the changed variable precisely.