Research Article Critique Assignment
(link 3) standards of adequacy for true experimental designs, quasi-experimental designs, and single-subject designs
 
Standards of Adequacy for True Experimental Designs, Quasi-Experimental Designs, and Single-Subject Designs
 
Quoted from McMillian, J. and Schumacher, S. (1997). Research in education: A conceptual introduction (4th edition), pp 348-349. NY: HarpersCollins College Publishers.
 
In judging the adequacy of the designs focus on a few key criteria. These criteria are listed below in the form of questions that should be asked for each type of design.
 

True Experimental Designs
 
1. Was the research design described in sufficient detail to allow for replication of the study?
 
2. Was it clear how statistical equivalence of the groups was achieved? Was there a full description of the specific manner in which subjects were assigned randomly to groups?
 
3. Was a true experimental design appropriate for the research problem?
 
4. Was there manipulation of the independent variable?
 
5. Was there maximum control over extraneous variables and errors of measurement?
 
6. Was the treatment condition sufficiently different from the comparison condition for a differential effect on the dependent variable to be expected?
 
7. Were potential threats to internal validity reasonable ruled out or noted and discussed?
 
8. Was the time frame of the study described?
 
9. Did the design avoid being too artificial or restricted for adequate external validity?
 
10. Was an appropriate balance achieved between control of variables and natural conditions?
 
11. Were appropriate tests of inferential statistics used?
 
back to top
 
 

Quasi-Experimental Designs
 
1. Was the research design described in sufficient detail to allow for replication of the study?
 
2. Was a true experiment possible?
 
3. Was it clear how extraneous variables were controlled or ruled out as plausible rival hypotheses?
 
4. Were all potential threats to internal validity addressed?
 
5. Were the explanations ruling out plausible rival hypothesis reasonable?
 
6. Would a different quasi-design have been better?
 
7. Did the design approach a true experiment as closely as possible?
 
8. Was there an appropriate balance between control for internal validity and for external validity?
 
9. Was every effort made to use groups that were as equivalent as possible?
 
10. If a time-series design was used,
(a) Was there an adequate number of observations to suggest a pattern of results?
(b) Was the treatment intervention introduced distinctly at one point in time?
(c) Was the measurement of the dependent variable consistent?
(d) Was it clear, if a comparison group was used, how equivalent the groups were?
 
back to top
 

Single-Subject Designs
 
1. Was the sample size one?
 
2. Was a single-subject design most appropriate, or would a group design have been better?
 
3. Were the observation conditions standardized?
 
4. Was the behavior that was observed defined operationally?
 
5. Was the measurement highly reliable?
 
6. Were sufficient repeated measures made?
 
7. Were the conditions in which the study was conducted described fully?
 
8. Was there stability in the base-line condition before the treatment was introduced?
 
9. Was there a difference between the length of time or number of observations between the base-line and the treatment conditions?
 
10. Was only one variable changed during the treatment condition?
 
11. Were threats to internal and external validity addressed?
 
back to top