There are many different types of “experiments.” Most are quite different from the common stereotype. All experimental research, however, has several elements in common. One of the most obvious is the division of the subjects into groups (control, experimental, etc.). Another is the use of a “treatment” (usually the independent variable) which is introduced into the research context or manipulated by the researcher. The four research parameters (discussed earlier in this module) will help us understand the other distinguishing characteristics of experimental research.
GENERAL APPROACH
Synthetic
(Holistic) |
|
Analytic
(Constituent) |
On the synthetic-analytic continuum, experimental research tends to fall on the analytic end. Unless it is very complicated, an experiment typically focuses on a specific element (a “constituent part”) of the larger process of language learning and teaching.
RESEARCH AIM
Deductive
(Hypothesis
Testing) |
|
Heuristic
(Hypothesis
Generating) |
The next parameter deals with the heuristic (hypothesis-generating) vs. deductive (hypothesis-testing) factor. In contrast to qualitative research, virtually all experiments are designed to test hypotheses.
CONTROL OVER THE RESEARCH CONTEXT
Experiments generally fall on the high end of this scale because they attempt to control the research environment to a considerable degree. This can be both a plus and a minus.
| On the one hand, it allows the researcher to isolate a particular variable and focus on it in order to determine its effect on other variables. Because of this feature, only experimental studies can claim to show any degree of causality. Qualitative and descriptive research can reveal only relationships or processes. |
|
On the other hand, control has several disadvantages. One is that it often makes the research situation unnatural. Consequently, subjects may not behave normally in an experiment. Another disadvantage is that it is virtually impossible to control all the variables in a research situation involving human beings. Finally, controlled experiments often raise serious questions about research ethics. |
EXPLICITNESS OF DATA COLLECTION PROCEDURES
The final parameter deals with the level of explicitness in data collection. Here again, experimental research falls toward the high end of the scale. Carefully focused instruments (tests, observations, questionnaires, etc.) that generate precise quantitative data are the norm in experiments. These data can be analyzed using statistical tests of significance in order to accept or reject the hypothesis.
| KINDS OF EXPERIMENTAL RESEARCH |
|
Within the realm of experimental research, there are three major types of design:
| <> |
PRE-EXPERIMENTAL
QUASI-EXPERIMENTAL
TRUE-EXPERIMENTAL |
If you choose to conduct experimental research, one of your most important tasks will be to choose the design that gives your research the best combination of internal and external validity. At the same time, it must be practical enough so that you can actually do the research in your own circumstances.
Remember, no particular type is right for all situations. Real-world constraints will often dictate what is practical or possible. In any case you need to be careful to recognize the weaknesses of the design you choose. Do not attempt to prove things or make claims in your findings that are beyond the capabilities of your design.
Actually there are several sub types of pre-experimental design. The one suggested for beginner research (for undergraduate students) is
The One-Group Pretest-Posttest Design
R à O à T à O
In this experiment, a single group is measured or observed not only after being exposed to a treatment of some sort, but also before.
The symbol R refer to the Randomized research subjects (through not necessary), while O refers to observation through measurement. The placement of the symbols indicate the order in time of T and O. As you can see, the T (treatment) comes after the first observation (measurement of the entry behavior) and before the second time of observation (measurement of the effect, final results )
PRE-EXPERIMENTAL DESIGNS are lacking in several areas of the true-experimental criteria. Not only do they lack random selection in most cases, but they usually just employ a single group. This group receives the “treatment,” there is no control group. Pilot studies, one-shot case studies, and most research using only one group, fall into this category.
The advantages are:
- Very practical
- Set the stage for further research
Disadvantages:
| QUASI-EXPERIMENTAL DESIGNS |
QUASI-EXPERIMENTAL DESIGNS are usually constructions that already exist in the real world. Those designs that fall into the quasi-experimental category fall short in some way of the criteria for the true experimental group. A quasi-experimental design will have some sort of control and experimental group, but
these groups probably weren’t randomly selected. Random selection is usually where true-experimental and quasi-experimental designs differ.
In some other books, this kind of a research is also calld “The Static Group Comparison” or “nonequivalent control group design”(Frankel and Wallen; 272, 2007)
The design is as follows:
R à T à O
———————-
O
Note: In some other books, the above design falls into the category of “one kind of pre experimental design”.
As you see in the diagram, the researcher only prepares one group that is intended to be treated (X) specifically planned. As the comparison, there is one other group that is no control over it (no randomized, no pre observation or pre measurement, no homogeneity test, etc). However, having finished with the treatment in one group, the researcher observes (measures) both groups and compared the effects.
The most principle of a quasi-experimental design is THERE IS NO randomization over both groups to compare. Both groups have been existed in the real world and the researcher JUST MATCH two groups; one to be the experimental group, and the other to be the control group.
There are several more designs of quasi-experiments. However the two designs of a quasi-experiment that is predicted feasible to implement by under graduate students are::
The Matched Groups with Post-test Only Design
Note: The symbol M here refers to the groups of research subject that are JUST MATCHED, without any randomization.
Treatment group M à T à O
——————–
Control group M à C à O
The design shows that there is no observation on the entry behavior (no pre test) to see the homogeneity of the two groups. The difference happens in the kinds of treatments. The effect of the treatments are observed and compared.
Some advantages of the quasi-experimental design include:
- Greater external validity (more like real world conditions)
- Much more feasible given time and logistical constraints
Disadvantages:
- Not as many variables controlled (less causal claims)
| TRUE-EXPERIMENTAL DESIGNS |
TRUE-EXPERIMENTAL DESIGNS must employ the following:
- Random selection of subjects
- Use of control groups
- Random assignments to control and experimental groups
- Random assignment of groups to control and experimental conditions
In order for an experiment to follow a true-experimental design, it must meet the preceding criteria. There is some variation in true-experimental designs, but that variation comes in the time(s) that the treatment is given to the experimental group, or in the observation or measurement (pre-test, post-test, mid-test) area.
There are several sub types of true experimental designs. The two suggested to under graduate students are:
(1) Randomized group design with Posttest Only
Treatment group R à T1 à O
——————-
Control group R à T2 à O
In this design, the entry behaviors of the subjects are neglected. It can be done whenever the researcher have made sure that the condition before treatment are equal, comparable, and homogeneous. For such a research, it is suggested there must be at least 40 individuals in each group (Frankel and Wallen; 2007, 273).
(2) Randomized group design with Pretest and Posttest
Treatment group R à O à T1 à O
—————————-
Control group R à O à T2 à O
In this design, the entry behaviors of the subjects are observed. The purpose is to make both groups are equal (similar and homogeneous). You see, the R refers to the process of randomization. The pretest is given in order to get “indicators” used by the researcher to check if the process of randomization “actually result two equivalent groups or not. If NOT, the researcher must redo the randomization. For an experiment with such a design, the smaller number of individual in each group (less than 30) is reasonable (Frankael and Wallen; 2007, 274)
Advantages of the true-experimental design include:
- Greater internal validity
- Causal claims can be investigated
Disadvantages:
- Less external validity (not like real world conditions)
- Not very practical
Finding a
RESEARCH QUESTION is probably the most important task in the reasearch process because the question becomes the driving force behind the research-from beginning to end.
| A research question is always stated in question form. It may start out being rather general and become focused and refined later on (after you become more familiar with the topic, learn what others have discovered, define your terms more carefully, etc.) |
|
|
The research question you start out with forms the basis for your review of related research literature. This general question also evolves into your hypothesis (or focused research question). When you draw conclusions, they should address this question. In the end, the success of your research depends on how well you answer this question. |
It is important to choose a question that satisfies certain criteria:
- It must not be too broad or general (although you will focus it even more later on in the process).
- It shouldn’t have already been answered by previous research (although replication with variation is certainly acceptable).
- It ought to be a question that needs to be answered (i.e., the answer will be useful to people).
- It must be a question that can be answered through empirical means.
You can go to many sources to find topics or issues that can lead to research questions. Here are a few:
- Personal experience
- Professional books
- Articles in professional periodicals
- Professional indexes (LLBA, MLA, ERIC etc.)
- Other teachers and administrators
- Bibliographies of various types
- Unpublished research by others
| It is wise to focus your research so that it is “do-able.” Be careful! Don’t try to do too much in one study. It is, however, very possible (and quite common) to address several related research questions in one study. This approach is “economical” in that it produces more results with about the same amount of effort. |
|
Here are a couple of examples:
|
Will students learn a foreign language better when they are in a relaxed state of mind? |
|
|
What is the relationship between learners' ages and their accents? |
|
A LITERATURE REVIEW is a formal survey of professional literature that is pertinent to your particular question. In this way you will find out exactly what others have learned in relation to your question. This process will also help frame and focus your question and move you closer to the hypothesis or focused question.
Once you have decided on a general research question, you need to read widely in that area. Use the same sources of information that you consulted when you came up with your general question, but now narrow your focus. Look for information that relates to your research question |
| HYPOTHESIS & FOCUSED QUESTION |
|
In deductive research, a HYPOTHESIS is necessary. It is focused statement which predicts an answer to your research question. It is based on the findings of previous research (gained from your review of the literature) and perhaps your previous experience with the subject. The ultimate objective of deductive research is to decide whether to accept or reject the hypothesis as stated. When formulating research methods (subjects, data collection instruments, etc.), wise researchers are guided by their hypothesis. In this way, the hypothesis gives direction and focus to the research.
Here is a sample HYPOTHESIS:
|
The "Bowen technique" will significantly improve intermediate-level, college-age ESL students' accuracy when pronouncing voiced and voiceless consonants and tense and lax vowels. |
| Sometimes researchers choose to state their hypothesis in “null” form. This may seem to run counter to what the researchers really expect, but it is a cautious way to operate. When (and only when) this null hypothesis is disproved or falsified, the researcher may then accept a logically “alternate” hypothesis. This is similar to the procedure used in courts of law. If a person accused of a crime is not shown to be guilty, then it is concluded that he/she is innocent. |
|
Here is a sample NULL HYPOTHESIS:
|
The Bowen technique will have no significant effect on learners' pronunciation. |
|
In heuristic research, a hypothesis is not necessary. This type of research employs a “discovery approach.” In spite of the fact that this type of research does not use a formal hypothesis, focus and structure is still critical. If the research question is too general, the search to find an answer to it may be futile or fruitless. Therefore, after reviewing the relevant literature, the researcher may arrive at a FOCUSED RESEARCH QUESTION.
Here is a sample FOCUSED RESEARCH QUESTION:
|
Is a contrastive presentation (showing both native and target cultures) more effective than a non-contrastive presentation (showing only the target culture) in helping students understand the target culture? |
|
Very simply, a VARIABLE is a measurable characteristic that varies. It may change from group to group, person to person, or even within one person over time. There are six common variable types: |
DEPENDENT VARIABLES
| . . . show the effect of manipulating or introducing the independent variables. For example, if the independent variable is the use or non-use of a new language teaching procedure, then the dependent variable might be students’ scores on a test of the content taught using that procedure. In other words, the variation in the dependent variable depends on the variation in the independent variable. |
INDEPENDENT VARIABLES
| . . . are those that the researcher has control over. This “control” may involve manipulating existing variables (e.g., modifying existing methods of instruction) or introducing new variables (e.g., adopting a totally new method for some sections of a class) in the research setting. Whatever the case may be, the researcher expects that the independent variable(s) will have some effect on (or relationship with) the dependent variables. |
INTERVENING VARIABLES
| . . . refer to abstract processes that are not directly observable but that link the independent and dependent variables. In language learning and teaching, they are usually inside the subjects’ heads, including various language learning processes which the researcher cannot observe. For example, if the use of a particular teaching technique is the independent variable and mastery of the objectives is the dependent variable, then the language learning processes used by the subjects are the intervening variables. |
MODERATOR VARIABLES
|
. . . affect the relationship between the independent and dependent variables by modifying the effect of the intervening variable(s). Unlike extraneous variables, moderator variables are measured and taken into consideration. Typical moderator variables in TESL and language acquisition research (when they are not the major focus of the study) include the sex, age, culture, or language proficiency of the subjects. |
CONTROL VARIABLES
|
Language learning and teaching are very complex processes. It is not possible to consider every variable in a single study. Therefore, the variables that are not measured in a particular study must be held constant, neutralized/balanced, or eliminated, so they will not have a biasing effect on the other variables. Variables that have been controlled in this way are called control variables. |
EXTRANEOUS VARIABLES
|
. . . are those factors in the research environment which may have an effect on the dependent variable(s) but which are not controlled. Extraneous variables are dangerous. They may damage a study’s validity, making it impossible to know whether the effects were caused by the independent and moderator variables or some extraneous factor. If they cannot be controlled, extraneous variables must at least be taken into consideration when interpreting results. |
Filed under: Uncategorized | Leave a Comment »