Samenvatting Research Methods: The essential knowledge base

-
ISBN-13 9781133954774
330 Flashcards en notities
1 Studenten
  • Deze samenvatting

  • +380.000 andere samenvattingen

  • Een unieke studietool

  • Een oefentool voor deze samenvatting

  • Studiecoaching met filmpjes

Onthoud sneller, leer beter. Wetenschappelijk bewezen.

Dit is de samenvatting van het boek "Research Methods: The essential knowledge base". De auteur(s) van het boek is/zijn Trochim, W m , Donnelly, J P , & Arora, K. Het ISBN van dit boek is 9781133954774. Deze samenvatting is geschreven door studenten die effectief studeren met de studietool van Study Smart With Chris.

PREMIUM samenvattingen zijn gecontroleerd op kwaliteit en speciaal geselecteerd om je leerdoelen nog sneller te kunnen bereiken!

Samenvatting - Research Methods: The essential knowledge base

  • 1 Chapter 8. Introduction to Design - pages 206-221

  • What are the different major types of research design?
    What are the different major types of research designs? You can classify designs into a simple threefold classification by asking some key questions. First, does the design use random assignment to groups? If random assignment is used, the design is a randomized experiment or true experiment. If random assignment is not used, ask a second question: Does the design use either multiple groups or multiple waves of measurement? If the answer is yes, label it as quasi-experimental design. If no, call it a nonexperimental design.
  • Research design can be thought of as the structure of research: the research design tells you yow all the elements in a research project fit together. Every research design should include the following elements:
    1. Time
    A causal relationship, by its very nature, implies that some time has elapsed between the occurrence of the cause and the consequent effect. While for some phenomena the elapsed time might be measured in microseconds and therefore might be unnoticeable to a casual observer, we normally assume that the cause and effect in social science arenas do not occur simultaneously, In design notation we indicate this temporal element horizontally - whatever symbol is used to indicate the presumed cause would be placed to the left of the symbol indicating measurement of the effect. Thus, as we read from left to right in design notation we are reading across time. Complex designs might involve a lengthy sequence of observations and programs or treatments across time.
    1. Program(s) or Treatment(s). 
    The presumed cause may be a program or treatment under the explicit control of the researcher or the occurrence of some natural event or program not explicitly controlled. In design notation we usually depict a presumed cause with the symbol "X". When multiple programs or treatments are being studied using the same design, we can keep the programs distinct by using subscripts such as "X1" or "X2". For a comparison group (i.e., one which does not receive the program under study) no "X" is used.
    1. Observation(s) or Measure(s). 
    Measurements are typically depicted in design notation with the symbol "O". If the same measurement or observation is taken at every point in time in a design, then this "O" will be sufficient. Similarly, if the same set of measures is given at every point in time in this study, the "O" can be used to depict the entire set of measures. However, if different measures are given at different times it is useful to subscript the "O" to indicate which measurement is being given at which point in time.
    1. Groups or Individuals. 
    The final design element consists of the intact groups or the individuals who participate in various conditions. Typically, there will be one or more program and comparison groups. In design notation, each group is indicated on a separate line. Furthermore, the manner in which groups are assigned to the conditions can be indicated by an appropriate symbol at the beginning of each line. Here, "R" will represent a group which was randomly assigned, "N" will depict a group which was nonrandomly assigned (i.e., a nonequivalent group or cohort) and a "C" will indicate that the group was assigned using a cutoff score on a measurement.
  • There are other ways than the research design that may be used to trule out interna validity threats. What does by  preventive  action?
    When potential threats are anticipated they can often be ruled out by some type of preventive action. For example, if the program is a desirable one, it is likely that the comparison group would feel jealous or demoralized. Several actions can be taken to minimize the effects of these attitudes including offering the program to the comparison group upon completion of the study or using program and comparison groups which have little opportunity for contact and communication. In addition, auditing methods and quality control can be used to track potential experimental dropouts or to insure the standardization of measurement.
  • There are other ways than the research design that may be used to trule out interna validity threats. What does by analysismeans?
    There are a number of ways to rule out alternative explanations using statistical analysis. One interesting example is provided by Jurs and Glass (1971). They suggest that one could study the plausibility of an attrition or mortality threat by conducting a two-way analysis of variance. One factor in this study would be the original treatment group designations (i.e., program vs. comparison group), while the other factor would be attrition (i.e., dropout vs. non-dropout group). The dependent measure could be the pretest or other available pre-program measures. A main effect on the attrition factor would be indicative of a threat to external validity or generalizability, while an interaction between group and attrition factors would point to a possible threat to internal validity. Where both effects occur, it is reasonable to infer that there is a threat to both internal and external validity.
  • There are other ways than the research design that may be used to trule out interna validity threats. What does by measurement means?
    In some cases it is possible to rule out a threat by measuring in and demonstrating that either it does not occur at all or occurs so minimally as to not be a strong alternative explanation for the cause-effect relationship. 

    Example: if you are studyin gthe effects of special mathematics training on math achievement scores of children, it might be useful to obsere everyday classroom behavior to verify that students were not receiving other math training in addition to what was provided in the study.
  • There are other ways than the research design that may be used to trule out interna validity threats. What does by argument means?
    THe most straightforward way to rule out a potential threat to validity is simply to make an argument that the threat in question is not a reasonable one.  Such an argument may be made beforte the fact or after the fact. The former is more convincing than the latter.
  • There are other ways than the research design that may be used to rule out internal validity threats. Which 4?
    - By argument
    - By measurement of observation
    - By analysis
    - By preventive action
  • CompensatorOne of the 4 major social interaction threats to internal validity: equalization of treatment
    What does it mean?
    Compensatory equalization is a threat to the treatment construct validity in intervention studies, in which comparison groups not obtaining the preferredtreatment are provided with compensations that make the comparison groups more equal than originally planned.
  • One of the 4 major social interaction threats to internal validity: Resentful demoralization
     What does it mean?
    Resentful demoralization is an issue in controlled experiments in which those in the control group become resentful of not receiving the experimental treatment. ... This may lead to significant systematic differences in the outcome of the control group, obscuring the results of the study and threatening their validity
  • One of the 4 major social interaction threats to internal validity: Compensatory rivalry  

    What does it mean
    Compensatory rivalry exists when the study group not receiving the experimental treatment (i.e., intervention) feels disadvantaged, disappointed, or left out and decides to obtain a similar intervention on its own. It may or may not involve intentional competition or rivalry with the experimental group.
  • One of the 4 major social interaction threats to internal validity: Diffusion or imitation of treatment
     What does it mean?
    This occurs when a comparison group learns about the program either directly or indirectly from program group participants. In a school context, children from different groups within the same school might share experiences during lunch hour.
  • What are the 4 major social interaction threats to internal validity?
    What are the 4 major social interaction threats to internal validity?
    -Diffusion or imitation of treatment
    This occurs when a comparison group learns about the program either directly or indirectly from program group participants. In a school context, children from different groups within the same school might share experiences during lunch hour.
    -Compensatory rivalry
    Compensatory rivalry exists when the study group not receiving the experimental treatment (i.e., intervention) feels disadvantaged, disappointed, or left out and decides to obtain a similar intervention on its own. It may or may not involve intentional competition or rivalry with the experimental group.
    -Resentful demoralization
    Resentful demoralization is an issue in controlled experiments in which those in the control group become resentful of not receiving the experimental treatment. ... This may lead to significant systematic differences in the outcome of the control group, obscuring the results of the study and threatening their validity.
    -Compensatory equalization of treatment
    Compensatory equalization is a threat to the treatment construct validity in intervention studies, in which comparison groups not obtaining the preferredtreatment are provided with compensations that make the comparison groups more equal than originally planned.
  • What are "Social" Threats?
    Threats to internal validity that arise because social research is conducted in real-world human contexts where people will react to not only what affects them.
  • How to solve single and multiple group threats?
    For instance, instead of doing a single group study, you could incorporate a control group. In this scenario, you would have two groups: one receives your program and the other one doesn't. In fact, the only difference between these groups should be the program. If that's true, then the control group would experience all the same history and maturation threats, would have the same testing and instrumentation issues, and would have similar rates of mortality and regression to the mean. In other words, a good control group is one of the most effective ways to rule out the single-group threats to internal validity. Of course, when you add a control group, you no-longer have a single group design. And, you will still have to deal two major types of threats to internal validity: the multiple-group threats to internal validity and the social threats to internal validity.

    There really is only one multiple group threat to internal validity: that the groups were not comparable before the study.  This ensures that each participant or subject has an equal chance of being placed in any group (called random assignment). Random assignment of participants helps to ensure that any differences between and within the groups are not systematic at the outset of the experiment.
  • What are the major type of threats to internal validity?
    Single Group threats
    Multiple group threats
    Social interaction threats
  • There really is only one multiple group threat to internal validity:
     that the groups were not comparable before the study. We call this threat a selection bias 
  • Regression to the mean
    This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment.
  • Morality threat
    It means that people are "dying" with respect to your study. Usually, it means that they are dropping out of the study.
  • Instrumentation threat
    the changes in the instrument, observers, or scorers which may produce changes in outcomes.


    Example: In many schools when they have to administer repeated testing they don't use the exact same test (in part because they're worried about a testing threat!) but rather give out "alternate forms" of the same tests. These alternate forms were designed to be "equivalent" in the types of questions and level of difficulty, but what if they aren't? Perhaps part or all of any pre-post gain is attributable to the change in instrument, not to your program.
  • Testing threat
    the effects of taking a test on the outcomes of taking a second test.

    What if taking the pretest made some of the children more aware of that kind of math problem -- it "primed" them for the program so that when you began the math training they were ready for it in a way that they wouldn't have been without the pretest. This is what is meant by a testing threat -- taking the pretest (not getting your program) affects how participants do on the posttest.
  • Maturation threat
    the processes within subjects which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment.
  • History threat
    the specific events which occur between the first and second measurement.

    For instance, we know that lot's of first graders watch the public TV program Sesame Street. And, we know that in every Sesame Street show they present some very elementary math concepts. Perhaps these shows cause the outcome and not your math program.
  • What are the 6 single group threats to internal validity?
    -          Regression to the mean
    This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment.


    -          Morality threat
    It means that people are "dying" with respect to your study. Usually, it means that they are dropping out of the study.


    -          Instrumentation threat
    the changes in the instrument, observers, or scorers which may produce changes in outcomes.
    Example: In many schools when they have to administer repeated testing they don't use the exact same test (in part because they're worried about a testing threat!) but rather give out "alternate forms" of the same tests. These alternate forms were designed to be "equivalent" in the types of questions and level of difficulty, but what if they aren't? Perhaps part or all of any pre-post gain is attributable to the change in instrument, not to your program.
    -          Testing threat
    the effects of taking a test on the outcomes of taking a second test.
    What if taking the pretest made some of the children more aware of that kind of math problem -- it "primed" them for the program so that when you began the math training they were ready for it in a way that they wouldn't have been without the pretest. This is what is meant by a testing threat -- taking the pretest (not getting your program) affects how participants do on the posttest.


    -          Maturation threat
    the processes within subjects which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment.


    -          History threat
    the specific events which occur between the first and second measurement.
    For instance, we know that lot's of first graders watch the public TV program Sesame Street. And, we know that in every Sesame Street show they present some very elementary math concepts. Perhaps these shows cause the outcome and not your math program.
  • Third-variable problem
    A type of confounding in which a third variable leads to a mistaken causal relationship between two others. For instance, cities with a greater number of churches have a higher crime rate.
  • What are single group threats?
    A threat to internal validity that occurs in a study that uses only a single program or treatment group and no comparison of control group
  • No Plausible Alternative Explanations
    All other possible explanations for the effect need to be ruled out.       

    That there is a relationship does not mean that there is a causal one.
    Suppose you measure the association between children's shoe size and their reading. In almost all cases you will find a positive relationship. But it is not the shoe size, but it's just that the child is getting older.

    It's always possible that some other variable or factor is causing the outcome. This is sometimes referred as the third-variable or missing-variable problem
  • Covariation of the cause and effect
    Before you can show that you have a causal relationship you have to show that you have a relationship at all. For instance, consider the logical argument if X then Y if not X then not Y. 

    Or in terms of a treatment and its effect: whenever you administer a program, you observe the outcome, but when you don't give the program, you don't observe the outcome. This provides the program and the outcome are related
  • Temporal precedence
    To establish temporal precedence you have to shown that the cause happened  BEFORE the effect.
  • What about internal validity? 
    IInternal validity : is the relationship causal?
    Internal validity is the approximate truth about inferences regarding cause-effect or causal relationships.

    Establishing internal validity is just a technical way of saying that you have ruled plausible alternative explanations an have thus shown that indeed a causal relationship likely exists between your program and the intended outcome.

    Internal and external validity are concepts that reflect whether or not the results of a study are trustworthy and meaningful.

    Internal validity refers specifically to whether an experimental treatment/condition makes a difference or not, and whether there is sufficient evidence to support the claim.
  • How do we establish a cause-effect (causal) relationship?  More specifically, what features need to be considered in you research design in order to evaluate how well it can support causal conclusions.
    What criteria do we have to meet? Generally, there are three criteria that you must meet before you can say that you have evidence for a causal relationship:
    1.Temporal precedence
    The cause needs to happen BEFORE the effect.

    2.Covariation of the cause and effect
    ALL other possible explanations for the effect need to be ruled out.

    Before you can show that you have a causal relationship you have to show that you have a relationship at all. For instance, consider the logical argument if X then Y if not X then not Y. 

    Or in terms of a treatment and its effect: whenever you administer a program, you observe the outcome, but when you don't give the program, you don't observe the outcome. This provides the program and the outcome are related.


    3.No Plausible Alternative Explanations
    All other possible explanations for the effect need to be ruled out.

    That there is a relationship does not mean that there is a causal one.
    Suppose you measure the association between children's shoe size and their reading. In almost all cases you will find a positive relationship. But it is not the shoe size, but it's just that the child is getting older.

    It's always possible that some other variable or factor is causing the outcome. This is sometimes referred as the third-variable or missing-variable problem
  • What is research?
    Research is a type of systematic investigation that is impirical in nature and is designed to contribute to public knowledge
    1.Systematic investigation
    2.Empirical endeavor
    3.Public effort 
  • What is research enterprise?
    The macro level effort to accumulate knowlege across multiple empirical systematic public research projects.
    The book concentrates most on how you learn to conduct a research project, a specific investigation of a question interest. But it is important that you understand the broader effort that each research projects contributes to.
  • What does translational research mean?
    The systematic effort to move research from initial discovery to practice and ultimately to impacts on our lives
  • Evidence-based practice:
    A movement to encourage practioners to employ practices that are based on research evidence
  • Research philosophy
    Positivism: the best way to acquire knowledge is to conduct true experiments (gold standard) (- Interpretivism
    who believe in the methods of the natural science. Believe that valid knowledge is based on such experiments
  • What does experimental design as the ' gold standard' means?
    Experimental designs are touted by many as the most ' rigorous' of all research designs. 

    From a theoretical perspective, they probably are the strongest of designs, but we have to be careful to understand what is meant by that .If you can implement an experimental design well , the experiment is probably the strongest design with respect to internal validitiy.
  • True experiment. You need to have
    ·experimental group and a control group.
    ·Random allocation (problisticalyy aquivalent)
  •  Treatment group, control group
    In a comparative research design, like an experimental or quasi experimental design, the program or treatment group receives the program of interest and is usually contrasted with a no-treatment comparison or control group or a group receiving another treatment. 
  • What is the ‘classical’ dictator game?
    The dictator game is a popular experimental instrument in social psychology and economics, a derivative of the ultimatum game. The term "game" is a misnomer because it captures a decision by a single player: to send money to another or not. The results – most players choose to send money.

    In the dictator game, the first player, "the dictator", determines how to split an endowment (such as a cash prize) between himself and the second player. The dictator's action space is complete and therefore is at his own will to determine the endowment, which means that the recipient has no influence over the outcome of the game.

    Most people give money because inequity aversion. John list does not believe this outcome. They say people dislike equality, he does not believe that. He started a true experiment
  •  What are the most important aspects of the research method of List?

    He started a true experiment. They are being asked how much they are going to share:
    4 groups
    1.Baseline (control group)
    2.Experiment group 2 : Take 1 (you can take and you can share )
    3.Experiment group 3: Take 5
    4.Experiment group 4: Take 5  + earnings (he need to do stupid work for 30 minutes and get money he knows that someone in the other room do the same) 
  • What are the main results?
    Moral cost function (social norms) . He changes the social norms and that makes the outcome different. It has nothing to do with inequality aversion  but with social norms.
    It shows that all scientific knowledge is tentative and provisional, and nothing is final
    (Example first we believe Homo economics than Inequity aversion than is Moral cost function)
  • Random sampling vs Random allocation
    ·Random sampling is about how you get people to your lab
    ·Random allocation is about how do you allocate people over the groups. 
  •  convenience sampling:
    ·convenience sampling:  John list used convenience (gemak) sampling: student from a specific school, probably the students who needed money
  • Proximal Similarity model:
    In the sampling model, you start by identifying the population you would like to generalize to. Generalize to groups with similaritys
  • Reverse causality 
    Reverse causality means that X and Y are associated, but not in the way you would expect. Instead of X causing a change in Y, it is really the other way around: Y is causing changes in X.
  • Validity

    Does your test actually measure what it is designed to measure? 
  • What about external validity?
    External validity: Cn we generalize to other persons, places, times?
    While internal validity relates to how well a study is conducted (its structure), external validity relates to how applicable the findings are to the real world.

    Can you generalize it to a different situation? The degree to which the conclusions in your study would hold for other places and at other times. 
  • What about construct validity?
    Construct validity: Can we generalize to the construts?

    The degree to which inferences can be legitimately be made from the operationalization in your study to the theoretical constructs on which those operationalizations are based.

    List his theoretical construct = social norms. You need to have a measurement in real life. His action sets according are about social norms. 
  •  Experiments conducted in a lab have certain strengths and weaknesses. Illustrate these with List's experiment (2007) and use the criteria internal validity, construct validity and external validity described on page 28 of the book.
    -Internal validity
    High on internal validity because it’s a true experiment conducted in the lab.
    -External validity
    List  is proving al former experiments are wrong. This his study is low on external validity. Are the results like this because of the students?

    -Construct validity
    List his construct is about social norms. Criticism on construct validity = Is there realy an room b?
Lees volledige samenvatting
Deze samenvatting. +380.000 andere samenvattingen. Een unieke studietool. Een oefentool voor deze samenvatting. Studiecoaching met filmpjes.