Lecture Notes

Details
Parent Category: Level 2

Concept 1:   Understanding research components

Concept 2:   Implementing research processes

Topic A:   Ethics in Research

Ethical Issues

Ethical issues in research may fall into four categories:

  • Protection from harm
  • Informed consent
  • Right to privacy
  • Honesty with professional colleagues

Protection of Human Subjects in Research - responsible for implementing regulations for the protection of human subjects and for providing guidance on the requirements for complying with the regulations.  

Protection of Animal Subjects in Research - Any research that causes suffering, distress, or death to animals must be described and adequately justified to an institutional animal care and use committee (IACUC). The researcher must also minimize or prevent such suffering and death to research animals to the extent that it is possible to do so.     

One example of a research institutions' IACUC   policies and procedures is listed below:

       http://www.ahsc.arizona.edu.uac/iacuc/index.shtml

Regulations under the Animal Welfare Act (Public Law 89-544) contain standards for the humane handling, care, treatment, and transportation of certain animals by dealers, exhibitors, researchers, and other regulated entities.

  • Guide for the Care and Use of Agricultural Animals in Agricultural Research and Teaching, published by the Federation of American Societies of Food and Animal Science
  • Guide for the Care and Use of Laboratory Animals, published by the Institute of Laboratory Animal Resources

Regulations & Institutional Responsibilities

Internal Review Board (IRB) requires that all covered research projects be reviewed and approved by the board before the research takes place. The IRB is made up of scholars and researchers across various disciplines, who checks proposed research studies to make sure that the procedures are not unduly harmful to participants, that appropriate procedures will be followed to obtain participants' informed consent, and that participants' privacy and anonymity are assured. The IRB is authorized to approve, request modification in, or disapprove research activities and to conduct continuing reviews of the research activities at intervals appropriate to the degree of risk, but not less than once a year.

Some examples of web sites listing institutional policies and procedures are listed below:

In order to conduct research at a potential research site, you must obtain a permission letter.   A permission letter is a document that you obtain from a potential research site when you are seeking to use their facilities, contact their members, or access data that they own.   When you are trying to get permission from a site, you must be sure you are contacting the appropriate authority.   Also you must check to see if the site you want to utilize has an Institutional Review Board.   If it does you will need to seek their approval as well as the one at your own institution.  

Ethical Principles in Conducting Research

Professional Codes of Ethics - Many national associations have developed their own codes of ethical standards governing research that involves human subjects and/or research involving animal subjects.     

Examples of some of these codes can be found from these professional associations' websites:

  • The American Psychological Association's Ethical principles of Psychologists and Code of Conduct, 2002, available at

www.apa.org/ethics/code.html

Topic B:   Six Steps of Research

When doing research there are six steps that a researcher typically follows:

  •   State the problem -The first thing you must do when embarking on a new research project is determine what the problem or issue is that you would like to investigate.   A good research question should bridge a gap in knowledge.   Examples of research questions:
    • Does calcium impact weight loss in African American post-menopausal women?
    • What is the influence of parental marital satisfaction on externalizing behaviors in adolescent boys?
  • Perform a literature review -Performing a literature review is essential for placing your research question within the context of known research in the subject area.   It provides a conceptual framework for your research question and can direct you towards your hypotheses.   See Level 1 for the steps to conducting a literature review.
  • State hypotheses -Hypotheses are predictions that the researcher makes about the research question.   These should be made before conducting an experiment.   Examples of research hypotheses:
    • Post-menopausal African American women who take 200mg of calcium/day will lose significantly more weight than those who do not.
    • Adolescent boys who have parents that report high levels of marital satisfaction will have a lower amount of externalizing behavioral problems such as alcohol use and truancy than those with parents who report low levels of marital satisfaction.
  • Test hypotheses -In order to test your hypotheses, first you must determine what type of research you are doing; quantitative, qualitative, or mixed methods which is when you utilize techniques of both quantitative and qualitative research.   The research question and literature review will dictate the type of research methods you use to test your hypotheses.   Data collection methods will be described later within this unit.
  • Analyze results-How the data will be analyzed depends on the type of research that you are doing.   Data analysis is described later within this unit.
  • State conclusions and disseminate research -The last step of the research problem is stating conclusions and disseminating the research results.   Result dissemination is important to furthering your field of study by contributing to the body of knowledge.   Research dissemination will be discussed further in Level 3.

Topic C:   Research Types

Quantitative Research is used to answer questions about relationships among measured variables with the purpose of explaining, predicting, and controlling phenomena. In quantitative research, the purpose statement, research questions, or hypotheses:  

  • Are specific and narrow
  • Seek measurable, observable data on variables

The major statements and questions of directions in a study, e.g., the purpose statement, the research questions, and the hypothesis, are specific and narrow because only a few variables are identified in a study.   From a study of these variables, the researcher obtains measures or assessments on an instrument or record scores on a scale from observations.  

For example:

A study of adolescent career choices, the variable, the role of the school counselor, narrows the study to a specific variable from among many variables that might be studied (e.g., role of parents, personal investment by student). To obtain the impact of the school counselor on adolescent career choices, data must be obtained from the students by collecting personal on each, such as gender, social economic status, grade attended and school achievement.

Qualitative Research is typically used to answer questions about the complex nature of phenomena, often with the purpose of describing and understanding the phenomena from the participants' point of view. In qualitative research, the purpose is more open-ended than in quantitative research.   Broader questions are asked in order to best learn from participants.

The investigator researches a single phenomenon of interest and statse the phenomenon in a purpose statement.  

For example:

'What does it mean to be a professional?' This question focuses on understanding a single idea.   The responses from this question will yield qualitative data, such as quotations.

Quantitative Research Designs

  • Experimental Designs are an investigation characterized by the direct manipulation or control of one variable (i.e., the cause) so its effect can be seen on another variable (i.e., the effect).   All other variables (i.e., extraneous variables) that might have an effect are controlled.
    • True experimental designs are characterized byrandom assignment of participants to groups to different conditions of the experimental variable.   For example, randomly assign students to one of two classrooms in which the same social studies unit is being taught; teach the first class using a traditional lecture approach, teach the second class using co-operative learning groups; examine the differences between the two groups to see if the type of instructional approach had an effect
    • Quasi-experimental include non-random assignment of participants to groups.   For example, get access to two classrooms in which the same social studies unit is being taught; teach the first class using a traditional lecture approach, teach the second class using co-operative learning groups; examine the differences between the two groups to see if the type of instructional approach had an effect.
  • Non-experimental Designs: an investigation describing current status of a variable or the relationships, other than causal, among variables
    • Descriptive: simple descriptive information about the frequency or amount of something.   Example:   Describe the current dropout rate in Georgia
    • Comparative:   descriptions of the differences between groups.

      Example: Describe the relationship between student attitude and student achievement

    • Causal-comparative designs involve looking at existing conditions that have already occurred and then collects data to investigate the possible relationships between these conditions. For example:   Identify a group of 'comparable adults some of whom drink excessively, some who drink moderately, and some who drink very little or not at all; compare the health of the three groups; discuss the effect that alcohol might have on the health of the subjects
    • Correlational:   The researcher uses this technique to measure the degree of relationships between two or more variables or sets of scores. Example:   Describe the relationship between student attitude and student achievement

Qualitative Research Designs   

  • Naturalistic
    • Ethnographic: The investigator is interested in studying one group of individuals, in examining them in the setting where they live and work, and in developing a portrait of how they interact.   An ethnographic designs are qualitative procedures for describing, analyzing, and interpreting a cultural group's shared patterns of behavior, beliefs, and language that develop over time.   The researcher provides a detailed picture of the culture-sharing group, drawing on various sources of information.   The ethnographer also describes the group within its setting, explores themes or issues that develop over time as the group interacts, and details a portrait of the group.   To understand the patterns of a culture-sharing group, the ethnographer typically spends considerable time 'in the field' interviewing, observing, and gathering documents about the group in order to understand their culture-sharing behaviors, beliefs, and language.  
    • Case study: An in-depth study of a single program (e.g., an activity, event, process, or individuals) based on extensive data collection. A 'case' may be a single individual, several individuals separately or in a group, a program, events, or activities (e.g., a student, several students, or the implementation of a new program).  
      • A 'case' may represent a process consisting of a series of steps (e.g., a teacher education curriculum process) that form a sequence of activities.  
      • A 'case' may be selected for study because it is unusual and has merit in and of itself.   When the case itself is of interest, it is called an intrinsic case.   'The study of a bilingual school illustrates this form of a case study (Creswell, 2005 & Stake, 2000). Alternatively, the focus of a qualitative study may be a specific issue, with a case or cases uses to illustrate the issue.   This type of case is called an instrumental case, because it serves the purpose of illuminating a particular issue.   Case studies may include multiple cases, called a collective case study (Stake, 1995), in which multiple cases are described and compared to provide insight into an issue.

        Example:   A case study researcher might examine several schools to illustrate alternative approaches to school choice for student
      • The researcher seeks to develop an in-depth understanding of the case by collecting multiple forms of data (e.g., pictures, scrapbooks, videotapes, and e-mails).   Providing this in-depth understanding requires that only a few cases be studied, because for each case examined, the researcher has less time to devote to exploring the depths of any one case
      •   The researcher also locates the 'case' or 'cases' within their larger context, such as geographical, political, social, or economic settings (e.g., the family constellation consisting of grandparents, siblings, and 'adopted' family members).
    • Phenomenology study describes the meaning of the lived experiences for several individuals about a concept or phenomenon. A phenomenological study may be challenging to use for the following reasons:
      • The researcher requires a solid grounding in the philosophical precepts of phenomenology.
      • The participants in the study need to be carefully chosen to be individuals who have experienced the phenomenon.
      • Bracketing personal experiences by the researcher may be difficult.  
      • The researcher needs to decide how and in what way his or her personal experiences will be introduced into the study.
    • Grounded theory designs -studying a particular environment to generate or discover a theory that describes it (e.g., understanding the meaning of 'inclusion' from the perspectives of the special needs student, the regular student, and the teachers).
      •   
  • Analytical   - A mode of inquiry in which events, ideas, concepts, or artifacts are investigated by analyzing documents, records, recordings, and other media. There are two types:
    • Historical analysis is the systematic gathering and criticism of documents, records, and artifacts to provide a description and interpretation of past events.   For example, Biography (Narrative research design) are qualitative procedures in which researchers describe the lives of individuals, collect and tell stories about these individuals' lives, and write narratives about their experiences.  
    • Content analysis is a detailed and systematic examination of the contents of a particular body of material for the purpose of identifying patterns, themes, or biases. Content analyses are basically performed on   forms of human communication,   including newspapers, books, films, television, art, music, videotapes of human interactions, and transcripts of conversation.
      For example, a researcher might use a content analysis to determine whether television commercials reflect traditional sex-role stereotypes, what religious symbols appear in works of art, how teachers spend their time in the classroom, or what attitudes are reflected in the speeches or newspaper articles of a particular era in history (Leedy and Ormrod, P. 155-156).

Topic D:   Data Collection Methods

Experiment

Random assignment is the process of process of assigning individuals at random to groups or to different groups in an experiment (Creswell, 2005)

Variable is defined as a symbolic representation of an item that can vary.   An example of a variable is class size.   There are two types of extraneous variables; independent and dependent.  

An independent variable (treatment variable) is the variable that is being manipulated or changed in some way.   Examples of independent variables:

  • Number of hours of sleep before a test
  • Method of SAT preparation

A dependent variable (outcome measure) is the response, criterion, or post-test.   It is the dependent variable that is the presumed effect of the treatment variable.   Examples of dependent variables:     

  • Achievement scores on an EOCT.
  • Test scores on the SAT

Control over extraneous variables:

  • Pre-test provides measure on some attribute or characteristic that you assess for participants in an experiment before they receive a treatment.
  • Post-test is a measure on some attribute or characteristic that is assessed for participants in an experiment after a treatment.
  • Covariates are variables that the researcher controls for using statistics and that relate to the dependent variable, but do not relate to the independent variable.  

Manipulation of the treatment conditions

  • Identify a treatment variable:   type of classroom instruction in the civics class
  • Identify the conditions (or levels) of the variable:   classroom instruction can   be (a) regular topics or (b) topics related to the health hazards of smoking
  • Manipulate the treatment conditions:   provide special activities on health hazards of smoking to one class and withhold them from another class (Creswell, 2005, P. 288).

Group comparisons is the process of a researcher obtaining scores for individuals or groups on the dependent variable and comparing the means and variance both within the group and between the groups.

Threat to validity should be designed to minimize compromises in drawing good conclusions from the scores that are obtained from the experiment. A threat to validity means that design issues may threaten the experiment so that the conclusions reached from data   may provide a false reading about probable cause and effect between the treatment and the outcome (Creswell, 2005, Bracht & Glass, 1968; Campbell & Stanley, 1963 & Campbell, 1979, P. 290).

Threats to internal validity are problems that threaten our ability to draw correct cause-and-effect inferences that arise because of the experimental procedures or the experiences of participants (Creswell, 2005, P. 290).

  • history
  • maturation
  • regression
  • selection
  • mortality
  • interactions with selection
  • diffusion of treatments
  • compensatory equalization
  • resentful demoralization
  • testing instrumentation

Threats to external validity are problems that threaten our ability to draw inferences from the sample data to other persons, settings, and past and future situations (Creswell, 2005) According to Creswell (2005), Cook and Campbell (1979), three threats may affect the results:

  • interaction of selection and treatment
  • interaction of setting and treatment
  • interaction of history and treatment

Steps in Conducting Experimental Research:

  1. Decide if an experiment addresses your research problem
  2. Form the hypotheses to test cause-and-effect relationship
    • A hypothesis is a prediction about the outcome
    • Independent variables contain one or more variables with multiple levels and the researcher needs to manipulate one of the levels
    • Dependent variables are outcomes (e.g., student learning and attitudes)
  3.   Select an experimental unit and identify study participants
    •   An experiment unit is the smallest unit receiving treatment (e.g.,   single individual, several individuals, a group, several groups, or an entire organization)
    • Participants in an experiment are the individuals tested by the researcher to determine if the intervention made a difference in one or more outcomes.
  4.   Select an experimental treatment
  5.   Choose a type of experimental design
  6. Conduct the experiment
    1. Administering a pre-test
    2. Introducing the experimental treatment to the experimental group or relevant groups
    3. Monitoring the process closely so that the threats to internal validity are minimized
    4. Gathering post-test measures (the outcome or dependent variable measures)
    5. Using ethical practices by debriefing the participants by informing them of the purpose and reasons for the experiment (Creswell, 2005 & Neuman, 2000).
  7. Organize and analyze the data
    1. coding the data
    2. analyzing the data
    3. writing the experimental report
  8. Develop an experimental research report.   The following information is typically included in the methods or procedures for an experiment:
    1. participants and their assignment
    2. the experimental design
    3. the intervention and materials
    4. control over extraneous variables
    5. dependent measures or observations

Observation

One method of collecting qualitative data. Observation is the process of gathering open-ended, firsthand information by observing people and places at a research site (Creswell, 2005, P. 211).

The three most popular roles of an observer are listed below:

  • A participant observer takes part in activities in the setting they observe to learn more about a situation.   The participant takes on the role of an 'insider' observer who engages in activities at the site, and records information at the same time.   The participant must seek permission to participate in the activities and as an observer.
  • A nonparticipant observer may not be familiar enough with the site and people to participate in the activities; therefore a nonparticipant observer visits a site and records notes without becoming involved in the activities of the participants. The nonparticipant observer takes on the role as an 'outsider' by sitting at the back of a classroom or some other advantageous place to watch and record the phenomenon under study.
  • Changing observational role is one where the researcher adapts their role to the situation.   For example, a researcher might first observe as an 'outsider,' then participate in the setting and observe as an 'insider.

The Observational Process as outlined by Creswell, 2005, P. 212-213.

  1. Select a site to observed and obtain the required permissions needed to gain access to the site.   The site selected should be based on how a site can best help the researcher understand the central phenomenon.
  2. Ease into the site slowly by looking around; getting a general sense of the site; and taking few notes.
  3. Identify who or what to observe, when to observe, and how long to observe at the particular site.
  4. Determine, initially, your role as an observer
  5. Conduct multiple observations over time to obtain the best understanding of the site and the individuals.  
  6. Design some means for recording notes during an observation
  7. Consider what information you will record during the observation
  8. Record descriptive (e.g., events, activities and people) and reflective (e.g., personal thoughts, hunches) field notes
  9. Make yourself known, be passive, friendly and respectful.
  10. After observing at the site, thank the participants and inform them of the use of the data.

Interview

The interview is another method of collecting qualitative data.   An interview is when the researcher asks one or more participants, general, open-ended questions and records their answers.  

Types of Interviews:

  1. One-on-one interviews
    • Most time-consuming and costly  
    • Ask questions and records information from only one participant in the study at a time  
    • Interview participants who are not hesitant to speak, or articulate, and who can share ideas comfortable  
  2. Focus group interviews
    • Collecting data through interviews with a group of people  
    • Ask small number of general questions and get responses from all individuals in the group  
    • Advantageous when the interaction among interviewees will yield the best information and when interviewees are similar to and cooperative with each other.  
    • Useful when the time to collect information is limited and individuals are hesitant to provide information.  
  3. Telephone interviews
    • Participants may be geographically dispersed and unable to come to a central location for an interview.  
    • Researcher does not have direct contact with the participant.  
    • May involve substantial costs for telephone time.  
  4. Electronic e-mail interviews
    • Collecting open-ended data through interviews with individuals using computers and the internet
    • Provides rapid access to large numbers of people and a detailed, rich text data base for qualitative analysis
    • Promote a conversation between the researcher and the participants to extend an understanding of the topic or central phenomenon being studied.
    • Will probably increase due to the expanding possibilities of technology  
  5. Open-ended questions on questionnaires
    • Permit the researcher to explore reasons for the comments.
    • Help to identify any comments people might have that are beyond the responses of any closed-ended questions.
    • Identify overlapping themes

Conducting the Interviews

  1.   Identify the interviewees
  2.   Determine the type of interview you will use
  3. During the interview, audiotape the questions and responses
  4. Take brief notes during the interview
  5. Locate a quiet, suitable place for conducting the interview
  6. Obtain consent from the interviewee to participate in the study
  7. Have a plan, but be flexible
  8. Use probes to obtain additional information
  9. Be courteous and professional when the interview is over

Survey   

Procedures in quantitative research in which investigators administer a survey to a sample or to the entire population of people in order to describe the attitudes, opinions, behaviors, or characteristics of the population.   Survey researchers collect quantitative, numbered data using questionnaires or interviews and statistically analyze the data to describe trends about responses to questions and to test questions or hypotheses. Survey designs are different from experimental research in that they do not involve a treatment given to participants by the researcher. Because survey researchers do not experimentally manipulate the conditions, they cannot explain cause and effect as well as experimental researchers.   Survey researchers often correlate variables, but their main focus is directed toward learning about a population and less on relating variables or predicting outcomes as is the focus in correlational research.  

Types of survey designs:   cross-sectional and longitudinal

  • Cross-sectional survey design   are used to collect data about current attitudes, opinions, or beliefs at one point in time.
  • Longitudinal survey designs are used to study individuals over time. It involves collecting data about trends with the same population, changes in a cohort group or subpopulation, or changes in a panel group of the same individuals over a period of time.

Document Analysis

Documents consist of public and private records that qualitative researchers can obtain about a site or participants in a study.

Some examples of public and private records are:  

  • newspapers
  • minutes of meetings
  • official memos
  • records in the public domain
  • archival materials in libraries and museums
  • personal journals/diaries
  • letters
  • personal notes
  • e-mail comments
  • website data

Artifact Analysis

Artifact analysis is another form of qualitative research where tangible items such as texts and photographs are collected and analyzed as data.   This method of data collection may help give a researcher a better sense of context or may also give insights to things that would have otherwise not been reported.   Artifacts include images or sounds that a researcher collect to help them better understand the central phenomenon under study.   Some examples of artifacts are:

  • photographs
  • videotapes
  • digital images
  • paintings
  • pictures

Topic E:   Instrumentation

Quantitative Research

An instrument is used to measure the variables in the study.  

An instrument is defined as a tool for measuring, observing, or documenting quantitative data.   The instrument contains specific questions and response possibilities that are established and developed in advance of the study.

Examples of these instruments are:

  • survey questionnaires
  • standardized tests
  • checklists

These instruments might be used to observe a student's or teacher's behaviors.   The instrument is administered to participants to collect data in the form of numbers.   The larger the number of individuals studied, the stronger is the case for applying the results to a large number of people.

Qualitative Research

In qualitative research, the researcher does not begin the data collection with a pre-established instrument to measure distinct variables. Instead, the data collection consists of:

  • collecting data using forms with general, emerging questions to permit the participant to generate responses
  • gathering word (text) or image (picture) data
  • collecting information from a small number of individuals or sites

With each form of data, the researcher gathers as much information as possible to collect detailed accounts for a final research report.

Topic F:   Measurement

Measurement Scales

  1. Nominal -The values of the scale have no numerical value.   Instead they represent categories.   Mathematical operations you can do with nominal scales include counting.   Examples of appropriate statistical operations include 'cross-tab operations' such as chi-square tests and kappa values.
    1. Race
    2. Gender
    3. Types of Schools (e.g. public, private, parochial)
  2. Ordinal -Ordinal scales show that people or objects with a higher value have more of some attribute.   In other words, they represent ordered categories.   In an ordinal scale, the intervals between adjacent scale values are indeterminate.   Mathematical operations you can do with ordinal scales include greater than or less than operations.   Examples of appropriate statistical operations include 'frequencies operations' such as medians, interquartile ranges, and gamma.  
    1. Finishing position in a race
    2. Ranks in the military
    3. Grade levels
  3. Interval -These values represent adjacent scale values that are equal with respect to the attribute being measured.   For example, the difference between 8 and 9 is the same as the difference between 899 and 900.   Mathematical operations you can do with interval scales include addition and subtraction of scale variables.   Examples of appropriate statistical operations include descriptive statistics such as mean, standard deviations, Pearson's correlations, t-tests, analysis of variance (ANOVA), factor analysis, regression, and multiple correlations.
    1. Test scores
    2. Achievement levels
  4. Ratio -These values have equal intervals and an absolute zero (0).   Also ratios are equivalent.   For example, the ratio of 3 to 6 is the same as the ratio of 1 to 3.   Mathematical operations you can do with ratios include multiplication and division of scale values.   Examples of appropriate statistical operations include the coefficient of variation.
    1. Height
    2. Weight
    3. Time

Validity of Measurements

Validity shows the degree at which constant error is controlled.   When a scale has a high validity then the average of each of the trials will be close to the measurement that you have set out to measure.   Individual measurements may be valid but not reliable.   There are two types of validity within an experimental design: internal and external.

Internal Validity:   Refers to the study design and the extent that researchers explored alternative explanations for experimental conclusions.

External Validity: The extent at which results of a study are generalizable or transferable.   In other words, external validity refers to the ability to apply conclusions of a study to a larger group.

Reliability of Measurement

Reliability is the extent that you get the same result each time you perform an action.   It is possible to be reliable without being accurate (or having validity).   If you repeat an experiment 100 times, the amount of times you get the same result will show you how reliable the tool of measurement is.   Reliability can be measured in three different ways:

  1.   The degree in which measurement remains the same after repeated action.
  2. The degree the measurement remains the same over time.
  3. The similarity within the measurements in a given time period.

Topic G:   Sampling

Subjects, participants, and samples:

A subject or a participant is a person from whom data are collected.   The term subject is typically used in a quantitative context.   In a qualitative context, the term participant is more generally used. A sample is defined as the collective group of subjects or participants from whom data are collected .

There are two types of sampling procedures:

  1.   Probability
    • Statistically driven sampling techniques where the probability of being selected is known
    • Purpose is to select a group of subjects representative of the larger group of subjects from which they are selected
  2. Non-probability
    • Pragmatically driven sampling techniques where the probability of being selected is not known
    • Purpose is to select subjects who can be particularly informative about the research issues

Probability sampling

Method of sampling in which subjects are selected randomly from a population in such a way that the researcher knows the probability of selecting each subject

  • In a sample of 10 from a population of 100, each subject has a 10% chance of being included in the sample
  • In a sample of 50 from a population of 100, each subject has a 50% chance of being in included in the sample

Population: a large group of individuals to whom the results of a study can be generalized

Target population: the group to whom the results are intended to be generalized  

Sampling frame (i.e., survey population or accessible population)

  • The group to whom the researcher has access and from which the actual sample will be drawn
  • Often the sampling frame and the target population are different
    • Example - The target population could be all fourth graders in a county; the sampling frame is fourth graders in public schools in a county (i.e., excluding private and parochial school students due to their inaccessibility)
    • Example - The target population could be all graduate students at a University; the sampling frame is all graduate students in the College of Education at a University (i.e., excluding graduate students from all other colleges due to the lack of specific enrollment data)

The goal of probability sampling is to select a sample that is representative of the population from which it is selected

Sampling error: the difference between the "true" result and the "observed" result that can be attributed to using samples rather than populations

  • In a sample of 99 from a population of 100
    • The observed result (i.e., that determined using the sample) is likely to be very, very close to the true result (i.e., that determined using all 100 subjects in the population).
    • Sampling error is minimal.
  • In a sample of 2 from a population of 100
    • The observed result is likely to be somewhat different from the true result.
    • Sampling error is high.

Sampling bias: the difference between the "observed" and "true" results that is attributed to the sampling mistakes of the researcher. Examples of sampling bias are:

  • Deliberately sampling subjects with certain attributes (e.g., positive attitudes, high self-esteem, high level of achievement, etc.)
  • Using subjects from different populations and assigning them to different treatment groups (e.g., males to an experimental treatment group and females to a traditional treatment group)

Types of probability sampling procedures:

  1. Simple random: a number is assigned to each subject in the population and a table of random numbers or a computer is used to select subjects randomly from the population
  2. Systematic sampling: a number is assigned to each subject in the population, and every nth member of the population is selected (e.g., 10, 20, 30, 40, etc,; 12, 22, 32, 42, etc.)
  3. Stratified sampling: similar to random sampling with the exception that subjects are selected randomly from strata, or subgroups, of the population
    • Strata: homogeneous subgroups within a population (e.g., males and females; low, middle and high socio-economic status; certified and non-certified teachers working with special needs students; etc.)
    • Strata should be related to the dependent variable (e.g., socio-economic status is related to achievement and therefore is a potential stratifying variable)
    • Strata ensure adequate numbers of subjects from specific subgroups
      • Proportional stratified sample: the proportions of subjects in each strata in the population are reflected in the proportions of subjects in each strata in the sample (e.g., if the population is 60% female and 40% male, the sample consists of 60% females and 40% males)
      • Disproportional stratified sampling: the proportions of subjects in each strata in the sample are the same regardless of the proportions of subjects in the strata in the population.   Even if the population of elementary school teachers is 90% female and 10% male, 50% of the sample is female and 50% male. This mitigates concerns when only a few subjects would be included in a sample (e.g., only 2 males in a sample of 20 teachers)
  4. Cluster sampling: similar to random sampling except that naturally occurring groups are randomly selected first, then subjects are randomly selected from the sampled groups
    1. Useful when it is impossible to identify all of the individuals in a population
    2. Typical educational clusters are districts, schools, or classrooms
    3. Example - 27 of the 54 school districts were randomly selected, one secondary school in each district was randomly selected, and students randomly selected from each school were tested

Steps in selecting probability samples

  1. Define the target population and sampling frame
  2. Determine the sample size
  3. Select the sampling strategy (i.e., procedure)
  4. Select the sample

Non-probability sampling

Non-probability sampling is a method of sampling in which the probability of selecting a subject is unknown.   It is often not possible to use probability sampling techniques due to access, time, resource or financial constraints.   It is often desirable to select subjects who can be particularly informative about the research issues (e.g., if the researcher is trying to understand how teachers use manipulatives, it makes sense to select teachers who do use these in their classes)

The goal of non-probability sampling is to identify information-rich participants

Three categories of non-probability sampling procedures

  1.   Convenience sampling: selecting a subject or group of subjects based on their availability to the researcher
    • Typical of much educational research given the constraints under which it is conducted
    • The major concern is the limited generalizability of the results from the sample to any population
    • Examples:
      • Students enrolled in the researcher's classes
      • Fourth-grade students in two local, parochial schools to which the researcher has access
  2. Purposive sampling: selection of particularly informative or useful subjects
    • Typically selects a few information-rich subjects who are studied in-depth
    • Also known as purposeful sampling
    • Examples
      • It is reasonable to select "expert" teachers if one is trying to understand how teachers use effective teaching strategies
      • It is reasonable to select physically fit individuals if one is trying to identify effective exercise behaviors
    • Quota sampling: non-random sampling representative of a larger population
      1. Used when the researcher cannot use probability sampling procedures but does want a sample that is somewhat representative of the population
      2. Similar to stratified sampling with the exception that the subjects are selected non-randomly

Types of sampling techniques:

  • Typical case: selecting a representative participant
    • An average student
    • A typical volleyball player
  • Extreme case: selecting a unique or atypical participant
    • A failing student
    • An all-state volleyball player
  • Maximum variation: selecting participants to represent extreme cases
    • Students whose special needs are being met by the schools and those students whose needs are not being met
    • Successful and unsuccessful students in math classes
  • Snowball (i.e., network): selecting participants from recommendations of other participants
    • The recommendations of algebra teachers using math manipulatives of others who are doing the same
    • Teacher's recommendations of student leaders in a school
  • Critical case: selecting the most important participants to understand the phenomena being studied
    • Professors who received laptop computers to help incorporate technology in their teaching
    • Fifth-year athletes who received scholarships to finish their academic work

The use of probability and non-probability sampling

Quantitative studies:

  • The desired use of probability sampling due to the ability to generalize the results to the larger population
  • Frequent use of non-probability techniques - particularly convenience sampling - due to access, time, resource, or financial constraints

Qualitative studies:

  • Almost exclusive reliance on non-probability techniques - particularly purposeful sampling

How subjects and sampling affect results

How do the sampling procedures affect the results?   There are four areas you should consider when you are considering how your sampling can affect the results of your study.  

  1.   Need to identify the sampling procedure used
  2. Sample error and sampling bias
  3. Need to evaluate the sampling procedure in light of the research problems and conclusions
  4. Considering the strengths and weaknesses of specific sampling procedures
  5. How do the characteristics of the subjects affect the usefulness and generalizability of the results?

Volunteer samples

Different characteristics between volunteers and non-volunteers can lead to different responses

  1. Educational level
  2. Socio-economic status
  3. Need for social approval
  4. Ability to socialize
  5. Conformity

Volunteer sample groups are commonly used due to their availability and willingness to participate.

Subject motivation

Specific characteristics of the sample can predispose them to respond in certain ways (e.g., only selecting teachers using holistic language strategies would likely predispose them to respond favorably to an attitudinal scale focusing on holistic language instruction)

Sample size

It is important to note if the sample represent the population?   

  • A sample of 99 of 100 likely represents the population
  • A sample of 1 of 100 is unlikely to represent the population

General rules of thumb when considering sample size

  • Quantitative studies
    1. 30 subjects for correlational research
    2. 15 subjects in each group for experimental research
    3. Approximately 250 responses for survey research
  • Qualitative studies - a sufficient number of subjects are needed to ensure that no new information is forthcoming from additional cases

Remember:   You need to interpret results very carefully - results form studies using very large or very small samples can be misleading.   Both results indicating "no difference" or "no relationship" in studies with small samples and results of 'differences' or 'relationships' in studies can be problematic.

Topic H:   Data Analysis

Analyzing Data Analysis

In quantitative research, the

  • data analysis consists of statistical analysis
  • data analysis involves describing trends, comparing group differences, or relating variables
  • interpretation consists of comparing results with prior predictions and past research

In qualitative research, the

  • data analysis consist of text analysis
  • data analysis involves developing a description and themes
  • interpretation consists of stating the larger meaning of the findings

Steps for Data Analysis and Collection in an Experimental Setting

Step 1.
Organize and prepare the data for analysis.   This involves transcribing interviews, optically scanning material, typing up fieldnotes, or sorting and arranging the data into different types depending on the sources of information.

Step 2.
Read through all that data.   A first general step is to obtain a general sense of the information and to reflect on its overall meaning.   What general ideas are participants saying?   What is the tone of the ideas?   What is the tone of the ideas?   What is the general impression of the overall depth, credibility, and use of the information.   Sometimes qualitative researchers write notes in the margins or start recording   general thoughts about the data at this stage.

Step 3
Begin detailed analysis with a coding process.
(Coding is the process of organizing the material into ‘chunks' before bringing means to those ‘chunks').

Step 4
Use   the coding process to generate a description of the setting or people as well as categories or themes for analysis.

Step 5
Advance how the description and themes will be represented in the qualitative narrative.     The most popular approach is to use a narrative passage to convey the findings of the analysis.

Step 6
A final step in data analysis involves making an interpretation or meaning of the data. 'What were the lessons learned' captures the essence of this idea.

There are two basic methods to show the results of your study: numerical and graphical.

Using the numerical approach one might compute statistics such as the mean and standard deviation. These statistics convey information about the average degree of shyness and the degree to which people differ in shyness.

Using the graphical approach one might create a stem and leaf display and a box plot. These plots contain detailed information about the distribution of shyness scores. Graphical methods are better suited than numerical methods for identifying patterns in the data. Numerical approaches are more precise and objective. Since the numerical and graphical approaches compliment each other, it is wise to use both.

Topic I:   Descriptive Statistics

Types of descriptive statistics

  • Frequency distributions: an organization of the data set indicating the number of times (i.e.) frequency each score was present.   Frequency distributions can be presented as a frequency table, frequency polygon, or as a histogram.   From these visual representations you are able to see the shape of the distributions.   There are two shapes that will be discussed: normal and skewed.
    1. Normal -a set of scores that are equally distributed around the middle score (i.e.) the mean.
    2. Positively skewed -a set of scores characterized by a large number of low scores and a small number of low scores.
  • Central Tendency -what is the typical score
    1. Mode: the most frequently occurring score
    2. Median: the score above and below which one-half of the scores occur
    3. Mean
      1. The arithmetic average of all scores
      2. Statistical properties make it very useful
      3. Concerns related to outlying Scores
    4. Variability -how different are the scores
  • Range: the difference between the highest and lower scores
  • Standard Deviation:
    1. The average distance of the scores from the mean
    2. The relationship to the normal distribution
      1. + 1 SD 68% of all scores in a distributionb.      
      2. + 2 SD 97% of all scores in a distributioN
      3. Use of percentile ranks -the percentage of scores at or below a specified score
  • Relationship -how do two sets of scores relate to one another
    1. Correlation

Central Tendency Measures

Measures of central tendency are measures of the location of the middle or the center of a distribution. The definition of "middle" or "center" is purposely left somewhat vague so that the term "central tendency" can refer to a wide variety of measures. The mean is the most commonly used measure of central tendency.  

Mean

The arithmetic mean is what is commonly called the average: When the word "mean" is used without a modifier, it can be assumed that it refers to the arithmetic mean. The mean is the sum of all the scores divided by the number of scores.  

The formula in summation notation is: μ = ΣX/N where μ is the population mean and N is the number of scores.

(If the scores are from a sample, then the symbol M refers to the mean and N refers to the sample size. The formula for M is the same as the formula for μ.)

The mean is a good measure of central tendency for roughly symmetric distributions but can be misleading in skewed distributions since it can be greatly influenced by extreme scores. Therefore, other statistics such as the median may be more informative for distributions such as reaction time or family income that are frequently skewed.

Median

The median is the middle of a distribution: half the scores are above the median and half are below the median. The median is less sensitive to extreme scores than the mean and this makes it a better measure than the mean for highly skewed distributions. The median income is usually more informative than the mean income, for example:   

The sum of the absolute deviations of each number from the median is lower than is the sum of absolute deviations from any other number.  

The mean, median, and mode are equal in symmetric distributions. The mean is higher than the median in positively skewed distributions and lower than the median in negatively skewed distributions   

Computation of Median:

  • When there is an odd number of numbers, the median is simply the middle number. For example, the median of 2, 4, and 7 is 4.
  • When there is an even number of numbers, the median is the mean of the two middle numbers. Thus, the median of the numbers 2, 4, 7, 12 is (4+7)/2 = 5.5.

Mode

The mode is the most frequently occurring score in a distribution and is used as a measure of central tendency. The advantage of the mode as a measure of central tendency is that its meaning is obvious. It is the only measure of central tendency that can be used with nominal data.

The mode is greatly subject to sample fluctuations and is therefore not recommended to be used as the only measure of central tendency. A further disadvantage of the mode is that many distributions have more than one mode. These distributions are called "multimodal."

In a normal distribution, the mean, median, and mode are identical.

Normal Distribution

Normal distributions are a family of distributions that have the shape shown below.

Normal distributions are symmetric with scores more concentrated in the middle than in the tails. They are defined by two parameters: the mean (μ) and the standard deviation (σ). Many kinds of behavioral data are approximated well by the normal distribution. Many statistical tests assume a normal distribution. Most of these tests work well even if the distribution is only approximately normal and in many cases as long as it does not deviate greatly from normality.

The formula for the height (y) of a normal distribution for a given value of x is:

One reason that the normal distribution is so important is that it is easy for mathematical statisticians to work with. This means that many kinds of statistical tests can be derived for normal distributions. Almost all statistical tests assume normal distributions. Fortunately, these tests work very well even if the distribution is only approximately normally distributed. Some tests work well even with very wide deviations from normality. Finally, if the mean and standard deviation of a normal distribution are known, it is easy to convert back and forth from raw scores to percentiles.

Standard Deviation /Variance

The variance and the standard deviation are measures of how spread out a distribution is. The variance is computed as the average squared deviation of each number from its mean. For example, for the numbers 1, 2, and 3, the mean is 2 and the variance is:

.

The formula (in summation notation) for the variance in a population is



where μ is the mean and N is the number of scores.

When the variance is computed in a sample, the statistic



(where M is the mean of the sample) can be used. S2 is a biased estimate of σ2, however. By far the most common formula for computing variance in a sample is:



which gives an unbiased estimate of σ2. Since samples are usually used to estimate parameters, s2 is the most commonly used measure of variance. Calculating the variance is an important part of many statistical applications and analyses.

The standard deviation formula is very simple: it is the square root of the variance. Standard deviation is the most commonly used measure of spread.

If the mean and standard deviation of a normal distribution are known, it is possible to compute the percentile rank associated with any given score. In a normal distribution, about 68% of the scores are within one standard deviation of the mean and about 95% of the scores are within two standard deviations of the mean.  

Confidence Interval

Before a simple research question such as "What is the mean number of digits that can be remembered?" can be answered, it is necessary to specify the population of people to which it is addressed. The researcher could be interested in, for example, adults over the age of 18, all people regardless of age, or students attending high school. For the present example, assume the researcher is interested in students attending high school. Once the population is specified, the next step is to take a random sample from it. In this example, let's say that a sample of 10 students were drawn and each student's memory tested. The way to estimate the mean of all high school students would be to compute the mean of the 10 students in the sample. Indeed, the sample mean is an unbiased estimate of μ, the population (Damean). But it will certainly not be a perfect estimate. By chance it is bound to be at least either a little bit too high or a little bit too low. For the estimate of μ to be of value, one must have some idea of how precise it is. That is, how close to μ is the estimate likely to be?   An excellent way to specify the precision is to construct a confidence interval. If the number of digits remembered for the 10 students were: 4, 4, 5, 5, 5, 6, 6, 7, 8, 9 then the estimated value of μ would be 5.9 and the 95% confidence interval would range from 4.71 to 7.09.   The wider the interval, the more confident you are that it contains the parameter. The 99% confidence interval is therefore wider than the 95% confidence interval and extends from 4.19 to 7.61.

Topic J:   Statistical Testing

Probability Value (p-Value)

The p-value represents a probability and is a number between zero and one.   The p-value is typically used to determine whether the results of a study are statistically significant.   In other words the p-value is used to determine whether to reject or fail to reject your null hypothesis.   In most scientific literature the p-value that is used is 0.05.   This value means that 95% of the time your results are significantly different from the population and you should reject your null hypothesis.

Significance Testing

Chi Squared Tests:

This statistical method is used to compare observed and hypothesized data.   The purpose of the test is to find out if there is a statistically significant difference between the data that you observe in an experiment and the data that you hypothesize.   In other words you are testing the validity of the null hypothesis.  

  • Null hypothesis: States that there is no difference in the observed and hypothesized (population) data.

The following are the steps in a chi-squared test.  

  1. The first thing you need to do is calculate the chi-squared value.   In a chi-square test numerical values must be used instead of percentages or ratios.   To do this you subtract the expected data from the observed data and square it.   Then you divide this data by the expected data.   This value is your chi-squared score.
    Chi Square Score =      (o-e) ² o = observed
                 e     e = expected
  2. Calculate the degree of freedom for your sample.   The degree of freedom is calculated by subtracting the number of categories by one.
  3. Once you have your degree of freedom and your chi-square score, you are ready to find your p-value to determine whether to reject or fail to reject the null hypothesis.   In order to find the p-value, you must look at the chi-square distribution table.   First, look at the first column on the table and find your degree of freedom.   Then look across the row to determine where your chi-square score fits.   When you find your chi-square score you can then look up the column to find out the p-value.
  4. Now that you have your p-value you can decide whether or not to reject the null hypothesis.   For most cases, if the p-value is 0.05 or below you will reject the null and conclude that there is a statistical significance between the expected and observed scores.

Chi-Square Distribution Table:

Degrees of

Freedom

(df)


Probability (p)

  

0.95

0.90

0.80

0.70

0.50

0.30

0.20

0.10

0.05

0.01

0.001

1

0.004

0.02

0.06

0.15

0.46

1.07

1.64

2.71

3.84

6.64

10.83

2

0.10

0.21

0.45

0.71

1.39

2.41

3.22

4.60

5.99

9.21

13.82

3

0.35

0.58

1.01

1.42

2.37

3.66

4.64

6.25

7.82

11.34

16.27

4

0.71

1.06

1.65

2.20

3.36

4.88

5.99

7.78

9.49

13.28

18.47

5

1.14

1.61

2.34

3.00

4.35

6.06

7.29

9.24

11.07

15.09

20.52

6

1.63

2.20

3.07

3.83

5.35

7.23

8.56

10.64

12.59

16.81

22.46

7

2.17

2.83

3.82

4.67

6.35

8.38

9.80

12.02

14.07

18.48

24.32

8

2.73

3.49

4.59

5.53

7.34

9.52

11.03

13.36

15.51

20.09

26.12

9

3.32

4.17

5.38

6.39

8.34

10.66

12.24

14.68

16.92

21.67

27.88

10

3.94

4.86

6.18

7.27

9.34

11.78

13.44

15.99

18.31

23.21

29.59

  

Nonsignificant

Significant

Source: R.A. Fisher and F. Yates, Statistical Tables for Biological Agricultural and Medical Research, 6th ed., Table IV, Oliver & Boyd, Ltd., Edinburgh, by permission of the authors and publishers

T-tests:

One Sample T-test:   In this type of t-test you are comparing your sample data to a known standard (usually population data).   In order to calculate a t-test you need to know the following data:

  • Sample average
  • Population average
  • Standard Deviation of the sample
  • Number of observation in the sample.

The following are the steps in a one sample t-test:  

  1. The first thing you need to do is calculate the t-score.   In order to do this you subtract the expected by the observed and then divide that number by the standard deviation multiplied by the square root of the number of observations divided by the number of observations minus one.

  

  1. The next thing you need to do is determine the degree of freedom.   You can determine the degree of freedom by subtracting one from the number of sample observations.
  2. Once you have found your t-score and your degree of freedom you are ready to find your p-value to determine whether to reject or fail to reject the null hypothesis.   In order to find the p-value, you must look at the t-test distribution table.  
  3. Now that you have your p-value you can decide whether or not to reject the null hypothesis.   For most cases, if the p-value is 0.05 or below you will reject the null and conclude that there is a statistical significance between the expected and observed scores.

  

T distribution critical values

df

.25

.20

.15

.10

.05

.025

.02

.01

.005

.0025

.001

.0005

1

1.000

1.376

1.963

3.078

6.314

12.71

15.89

31.82

63.66

127.3

318.3

636.6

2

.816

1.061

1.386

1.886

2.920

4.303

4.849

6.965

9.925

14.09

22.33

31.60

3

.765

.978

1.250

1.638

2.353

3.182

3.482

4.541

5.841

7.453

10.21

12.92

4

.741

.941

1.190

1.533

2.132

2.776

2.999

3.747

4.604

5.598

7.173

8.610

5

.727

.920

1.156

1.476

2.015

2.571

2.757

3.365

4.032

4.773

5.893

6.869

6

.718

.906

1.134

1.440

1.943

2.447

2.612

3.143

3.707

4.317

5.208

5.959

7

.711

.896

1.119

1.415

1.895

2.365

2.517

2.998

3.499

4.029

4.785

5.408

8

.706

.889

1.108

1.397

1.860

2.306

2.449

2.896

3.355

3.833

4.501

5.041

9

.703

.883

1.100

1.383

1.833

2.262

2.398

2.821

3.250

3.690

4.297

4.781

10

.700

.879

1.093

1.372

1.812

2.228

2.359

2.764

3.169

3.581

4.144

4.587

11

.697

.876

1.088

1.363

1.796

2.201

2.328

2.718

3.106

3.497

4.025

4.437

12

.695

.873

1.083

1.356

1.782

2.179

2.303

2.681

3.055

3.428

3.930

4.318

13

.694

.870

1.079

1.350

1.771

2.160

2.282

2.650

3.012

3.372

3.852

4.221

14

.692

.868

1.076

1.345

1.761

2.145

2.264

2.624

2.977

3.326

3.787

4.140

15

.691

.866

1.074

1.341

1.753

2.131

2.249

2.602

2.947

3.286

3.733

4.073

16

.690

.865

1.071

1.337

1.746

2.120

2.235

2.583

2.921

3.252

3.686

4.015

17

.689

.863

1.069

1.333

1.740

2.110

2.224

2.567

2.898

3.222

3.646

3.965

18

.688

.862

1.067

1.330

1.734

2.101

2.214

2.552

2.878

3.197

3.611

3.922

19

.688

.861

1.066

1.328

1.729

2.093

2.205

2.539

2.861

3.174

3.579

3.883

20

.687

.860

1.064

1.325

1.725

2.086

2.197

2.528

2.845

3.153

3.552

3.850

21

.663.

.859

1.063

1.323

1.721

2.080

2.189

2.518

2.831

3.135

3.527

3.819

22

.686

.858

1.061

1.321

1.717

2.074

2.183

2.508

2.819

3.119

3.505

3.792

23

.685

.858

1.060

1.319

1.714

2.069

2.177

2.500

2.807

3.104

3.485

3.768

24

.685

.857

1.059

1.318

1.711

2.064

2.172

2.492

2.797

3.091

3.467

3.745

25

.684

.856

1.058

1.316

1.708

2.060

2.167

2.485

2.787

3.078

3.450

3.725

26

.684

.856

1.058

1.315

1.706

2.056

2.162

2.479

2.779

3.067

3.435

3.707

27

.684

.855

1.057

1.314

1.703

2.052

2.15

2.473

2.771

3.057

3.421

3.690

28

.683

.855

1.056

1.313

1.701

2.048

2.154

2.467

2.763

3.047

3.408

3.674

29

.683

.854

1.055

1.311

1.699

2.045

2.150

2.462

2.756

3.038

3.396

3.659

30

.683

.854

1.055

1.310

1.697

2.042

2.147

2.457

2.750

3.030

3.385

3.646

40

.681

.851

1.050

1.303

1.684

2.021

2.123

2.423

2.704

2.971

3.307

3.551

50

.679

.849

1.047

1.295

1.676

2.009

2.109

2.403

2.678

2.937

3.261

3.496

60

.679

.848

1.045

1.296

1.671

2.000

2.099

2.390

2.660

2.915

3.232

3.460

80

.678

.846

1.043

1.292

1.664

1.990

2.088

2.374

2.639

2.887

3.195

3.416

100

.677

.845

1.042

1.290

1.660

1.984

2.081

2.364

2.626

2.871

3.174

3.390

1000

.675

.842

1.037

1.282

1.646

1.962

2.056

2.330

2.581

2.813

3.098

3.300

inf.

.674

.841

1.036

1.282

1.64

1.960

2.054

2.326

2.576

2.807

3.091

3.291

Two Sample T-test:   In this type of t-test you are comparing two random samples of independent variables.   Each sample set must come from a normal distribution.   In order to calculate a t-test you need to know the following data:

  • Two sample averages: x1 and x2
  • Two standard deviations: SD1 and SD2
  • The number of observations in each sample set: n1 and n2

The following are the steps in a two sample t-test:  

  1. The first thing you need to do is calculate the t-score.   In order to do this you subtract the mean of sample two from the mean of sample one.   Then you divide this number by the square root of the standard deviation of sample one squared divided by the number of observations from sample one added to the standard of deviation of sample two squared divided by the number of observations in sample two.

  

  1. The next thing you need to do is determine the degree of freedom.   You can determine the degree of freedom by subtracting two from the sum of the number of observations in sample one and the number of observations in sample two.
  2. Once you have found your t-score and your degree of freedom you are ready to find your p-value to determine whether to reject or fail to reject the null hypothesis.   In order to find the p-value, you must look at the t-test distribution table.  
  3. Now that you have your p-value you can decide whether or not to reject the null hypothesis.   For most cases, if the p-value is 0.05 or below you will reject the null and conclude that there is a statistical significance between the expected and observed scores.

ANOVA:

ANOVA analysis is used to examine the differences in the means of several different groups at once.   All of the should be independent with different means.

  1. Calculate the variance of each group:   (this may be referred to as the Between Mean Squares (BMS))
    1. First calculate the BSS (Between Sum of Squares) using the following formula:
      2. BSS = n1 (x1-x) ² + n2(x2-x)  ² + n3 (x3-x)  ²

      n1 = number of observations in sample one

      n2 = number of observations in sample two

      n3 = number of observations in sample three

      x = average of all sample means

      x1 = mean of sample one

      x2 = mean of sample two

      x3 = mean of sample three
    2. Next find the degrees of freedom by subtracting one from the number of groups.
    3. Lastly, find the Between Mean Squares (BMS) by dividing the BSS by the df
    2.    BSS_
    df
  2. Calculate the Variation within the groups: (this may be referred to as the Within Mean Squares value (WMS))
    1. First calculate the WSS using the following formula:
      WSS = (n1-1) SD1 ² + (n2-1) SD2 ² + (n3-1) SD3 ²

      n1 = number of observations in sample one

      n2 = number of observations in sample two

      n3 = number of observations in sample three

      SD1 = standard deviation in sample one

      SD2 = standard deviation in sample two

      SD3 = standard deviation in sample three
    2. Calculate the degree of freedom (DFw) by subtracting the number of groups from the number of cases in the total sample.
    3. Lastly, find the Within Mean Squares (WMS) by dividing the WSS by the DFw)
  3. Calculate the F-Value by dividing the Between Mean Squares (from step one) by the Within Mean Squares (from step two).
    4. F value =       Between Mean Squares_
                                     Within Mean Square
  4. Use the F-Value on the F-chart to determine the p-value.
  5. Now that you have your p-value you can decide whether or not to reject the null hypothesis.   For most cases, if the p-value is 0.05 or below you will reject the null and conclude that there is a statistical significance between the expected and observed scores.