Introduction and Class Overview

• Welcome
• Homepage: http://personal.stevens.edu/~ysakamot/730A/
• Syllabus, schedule, and announcement
• What "MGT 730A: Design and Analysis of Experiments" is about
• Goals of course
• How to make sense of numbers and test ideas
  You can be a sophisticated consumer of data
  or you can be everyone else
• How to obtain meaningful data
  Plan your research - select which method to use
• Scope
• Research methods
• Scientific methods (controlled experiment, quasi-experiment or natural experiment, naturalistic observation, etc.)
• Business research methods (survey, questionnaires, interview, case study, focus group, etc.)
• Statistics
• Descriptive (mean, standard error, etc.)
• Inferential (t-test, ANOVA, etc.)
• Exploratory (data reduction - factor analysis, etc.)

• Design of experiments
• Scientific methods
• Operational definitions and variables
• Validity, precision, reliability, and accuracy
• Sampling, randomization, and counterbalancing
• Between, within, matched and factorial designs
• Business research methods
• Research planning
• Analysis of experiments
• Hypothesis testing and confidence intervals
• z, t, ANOVA, and Chi-square tests
• Correlation and regression
• Factor analysis

• Demo on detecting patterns and randomness: http://faculty.vassar.edu/~lowry/wolfgang.html
• Regression to the mean
• Example 1: A person X scored 780 out of 800 on SAT. X takes the SAT again. Assuming that there was no learning or practice effect, what score would you expect X to get on the second test?
  • Below 780, between 780 and the mean.
  • People's scores fluctuate just by luck.
  • Regression to the mean occurs any time people are chosen in such a way that their scores are systematically different from their "true" scores.
• Example 2: You flip a coin 10 times and ask your class to predict the outcome of each flip. Student X was correct 8 out of 10 times. You test X for a second time. How many times X will predict correctly this time?
• The best prediction is 5 correct guesses.
• This test is all luck.
• The score of 8 will regress all the way to the mean of 5.
• Regression to the mean is the tendency for scores to average out. Extreme scores tend to happen rarely and fall back toward the mean.
• Francis Galton (1822-1911), cousin of Charles Darwin
• Tall parents had tall children but not so tall on average and short parents had short children but not so short on average - regression to the mean.
• "Natural Inheritance" (1889) - statistical methods to verify theories (line of regression)
• Regression to the mean and data gathering
• Suppose you choose fifth-grade children who scored the worst on a reading test, administer a drug that may improve their reading ability, and retest them on the reading test.
• The lowest-scoring children are likely to have scored lower than their "true" scores and thus improve on a retest.
• You cannot conclude that the drug improved their reading skills because the improvement can also be due to regression to the mean.
• Other Examples?
• Sports
• Financial markets
• Some more examples: http://www.skepdic.com/regressive.html

• Scientific method
• Develop a theory based on observations (empirical)
• Form a hypothesis based on the predictions of the theories (predictive)
• Test the hypothesis by experiments (testable)
• If the hypothesis is supported, the theory survives
• If the hypothesis fails, the theory fails (falsifiable)
• Modify the old theory or develop a new theory (tentative)
• Test the theory again (rigorously evaluated)
• Experiment
• A test under controlled conditions
• Examine the validity of the hypothesis
• Statistical analysis
• Stupid questions get stupid answers

• Kuhn paradigm (1962)
• What you will accept as evidence depends on theory.
• Different people look at different things.
• Who is a better baseball player?
• Stats freak: Look at batting average and r.b.i.
• Owner: Look at salary and gate attendance
• These two are not speaking the same language.
• A paradox of empiricism
• We make observations in our studies.
• Observations are used to build theories.
• What counts as an observation depends on theory.
• A study of human communication - videotape a conversation
• What should we code?
• How often people hesitate in speech?
• Use of space filling (er, um)?
• Sneezes?
• How often they gesture?
• What kind of gestures they make?
• What if our theory is wrong?
• You might miss key pieces of information.
• Observation poses a particular problem when dealing with people.
• Awareness of the experimental situation - demand characteristics.
• Most people in a study want to be cooperative.
• People will try to figure out what you want them to do and then they will do it.
• Cues other than the ones central to the research may guide people to the "right" behavior.
• Could be aspects of the instructions or the tasks.
• Results are not interesting if you tell them what to do.
• Study of problem solving - people might use tools just because they were given.
• Issues to consider
• Minimize interference - Demand characteristics, reactivity.
• Maximize precision - Precision and validity.
• Maximize objectivity - Expectancy and reliability.