Business Research Methods 1

• Research plans depend on what information you need and what information is available.
• Customer complaints, improve products, improve process, convince funders.
• Breadth and depth of information you get.
• More effective and efficient research by having more focused goals.

• What do you want to be able to decide as a result of the research?
• Who are the audiences for the information from the research?
• What kinds of information are needed to make the decisions - do you need information to understand a process, the customers who buy certain products, how the product or service or program failed some customers and why, etc.?
• From what sources should the information be collected - employees, customers, certain documentation, etc.?
• How can that information be collected in a reasonable fashion - questionnaires, interviews, examining documentation, observing staff and clients in the program, conducting focus groups among staff and clients, etc.?
• What resources are available to collect the information?
• When is the information needed?

• Will the methods get all of the needed information?
• What additional methods could be used if additional information is needed?
• A questionnaire to quickly collect a great deal of information from a lot of people.
• Interviews to get more in-depth information from certain respondents to the questionnaires.
• Case studies could then be used for more in-depth analysis of unique and notable cases - those who benefited or not from the program, those who quit the program, etc.
• How accurate will the information be?
• Will the information appear as credible to decision makers?
• Who can administer the methods?
• How can the information be analyzed?

• Reactions and feelings - feelings are often poor indicators that your service made lasting impact
• Learning - enhanced attitudes, perceptions or knowledge
• Changes in skills - applied the learning to enhance behaviors
• Effectiveness - improved performance because of enhanced behaviors

• are written summaries of real-life business situations based upon data and research.
• provide a picture of what has happened to a company over a period of time.
• include events such as organizational change and strategy decisions.
• consider factors inside and outside the organization.

• Gather data - if the study is to highlight a program's failure with a client, data would be collected about the program, its processes, and the client.
• Documents (applications, histories, records, etc.).
• Questionnaires.
• Interviews.
• Naturalistic observation.
• Organize data to match the focus of the study - a chronological order to portray how the client started the program, went through the program, and did not receive effective services.
• Develop a case study narrative - a story that summarizes key information and is understandable to an outside reader, e.g., key demographic information about the client, phases in the program's process through which the client passed, any major differences noticed about that client during the process, early indicators of failures, key quotes from the client, etc..
• Validate case study by review from the participant.
• Compare case studies to isolate underlying patterns - commonalities in clients' experiences and how they went through the program will highlight where in the program the process needs to be strengthened.

• group interviews.
• can be used to evaluate services or test new ideas.

• Identify the major objective of the meeting.
• Carefully develop main questions.
• Session should last one to 1.5 hours - typically, five or six questions.
• Ask yourself what problem or need will be addressed by the information gathered during the session - e.g., examine if a new service or idea will work, further understand how a program is failing, etc..
• Focus groups are basically multiple interviews - many of the same guidelines for conducting focus groups are similar to conducting interviews (coming soon).
• Plan your session.
• Scheduling: Plan meetings to be 1 to 1.5 hours long - Over lunch seems to be a very good time for others to find time to attend.
• Setting and Refreshments: Hold sessions in a conference room, configure chairs so that all members can see each other, provide name tags for members, provide refreshments, especially box lunches if the session is held over lunch.
• Ground Rules: It's critical that all members participate as much as possible, yet the session move along while generating useful information - Because the session is often a one-time occurrence, it's useful to have a few, short ground rules that sustain participation, yet do so with focus - a) keep focused, b) maintain momentum and c) get closure on questions.
• Agenda: Consider the following agenda - welcome, review of agenda, review of goal of the meeting, review of ground rules, introductions, questions and answers, wrap up.
• Membership: Focus groups are usually conducted with 6-10 members who have some similar nature, e.g., similar age group, status in a program, etc. - Select members who are likely to be participative and reflective. and attempt to select members who don't know each other.
• Plan to record the session with either an audio or audio-video recorder: If this isn't practical, bring someone to take notes - Don't count on your memory.
• Invite potential members to the meeting: Let them inform about the proposed agenda and the questions the group will discuss.

• Major goal of facilitation is collecting useful information to meet goal of meeting.
• Introduce yourself and the co-facilitator, if used.
• Explain the means to record the session.
• Carry out the agenda.
• Carefully word each question before that question is addressed by the group. Allow the group a few minutes for each member to carefully record their answers. Then, facilitate discussion around the answers to each question, one at a time.
• After each question is answered, carefully reflect back a summary of what you heard (the note taker may do this).
• Ensure even participation. If one or two people are dominating the meeting, then call on others. Consider using a round- table approach, including going in one direction around the table, giving each person a minute to answer the question. If the domination persists, note it to the group and ask for ideas about how the participation can be increased.
• Closing the session - Tell members that they will receive a copy of the report generated from their answers, thank them for coming, and adjourn the meeting.

• Useful for getting the story behind a participant's experiences.
• In-depth information around a topic.
• Follow-up to certain respondents to questionnaires, e.g., to further investigate their responses.
• Usually open-ended questions are asked during interviews.
• Keep clear focus on the intent of each question - like everything else, clearly articulate to yourself what problem or need is to be addressed using the information to be gathered by the interviews before you start to design your interview questions and process.

• Choose a setting with little distraction.
• Avoid loud lights or noises, ensure the interviewee is comfortable (you might ask them if they are).
• Often, they feel more comfortable at their own places of work or homes.
• Explain the purpose of the interview.
• Address terms of confidentiality.
• Explain who will get access to their answers and how their answers will be analyzed.
• If their comments are to be used as quotes, get their written permission to do so.
• Explain the format of the interview.
• Explain the type of interview you are conducting and its nature.
• If you want them to ask questions, specify if they're to do so as they have them or wait until the end of the interview.
• Indicate how long the interview usually takes.
• Tell them how to get in touch with you later if they want to.
• Ask them if they have any questions before you both get started with the interview.
• Ask for permission to record the interview or bring along someone to take notes - don't count on your memory to recall their answers.

• Informal, conversational interview - no predetermined questions are asked, in order to remain as open and adaptable as possible to the interviewee's nature and priorities; during the interview, the interviewer goes with the flow.
• General interview guide approach - ensure that the same general areas of information are collected from each interviewee, which provides more focus than the conversational approach, but still allows a degree of freedom and adaptability in getting information from the interviewee.
• Standardized, open-ended interview - the same open-ended questions are asked to all interviewees, which results in faster interviews that can be more easily analyzed and compared.
• Closed, fixed-response interview - all interviewees are asked the same questions and asked to choose answers from among the same set of alternatives, which is useful for those not practiced in interviewing.

• Behaviors - about what a person has done or is doing.
• Opinions/values - about what a person thinks about a topic.
• Feelings - about what a person feels about a topic.
• Knowledge - about what facts a person knows about a topic.
• Sensory - about what a person has seen, touched, heard, tasted, or smelled.
• Background/demographics - standard background questions, such as age, education, etc..

• Get the respondents involved in the interview as soon as possible.
• Before asking about controversial matters (such as feelings and conclusions), first ask about some facts so that respondents can more easily engage in the interview before warming up to more personal matters.
• Intersperse fact-based questions throughout the interview to avoid long lists of fact-based questions, which tends to leave respondents disengaged.
• Ask questions about the present before questions about the past or future - usually easier for them to talk about the present and then work into the past or future.
• The last questions might be to allow respondents to provide any other information they prefer to add and their impressions of the interview.

• Questions should be as neutral as possible. Avoid wording that might influence answers, e.g., evocative, judgmental wording.
• Questions should be asked one at a time.
• Questions should be worded clearly - know any terms particular to the program or the respondents' culture.
• Be careful asking "why" questions - "why" infers a cause-effect relationship that may not truly exist, and justifying their response may inhibit their responses to this and future questions.

• Occasionally verify the tape recorder (if used) is working.
• Ask one question at a time.
• Attempt to remain as neutral as possible - don't show strong emotional reactions to their responses.
• Encourage responses with occasional nods of the head, etc..
• Be careful when note taking - if you jump to take a note, it may appear as if you're surprised or very pleased about an answer, which may influence answers to future questions.
• Provide transition between major topics - we've been talking about (some topic) and now I'd like to move on to (another topic).
• Don't lose control of the interview - can occur when respondents stray to another topic, take so long to answer a question that times begins to run out, or begin asking questions to the interviewer.
• Observe - does the respondent look nervous, surprised, etc.?

• Questionnaires demand careful attention but are an inexpensive way to gather data from a potentially large number of respondents.
• Questionnaires are like any scientific experiment in that you form a hypothesis and see if the obtained data support or disprove the hypothesis - typically you don't collect data and then see if you found something interesting.
• A questionnaire is only as good as the questions it contains - you need to make the questionnaire understandable and free of bias.
• A questionnaire is a multi-stage process beginning with definition of the aspects to be examined and ending with interpretation of the results.
• Defining the objectives of the survey.
• Determining the sample.
• Writing the questionnaire.
• Administering the questionnaire.
• Interpreting the results.
• Questionnaires are flexible in what they can measure, but the researchers have limited control over the environment.
• This is why questionnaires are inexpensive to administer.
• Loss of control means the validity of the results are more reliant on the honesty of the respondent.
• More difficult to claim objectivity with questionnaire data than with results of a tightly controlled lab test.
• If a group of participants are asked on a questionnaire how long it took them to learn a particular function on a piece of software, it is likely that they will be biased towards themselves and answer, on average, with a lower than actual time.
• A more objective usability test of the same function with a similar group of participants may return a significantly higher learning time.
• In general, questionnaires are better suited to gathering reliable subjective measures, such as user satisfaction, of the system or interface in question.
• Questions may be designed to gather either qualitative or quantitative data.
• Quantitative questions are more exact then qualitative.
• Qualitative measures, such as easy and difficult, can mean radically different things to different people.
• Questions that assess a qualitative measure must be phrased to avoid ambiguity.
• Qualitative questions may also require more thought on the part of the participant and may cause them to become bored with the questionnaire sooner.
• Qualitative questions require more care in design, administration, and interpretation.

• Key preparation
• Again, like everything else, clearly articulate what problem or need is to be addressed using the information to be gathered by the questions.
• Review why you're doing the evaluation and what you hope to accomplish by it - focus on what information you need and, ultimately, on what questions should be used.
• Directions to respondents
• Include a brief explanation of the purpose of the questionnaire.
• Include clear explanation of how to complete the questionnaire.
• Include directions about where to provide the completed questionnaire.
• Note conditions of confidentiality, e.g., who will have access to the information - you will make every reasonable attempt to protect access to their answers.
• Content of questions
• Ask about what you need to know - get information in regard to the goals or what you want to address by the evaluation.
• Will the respondent be able to answer your question - do they know the answer?
• Will respondents want to answer the question - is it too private or silly?
• Wording of questions
• Will the respondent understand the wording - are you using any slang, cultural-specific or technical words?
• Are any words so strong that they might influence the respondent to answer a certain way? - avoid use of strong adjectives, e.g., highly effective, etc..
• To ensure you're asking one question at a time, avoid use of the word "and" in your question.
• Avoid using "not" in your questions if you're having respondents answer "yes" or "no" to a question - use of "not" can lead to double negatives and cause confusion.
• If you use multiple choice questions, be sure your choices are mutually exclusive and encompass the total range of answers.
• Respondents should not be confused about whether two or more alternatives appear to mean the same thing.
• Respondents also should not have a clearly preferred answer that is not among the alternative choices of an answer to the question.
• Developing questions
• Attempt to get respondents' motivation to complete the questionnaire.
• Start with fact-based questions and then go on to opinion-based questions, e.g., ask people for demographic information about themselves and then go on to questions about their opinions and perspectives.
• This gets respondents engaged in the questionnaire and warmed up before more challenging and reflective questions about their opinions.
• (Consider if they can complete the questionnaire anonymously - if so, indicate this on the form where you ask for their name.)
• (Be careful not to include so many questions that potential respondents are dissuaded from responding.)
• Attempt to get respondents' commentary in addition to their ratings - if the questionnaire asks respondents to choose an answer by circling an answer or provide a rating, ask them to provide commentary that explains their choices.
• Include a question to get respondents' impressions of the questionnaire itself - ask them if the questionnaire was straightforward to complete ("yes" or "no), and if not, to provide suggestions about how to improve the questionnaire.
• Pilot or test your questionnaire on a small group of clients or fellow staff.
• Ask them if the form and questions seemed straightforward. Carefully review the answers on the questionnaires.
• Does the information answer the evaluation questions or provide what you want to know about the program or its specific services?
• What else would you like to know?
• Finalize the questionnaire - use the results of the pilot.

• When resources and money are limited
• A questionnaire can be quite inexpensive to administer.
• Potentially several thousand people could respond in a few days. It would be impossible to get a similar number of usability tests completed in the same short time.
• When it is necessary to protect the privacy of the participants.
• Questionnaires are easy to administer confidentially.
• Minimize reactivity - ensure participants will respond honestly (especially to embarrassing questions about private or personal behavior).
• When corroborating other findings.
• Questionnaires can be a useful confirmation tools.
• There will not be resources to run more costly tests on large enough participants to make the results statistically significant - a follow-up large scale questionnaire may be necessary to corroborate these earlier results.

• The importance of well-defined objectives can not be over emphasized.
• A questionnaire that is written without a clear goal and purpose is going to overlook important issues and waste participants' time by asking useless questions.
• A questionnaire that is written without a clear goal and purpose may lack a logical flow and thereby cause the participant to lose interest.
• How would it be possible to reach insightful conclusions if one didn't actually know what they had been looking for or planning to observe?
• The questionnaire designer must clearly define the measures.
• An objective "to identify points of user dissatisfaction with the interface and how these negatively affect the software's performance" may sound clear but it is not.
• what is meant by user dissatisfaction? Is this dissatisfaction with the learning of the software, the power of the software, of the ease of learning the software?
• What is meant by the software's performance? How accurate must the measurements be?
• All of these issues must be carefully considered when formulating questions.
• A good rule of thumb: if you are finding it difficult to write the questions, then you haven't spent enough time defining the objectives of the questionnaire - the questions should follow quite naturally from the objectives.

• Once we have formulated the objectives of the investigation, decided what kind of data we are to measure, and decided on a participant group, we must compose our questions.
• Most questionnaires gather demographic data on the participants.
• Typically, demographic data is collected at the beginning of the questionnaire, but such questions could be located anywhere or even scattered throughout the questionnaire.
• Background questions are easier to answer and can ease the respondent into the questionnaire - you do not want to put off the participant by jumping in to the most difficult questions.
• It is important to ask only those background questions that are necessary - e.g., do not ask income of the respondent unless there is at least some rational for suspecting a variance across income levels.
• Open format or closed format questions?
• Open format questions are those that ask for unprompted opinions - there are no predetermined set of responses, and the participants are free to answer however they choose.
• Open format questions are good for soliciting subjective data or when the range of responses is not tightly defined.
• An advantage is that the variety of responses will be wider and more truly reflect the opinions of the respondents, which increases the likelihood of obtaining unexpected and insightful suggestions (as it is impossible to predict the full range of opinion).
• It is common for a questionnaire to end with and open format question asking the respondent for ideas for changes or improvements.
• One disadvantage is that there is no way to automatically perform statistical analysis on open format questions.
• A related disadvantage is that analyzing the open format questions is open to the influence of the reader, as no two people will interpret an answer in precisely the same way - inter-rater reliability.
• Another disadvantage is that open format questions require more thought and time on the part of the respondent, which can increase the chance of tiring or boring the respondent.
• Closed format questions usually take the form of a multiple-choice question - they are easy for the respondent.
• There needs to be sufficient choices to fully cover the range of answers but not so many that the distinction between them becomes blurred - usually this translates into five to ten possible answers per questions.
• Odd number of choices - allows for a neutral or no opinion response but this option might be over used (especially by bored respondents).
• Even number of choices - forces the respondent to get off the fence but may induce some inaccuracies when the respondent actually has no opinion.
• For larger questionnaires that test opinions on a very large number of items, it may be best to use an even number of choices to prevent large numbers of no-thought neutral answers.
• Closed format questions offer many advantages in time and money - easy to calculate statistics, easy to filter out useless or extreme answers.
• One disadvantage is that the respondents may not find the choice that reflect their opinions.

• Clarity: The goal is to eliminate the chance that the question will mean different things to different people - If the designers fails to do this, then participants will be answering different questions.
• Not clear, open to interpretation:
• Very Often
• Often
• Sometimes
• Rarely
• Never
• Clearer when you quantify the choices:
• Every Day or More
• 2-6 Times a Week
• About Once a Week
• About Once a Month
• Never
• Leading Questions: It is easy to make this mistake not in the question, but in the choice of answers - A closed format question must supply answers that not only cover the whole range of responses, but that are also equally distributed throughout the range and are equally likely.
• The result will be biased toward positive:
• Superb
• Excellent
• Great
• Good
• Fair
• Not so Great
• Another example: Is this the best interface you have ever used? [yes/no]
• Even if the participant loved the interface, but had a favorite that was preferred, the answer will be no - the negative response covers too wide a range of opinions.
• Phrasing: Most adjectives, verbs, and nouns in English have either a positive or negative connotation.
• "best" in "Is this the best interface you have every used?" has strong overtones that deny the participant an objective environment to consider the interface - the designers think it is the best interface, and so should everyone else.
• This kind of questioning is common practice.
• Another example: Consider these two questions.
• Do you agree with the Governor's plan to oppose increased development of wetlands?
• Do you agree with the Governor's plan to support curtailed development of wetlands?
• Both ask the same thing, but will likely produce different data - one asks in a positive way, and the other in a negative.
• It is impossible to predict how the outcomes will vary, so one method to counter this is to be aware of different ways to word questions and provide a mix in your questionnaire.
• If the participant pool is very large, several versions may be prepared and distributed to cancel out these effects.
• Embarrassing Questions: Avoid personal or private matters.
• If you make them feel uncomfortable, you will lose their trust.
• Your data are only as good as the trust and care that your respondents give you.
• Hypothetical Questions: Avoid questions in which the respondents have to give thought to something they have never considered.
• If you were governor, what would you do to stop crime?
• This type of question will not produce clear and consistent data representing real opinion.
• Prestige Bias: tendency for respondents to answer in a way that make them feel better.
• People may not lie directly, but may try to put a better light on themselves.
• it is not uncommon for people to respond to a political opinion poll by saying they support Samaritan social programs, such as food stamps, but then go on to vote for candidates who oppose those very programs.
• People tend to say they are faster learners than they are.
• Not much you can do - just make the questionnaire as private as possible: The farther away the critical eye of the researcher is, the more honest the answers (telephone interviews are better than person-to-person interviews, and written questionnaires mailed to participants are even better still).

• Quality testing.
• Clear and understandable to all - the need for accuracy.
• You must observe a few test questionnaire takers.
• Review the questionnaire with the test takers and discuss all points that were in any way confusing and work together to solve the problems.
• Revise the questionnaire