A. PPA 696/697   STEPS IN EMPIRICAL RESEARCH

The ideal research proposal should be comprehensive enough to enable the reader to know everything that could be expected to happen if the project were actually carried out--including anticipated obstacles as well as anticipated benefits. In order to design a research project, you may wish to ask yourself the following series of questions:

1. PROBLEM STATEMENT, PURPOSES, BENEFITS What exactly do I want to find out? What is a researchable problem? What are the obstacles in terms of knowledge, data availability, time, or resources? Do the benefits outweigh the costs?

2. THEORY, ASSUMPTIONS, BACKGROUND LITERATURE What does the relevant literature in the field indicate about this problem? To which theory or conceptual framework can I link it? What are the criticisms of this approach, or how does it constrain the research process? What do I know for certain about this area? What is the history of this problem that others need to know?

3. VARIABLES AND HYPOTHESES What will I take as given in the environment? Which are the independent and which are the dependent variables? Are there control variables? Is the hypothesis specific enough to be researchable yet still meaningful? How certain am I of the relationship(s) between variables?

4. OPERATIONAL DEFINITIONS AND MEASUREMENT What is the level of aggregation? What is the unit of measurement? How will the research variables be measured? What degree of error in the findings is tolerable? Will other people agree with my choice of measurement operations?

5. RESEARCH DESIGN AND METHODOLOGY What is my overall strategy for doing this research? Will this design permit me to answer the research question? What other possible causes of the relationship between the variables will be controlled for by this design? What are the threats to internal and external validity?

6. SAMPLING How will I choose my sample of persons or events? Am I interested in representativeness? If so, of whom or what, and with what degree of accuracy or level of confidence?

7. INSTRUMENTATION How will I get the data I need to test my hypothesis? What tools or devices will I use to make or record observations? Are valid and reliable instruments available, or must I construct my own?

8. DATA COLLECTION AND ETHICAL CONSIDERATIONS Are there multiple groups, time periods, instruments, or situations that will need to be coordinated as steps in the data collection process? Will interviewers, observers, or analysts need to be trained? What level of inter-rater reliability will I accept? Do multiple translations pose a potential problem? Can the data be collected and subjects' rights still preserved?

9. DATA ANALYSIS What combinations of analytical and statistical process will be applied to the data? Which will allow me to accept or reject my hypotheses? Do the finding show numerical differences, and are those differences important?

10. CONCLUSIONS, INTERPRETATIONS, RECOMMENDATIONS Was my initial hypothesis supported? What if my findings are negative? What are the implications of my findings for the theory base, for the background assumptions, or relevant literature? What recommendations can I make for public policies or programs in this area? What suggestions can I make for further research on this topic?

Source: http://www.csulb.edu/~msaintg/ppa696/696steps.htm#Steps

OUTLINE FOR A RESEARCH PROSPECTUS

1. Title Page
2. Table of Contents
3. Executive Summary or Summary Abstract
4. Statement of the Problem
5. Background and Literature Review
6. Research hypothesis and Null hypothesis
7. Definition of research variables and operationalization
8. Research design and strategy
9. Threats to internal and external validity
10. Sampling
11. Data collection and ethics
12. Data analysis and statistical tests
13. Conclusions
14. Recommendations
15. References
16. Annotated Bibliography

See  http://www.csulb.edu/~msaintg/ppa696/696menu.htm

Computers in Biblical Studies http://www.balboa-software.com/semcomp/semcomp.htm#OUTLINE

B. Document Research

Existing records often provide insights into a setting and/or group of people that cannot be observed or noted in another way. This information can be found in document form. Lincoln and Guba (1985) defined a document as "any written or recorded material" not prepared for the purposes of the evaluation or at the request of the inquirer. Documents can be divided into two major categories: public records, and personal documents (Guba and Lincoln, 1981).

Public records are materials created and kept for the purpose of "attesting to an event or providing an accounting" (Lincoln and Guba, 1985). Public records can be collected from outside (external) or within (internal) the setting in which the evaluation is taking place. Examples of external records are census and vital statistics reports, county office records, newspaper archives, and local business records that can assist an evaluator in gathering information about the larger community and relevant trends. Such materials can be helpful in better understanding the project participants and making comparisons between groups/communities.

For the evaluation of educational innovations, internal records include documents such as student transcripts and records, historical accounts, institutional mission statements, annual reports, budgets, grade and standardized test reports, minutes of meetings, internal memoranda, policy manuals, institutional histories, college/university catalogs, faculty and student handbooks, official correspondence, demographic material, mass media reports and presentations, and descriptions of program development and evaluation. They are particularly useful in describing institutional characteristics, such as backgrounds and academic performance of students, and in identifying institutional strengths and weaknesses. They can help the evaluator understand the institution’s resources, values, processes, priorities, and concerns. Furthermore, they provide a record or history not subject to recall bias.

Personal documents are first-person accounts of events and experiences. These "documents of life" include diaries, portfolios, photographs, artwork, schedules, scrapbooks, poetry, letters to the paper, etc. Personal documents can help the evaluator understand how the participant sees the world and what she or he wants to communicate to an audience. And unlike other sources of qualitative data, collecting data from documents is relatively invisible to, and requires minimal cooperation from, persons within the setting being studied (Fetterman, 1989).

The usefulness of existing sources varies depending on whether they are accessible and accurate. In the hypothetical project, documents can provide the evaluator with useful information about the culture of the institution and participants involved in the project, which in turn can assist in the development of evaluation questions. Information from documents also can be used to generate interview questions or to identify events to be observed. Furthermore, existing records can be useful for making comparisons (e.g., comparing project participants to project applicants, project proposal to implementation records, or documentation of institutional policies and program descriptions prior to and following implementation of project interventions and activities).

The advantages and disadvantages of document studies are outlined in Exhibit 1.

Source:  http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/CHAP_3.HTM  

1. Research Questions:

1.1 Write a research question that consists of these three parts:
a. the type of research (What is the relationship, difference, effect; what characteristics)                                             b. the main variables (faith, salvation, or any other key-words)
c. the source or population (I Peter 1:8,9, or other Bible texts, manuscripts, etc.)

1.2 Define the key-words. Use online dictionaries.

1.3 Divide the pertinent texts into short thought-sections (like a poem).

(What is/should be? Is it an understanding/procedure/people/materials/old/new/skills problem? What results do you expect? How to use/apply it? How much time/money/people/change?)

Example Questions: 1.4 What is the / relationship between / faith and salvation / in I Peter 1:8,9 ?

2. Literature Review:

2.1 Study and make notes on the biblical text, context, and concept. Collect all the online literature that you will use in the review. For each source list the title and URL (web-address). Search the above links. Makes notes on the bias or lack of validity or reliability of any of the sources. Studies on I Peter and Stedman

2.2 Study and make notes on the historical and geographic context of the text or concept.

2.3 Study and make notes on the related commentaries, studies, sources and sermons.

Example Questions: 2.4 What does the literature say about I Peter 1:8,9 and faith / about salvation?

3. Data Analysis:

3.1 List your findings from the literature search in a series of tables. Identify the main concepts, place them on a separate page, and circle each individually.

3.2 Connect the concepts and thus create a model of relationships, differences, etc.

Example Questions: 3.3 What model can express the relationship between faith / and salvation?

4. Conclusion:

4.1 Summarize the above sections, with special emphasizes to section 3.2

4.2 Answer the research question presented in 1.1 in the conclusion.

Example Questions: 4.3 What is the conclusion? Explore the PeterPlan to Find the Trustworthy.

C. Using Qualitative Research Methods

One important methodological option in conducting management research is the use of qualitative methods for data collection and analysis. Qualitative research, with its emphasis on understanding complex, interrelated and/or changing phenomena, is particularly relevant to the challenges of conducting management research. Qualitative methods combined with quantitative ones can provide particularly rich and robust inquiries. Either alone or in combination, qualitative research must be conducted with methodological rigor.

This section does not attempt to provide a primer on qualitative methods. The role, benefits and appropriate use of qualitative research have been discussed extensively in the literature. Several references to excellent articles can be found in the references section and links to references.

Our more limited aims here are:

• To offer, for those who are not familiar with qualitative methods, a brief overview of how they are used and what value they offer, drawing heavily from articles by Shoshanna Sofaer. ⁶
  
  
• To propose the use of qualitative methods in hypothesis testing. Qualitative methods are often used inductively, for exploration, theory building and description. Less attention has been given to their use in deductive hypothesis testing. The white paper, prepared by Brian Mittman ⁷ discussed in this section explores those potential uses.

Specially this section addresses four questions:

• What are the uses and value of qualitative research?
• What are the methodological challenges in qualitative research?
• What are some key qualitative research methods?
• What is the role of qualitative research in hypothesis testing?

What are the uses and value of qualitative research?

Qualitative research is characterized by an emphasis on describing, understanding, and explaining complex phenomena - on studying, for example, the relationships, patterns and configurations among factors; or the context in which activities occur. The focus is on understanding the full multi-dimensional, dynamic picture of the subject of study.

Its approaches contrast with quantitative methods that aim to divide phenomena into manageable, clearly defined pieces, or variables. Quantification is good for separating phenomena into distinct and workable elements of a well-defined conceptual framework. But when we focus research on what we already know how to quantify, (e.g., what can be reliably quantified), we may miss factors that are key to a real understanding of the phenomena being studied. The downside of quantification is that it does not always support (as well as qualitative methods) understanding of complex, dynamic, and multi-dimensional wholes.

Qualitative methods are useful, not only in providing rich descriptions of complex phenomena, but in constructing or developing theories or conceptual frameworks, and in generating hypotheses to explain those phenomena.

What are the methodological challenges in qualitative research techniques?

Key challenges to conducting rigorous qualitative research range from instrument development through data collection to data analysis. In addition, results need to be documented and reported using formal accepted methods.

For example, typical deficiencies are unfocused instrument development and lack of supporting theory. Rigor related to instrument protocol development requires attention to validity, intrusiveness (the Hawthorne effect) and triangulation. In addition, attention must be paid to distinguishing between collecting subjective and objective data, information on the formal vs. the informal organizational structures and processes and the differences between collecting facts vs. opinions vs. interpretations.

Planned, systematic, comprehensive data collection requires variable definitions and measures, document coding form protocols, administrative database specifications and survey instrument question libraries. In the data collection phase, problems can be minimized through pilot-testing and pretesting, validity/quality checks, triangulation and monitored flexibility. Sole reliance on subjective data, self-reports, etc. can reduce validity. Some tips to insure rigor in data collection management include training of all data collection staff and conducting immediate post-collection coding for time/memory sensitive data. Other methods to ensure the validity of data include tape recording interviews, performing real time data entry and editing, using paired interviewers, and implementing quality assurance fo each instrument. And, to avoid further problems, incomplete, missing or unusable data should be corrected immediately.

Pitfalls related to data analysis include using ad hoc, emergent, exploratory, informal analyses that may lead to inappropriate conclusions and unpublishable results. Rigorous analysis requires an a priori theoretical model and hypothesis, a formal framework guiding data collection and analysis and adherence to the formal framework and research best practices.

Finally, reporting requires results structured by hypotheses and an analysis plan. Reports need to include data syntheses and summaries with a focused analysis of the data. Conclusions must have a documented basis and systematic formal analysis methods, and validity must be documented.

What are some key qualitative research methods?

A wide range of tested qualitative research methods are available to address these challenges. The selection of method, or combination of methods, will be tailored to the questions being studied and the setting for research. Typical methods include:

• Naturalistic inquiry and participant observation
• Case study research
• Structured observations of meetings and events
• Content analysis of documents
• Collection and analysis of other archival, administrative and performance data
• Focus groups
• Cognitive interviews
• Mail and telephone surveys

Naturalistic inquiry, or ethnography, has its roots in anthropology and sociology and involves long-term exposure to a setting or a group of people. Extensive use of unstructured observations and conversations documented by detailed field notes form the basis for this type of research, often considered the purest form of qualitative research. Naturalistic inquiry is used when situations are unique or complex, when the level of uncertainty about the questions to ask is high and when there is little or no theory to direct the investigator.

A subset of this type of inquiry involves participant observation in which the investigator becomes a part of the setting or the process being studied. (Sofaer) reports that she was able to learn more from attending a few group meetings in a particular setting than she could have by using more structured qualitative methods such as interviews or surveys.

Case studies are the preferred strategy when 'how' or 'why' questions are being posed, when the investigator has little control over events, and when the focus is on a contemporary phenomenon within some real-life context. The case study is especially appropriate when the boundaries between phenomenon and context are not clearly evident. The case study copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result relies on multiple sources of evidence, with data needing to converge in a triangulating fashion.

The case study approach can involve a single event or multiple cases and can be short or long term. However, rather than requiring total immersion in the setting or culture, sampling of sites, experiences and/or informants is typical. The methods used in case study research is similar to those of naturalistic inquiry. However the data collection is often more structured, using key informant interviews, structured observations of events and interactions and the collection and content analysis of relevant documents (e.g., to help establish the facts, the assumptions, values and priorities, or to illuminate differences in perceptions). Case studies often also include quantitative data for background or to help generate questions to ask informants (e.g., data on demographics, heath status, utilization, finances, etc.).

Structured Observations of meetings
This involves attending meetings of the group that you wish to research. This can also be extended to observation of individuals in their daily work routine or on special tasks. The purpose of observing is to learn what is going on at the meeting and witness the group dynamic in process. This can be a rich information source as it can give researchers insight into the group.

Content analysis of documents
This is a non-intrusive form of research. This involves reviewing documents, memos or other pieces of written information for content and themes. By examining written word, the researcher is studying one type of communication that occurs in the selected sample.

Collection and analysis of other archival, administrative and performance data
This method also is non-intrusive. Information that has been previously collected, or secondary data, is reviewed to gain a better understanding into the topic. This information is part of the organization’s history and can be a valuable key to understanding the past.

Focus groups usually explore specific issues. The focus group brings together individuals chosen to meet a specific profile. They may be homogenous along some dimensions and heterogeneous along others and a structured, yet informal, setting is used to explore a limited number of questions. Focus groups, unlike individual interviews, provide the added dimension of the interactions among members. Focus groups are often combined with more quantitative approaches such as surveys that can be administered at different points in the group discussion and even used as grist for additional discussion.

Cognitive interviews are typically used in survey development. One-to-one interviews are conducted (with people meeting the criteria for completing a particular survey) as the individuals complete the instrument being tested. This method helps investigators understand how people perceive and interpret language and their own experiences as they refine the survey instruments.

Mail and telephone surveys are a method of collecting information by sending surveys via email or postal mail. Participants return completed forms to the researcher or an outside vendor. Surveys may ask respondents to rate items on a scale (e.g., Likert scale of 1-5). Some surveys also allow respondents to write their feelings or attitudes about a particular event or to elaborate in more detail on an item, or to express suggestions, etc.

What is the role of qualitative research in hypothesis testing?

The origins and development of qualitative research methods and their close association with inductive, interpretive and historical research have led many researchers to associate these methods exclusively with these forms of research and to fail to recognize their value in conventional deductive empirical research.

Some investigators, however, contend that hypothesis-testing, deductive research can benefit from the use of qualitative research methods - and that these methods can be used in a manner consistent with accepted standards of rigor and validity. In particular they believe that the acknowledged strength and unique contribution of qualitative methods in developing insights into actors' values, beliefs, understandings and interpretations of events and other phenomena, or in explaining historical occurrences, can enhance "conventional" forms of empirical research.

Brian Mittman, in a white paper prepared for the MDRC workshop on Management Research in VA, argues for the use of qualitative methods in hypothesis testing, and outlines the key components of the rigorous approach needed to use these methods successfully. His paper is motivated by two interests: first, convincing researchers not experienced in qualitative methods that they can enhance their empirical, deductive work, and, second, minimizing the misuse of qualitative methods in ways that threaten the validity of studies. Dr. Mittman's paper is linked here.

Source: http://www.colmr.research.med.va.gov/mgmt_research_in_va/methodology/qualitative_research.cfm#4

D. Data Research

Now take the TEST

Study this web-site for 3 hours for an approved (RN-CEP 11430, MFT- PCE 39) 3-hours Continuing Education Certificate (0.3 CEUs). Click here for the self-correcting test

© 1994-2006,  LearnWell Resources, Inc, a California nonprofit public benefit 501(c)(3) corporation, PO Box 944, Camino CA 95709. Updated April 15, 2009  privacy  feedback  email us  login.  After finishing a course, consider taking a related course.