Week #5: Introduction to Critiquing Research
By
Michelle Khatib,
Brigitte Lepine,
and Thomas Mortlock
This week’s
assignment focuses on reviewing as well as critiquing literature, and writing
reports.
Brigitte: I liked doing this chapter because it has given me direction on how to go about doing research. It also gave me tools to get there.
Michelle: Critiquing research has enabled me to distinguish between research that is quantitative in nature from that which is qualitative. It has also taught me the value of “doing your homework” before you research.
Tom: In order to enhance our own personal / professional development we must become great filters of the massive amounts of information we take in. Hayes and Neuman have introduced practical steps to evaluating research.
The following
information is presented in point form as it is assumed that you have read the
chapter.
Neuman emphasizes the
importance of communal social research, which means that a researcher is
obligated to communicate how they conducted their research along with their
findings and results. He also feels that without this communication the
research is considered to be incomplete. Therefore, his focus for this chapter
is to teach us the format for reporting on a study, the system of storing,
retrieving, referencing (includes the use of the Internet) and writing research
reports that other researchers have conducted. (p.445)
Neuman also introduce
us to the importance of literature reviews and explains how they may be used
for a variety of reasons such as to compare, replicate, criticize and to gain
knowledge. As Neuman points out, “a
literature review is based on the assumption that knowledge accumulates and
that we learn from and build on what others have done.” (p. 445) It is
important for us to remember that the most successful research findings were
built from the knowledge of others.
Neuman states, ”All reviews follow the first goal – to show familiarity
and establish credibility – to some degree.” (p.446). The following are the goals of a literature review:
·
To establish
credibility by demonstrating a familiarity with a body of knowledge
·
To show the path of
prior research and to show how your current research is linked to it.
·
To integrate and
summarize what is already known in a particular area.
·
To learn from others
and their research, and to stimulate new ideas.
1. Six types of reviews:
Self Study-
is used to build confidence and knowledge in an area by reviewing the works of
others.
Context Reviews
is the most common type of review because it is used to show a link to prior
research reports and it helps to link the current project to others. It also outlines the direction that the
researcher will be taking on a question and shows the development of
knowledge.
Historical Reviews
–is just what is says. It is a summary
of the history of research regarding a certain issue. It is used to show others
where your idea has originated and where it is progressing to in the future.
Theoretical Reviews –
pulls together ideas from many past research reports and tries to show how they
differ, how they are the same, what information is known in the present time,
and what direction is needed to advance. Neuman states,” In addition to examining
the consistency of predictions with findings, a theoretical review may compare
theories for the soundness of their assumptions, logical consistency, and scope
of explanation.” (p. 446)
Integrative Reviews –
this is used to show the current knowledge that we have obtained in a fast
growing area of knowledge by pulling together completely distinct or different
types of research.
Methodological Reviews
– “is a specialized type of the integrative review.” (Neuman p.447) It analyzes the methods used to obtain data
and points out where the flaws of the research might be in their designs,
samples, and measures.
When collecting
Research Literature you need to know where to locate it, but you also need to
make sure to record the information in the ‘wanted’ format: APA, CMS, or
others.
APA Style Guide - M.
Plonsky, Ph.D.
http://www.uwsp.edu/psych/apa4b.htm
MLA Works Cited
Format adapted from the MLA Handbook, 4th ed. 1995
http://wwwshs1.bham.wednet.edu/curric/cool/mla.htm
If you are aware of
better ones, please let us know at:
or
or
Research Literature
can be found in different forms such as:
Scholarly
journals,
Books,
Dissertations,
Government
documents,
Policy
reports
Presented
papers.
L Beware of other literature or media who often present incomplete and bias reports.
Scholarly journals
are often the first place to look, because as we now know, researcher are
publishing (or trying to) their report in this fashion. Usually you find them
in libraries (periodicals or serials, indexes). Neuman has a 2 pages list (p 449-450) of many Scholarly Journals
in the Social Science. These can be
located in/on:
%
college and
universities.
-
the Internet (you may
have to register first).
The
Qualitative Report
http://www.nova.edu/ssss/QR/index.html
-An
online journal dedicated to qualitative research and critical inquiry since
1990.
Dianne’s List:
www.pacprospector.com/base.html
For educational researchers, Pacprospector is a
useful search engine that works with a question database.
www.aera.com
(Measurement & Research Methodology
Forum)
www.aera.net/meeting/papers99.html
(The American Educational Research Assoc.)
www.gsh.org/wce/articles.html (education
features)
www.ucalgary.ca/~iejll (International
Electronic Journal of Leadership & Learning)
www.mci.com/marcopolo Created by MCI, the MARCOPOLO site grants
access to research and information compiled by reliable organizations
Books contain a
collection of research articles and research reports. Again you’ll find this mostly in colleges or universities. It is important to consider this literature
because some types of research are only presented in this format, but they are
difficult to find because there are few sources that list them.
Dissertations
of other students are a valuable source of information. Found mostly in universities.
Government documents. The government often sponsors research;
therefore, they also have many research reports. Many Colleges and Universities have a collection of ‘most
requested’ documents in it is a “government documents” section. This is not
found in catalog system of the university.
It is found in special lists of indexes and publications.
Policy Reports and Presented Papers are the most difficult sources to find. First, try: to write to the research center
and ask for lists. Then, ask to get a
list of papers at the meeting or the conference. Finally, after you found an interesting article, write to the
author or institute to have the article sent to you.
We have summarized
the search for literature reviews. In
the textbook (p449-450), there is a long list of references in the Scholarly
Journal written in English for the Social Science. Next is the procedures needed for a good
review.
3. Procedures to a Systematic Literature Review.
In the textbook
(p.461), Neuman uses “good and bad” examples reviews to emphasize the
importance of organizing common findings or arguments together, addressing the
key ideas first, logically link the statements or findings, and to note
weaknesses and discrepancies in research.
Following are the basic steps.
Define and refine a topic
Design a search
(where to start? plan a search strategy, set parameters)
Locate research reports
(look at previous lists in other research literature)
Knowing what to record
(write everything about the source of the report and try to group the
information relevant to your research: concept, finding, etc.)
Organizing Notes
(decide on what approach you will structure your information, depending on the
purpose of your review –test a few)
Writing the Review
The Internet was then
brought forward as a good tool for locating literature on a specific topic, but
Newman emphasizes that it is only one tool – not the only one. It has many benefits such as speed and cheap
access, links to other sources etc… He
also cautions us to beware of the down side to the Internet. Anyone is allowed
to publish on the Internet, and it is very difficult to distinguish the “trash”
from the genuine research. Neuman
also states that many of the excellent sources are not available on the net
unless you subscribe. As we all know,
the Internet can be very time consuming and URL’s tend to change from time to
time leaving the researcher feeling in an unstable environment.
You are probably
already aware of how important the writing process is. Here’s the summary of the major points with
some resources online:
Ø
Organizing thoughts
(an outline can be handy)
Ø
Going back to the library (you may have to go
back to check for updated sources etc.)
Ø
The process of writing
(prewrite, then compose, and finally rewrite)
University of Richmond Writer's Web
http://www.urich.edu/~writing/wweb.html
-Explore
topics by stages of the writing process
Purdue
University's Online Writing Lab
http://owl.english.purdue.edu/
-More
resources
Writing
Guides on the Writing Center at Colorado State University
http://writing.colostate.edu/reference.htm
-Think
of our Writing Guides as online textbooks.
THE
RESEARCH PROCESS -5 steps to success
http://www.auckland.ac.nz/lbr/instruct/research.htm
The
Research Process -Basic steps to follow for any research project.
http://libweb.sonoma.edu/research/process/default.html
The
Seven Steps of the Research Process
http://www.library.cornell.edu/okuref/research/skill1.htm
How
to Write Technical Papers and Reports
http://ciips.ee.uwa.edu.au/~mike/swc/
How
to do Research
http://www.geocities.com/Athens/3238/index.htm
-This
site covers all aspects of conducting research. It includes the What? Why? and
How? of conducting formal research, various stages of writing of research,
selection of supervisor, and writing of graduate & postgraduate theses.
5. Quantitative Research or Qualitative
Research Report
As discussed in
previous chapters, you must decide on what type of research process you are
going to use. When writing quantitative
research or a qualitative research report there are steps that need to be
followed. See the following chart:
Vs.
Quantitative Research
1. Abstract
or Executive Summary |
Qualitative Research
1. Does not
follow the fixed format with standard sections. Data and theoretical generalizations are intertwined with the
evidence, which can be very detailed with frequent quotes. |
2.Presenting the Problem
|
Tend to be longer and book style because of the extensity of the data.
|
3.Describing the Methods
|
May be written in the first person.
|
4.Results and Tables
|
Has the same purpose as Quantitative Research – to clearly communicate
the research process and data collected.
|
5.Discussion
|
Field Studies, and Historical –Comparative Research fall into this
category.
|
6.Drawing Conclusions
|
|
Neuman concludes by
introducing what a proposal is and how you find more information about
grantsmanship. He stresses the
importance of building a positive track record, constructing neat and
professional proposals (with a budget intact, which is note worthy to
investors).
Social
Science Research Council The Art of Writing Proposals
http://www.ssrc.org/artprop.htm
Planning
and Writing a Research Proposal:
A
guide for teachers doing research
http://www.scre.ac.uk/spotlight/spotlight47.html
It is strongly recommended that the reviewers of the proposal check to see if it contains the following points:
Ø
addresses a valuable
research question
Ø
follows instructions,
is written well, clear and easy to follow
Ø
the research
procedures are clearly defined with high methodology and research techniques
that are appropriate and related to the research question.
Ø
has an evaluation
process to make sure that the objectives to the project are met.
Ø
is well planned with
realistic budgets and schedules
Ø the experience and background is there to complete a successful research project.
Hayes Reviewed
A. A Brief Guide to Critiquing Research
Hayes stated that research is between two extremes; solutions to all problems or completely useless. Thus we as educators must become familiar with some of the basic elements of the research process in order to assess valuable literature to enhance our professional development.
B.
Why Read Research?
1. Intuitive skill and accumulated experience account for much good teaching practice however, good research with systematic inquiry can offer valuable explanation to puzzling phenomena, challenge conventional wisdom and introduce alternative strategies.
2. Research also impacts educational policy decisions, finding sources, development of curriculum and other arena that have a direct effect of you in the classroom.
3. Individual professional growth requires us to reflect on our practices as educators and on the perspectives of others. Through the process of evaluating research we can thoughtfully consider both the strengths and limitations so we can be proactive rather than reactive to current trends.
C.
Understanding Different Approaches to
Research
1. Basic Research - done to test a theory
2. Evaluation Research – done to assess existing practices for its merits
3. Action Research – done to solve specific problems
There are two ways to collect and analyze information:
1. Quantitative – empirical data / positivist philosophy of science
2. Qualitative – holistic manner to collecting data / naturalistic philosophy
The debate between these two approaches will remain throughout time but it is possible to utilize both when collecting data.
D.
Research Methods
1.
Survey Methods
-goal of generalizing to the larger population from quantitative measures of a sub group
-data collecting through questionnaires or interviews
-evidence of validity and reliability needed
2.
Experimental and Quasi-Experimental Methods
-goal is to establish cause and effect
-manipulate independent variables to see its effect on dependent variables
-a hypotheses is suggested and subjects are assigned to an experimental or control group
-identification and control of extraneous factors is important
3.
Ethnographic Methods
-goal is to describe the culture (unique and distinctive in their qualities) of a particular setting
-data collection is open-ended using a combination of qualitative methods like observations and interviews of people in their natural setting
4.
Case Study Methods
-goal to develop comprehensive understanding of an individual group, institution or other clearly defined social entity
-three kinds of Case Studies
- qualitative measures predominate but quantitative measures are also appropriate
a) Descriptive – basic information into phenomena
b) Interpretive – new concepts developed
c) Evaluative – judges worth of phenomena
***Be careful about generalizing information gained from this research method into other populations***
5.
Historical Methods
-goal of understanding the past by examining historical materials
-concerns a) quality of source
b) genuineness of source
c) be sensitive to presentism (using recent times to explain things)
6. Participatory Research
-goal is to challenge and transform inequitable social and political structures
-active participation of people to acquire knowledge and become self-empowered
-immediate problems are investigated so collective action can be taken to implement solutions
-evaluated by its positive change for the individuals involved
E. What Makes Good Research?
1.
Research reporting
-process is well documented
a) identifying a problem
b) establishing a conceptual framework
c) specifying the phenomena to be studied
d) selecting and using a particular method of data collection
e) analyzing data
2. Substance of
the Research Study
-research has to be adequately reported in order to make assessment possible of the substance of the study
-no clear-cut rules to evaluating the substance of a study but its significance in one criterion
F.
How to Read a Research Report
1.
Wrong Way
We didn’t think the group needed to be aware of this area!
2.
A Better Way to Read Research
a) skim abstract
b) read introduction, problem statement, and conclusion to provide insight into importance of study
c) if interest holds read more thoroughly for meaning
3.
How to Read Statistics (for Nonstaisiticians)
a) clarify the purpose of the statistical procedure
b) is sample size adequate for the statistical methods used
c) watch for lack of distinction between statistical significance and practical significance
G.
Conclusion
On a personal note, we feel that in order to enhance our professional development as educators we must become familiar with some of the basic elements of the research process. Valuable literature is a key ingredient to providing solutions to the challenges we face in the classroom.
Michelle – Just this week, I had the opportunity to teach the research process to my grade six class as part of their language arts program. It has been a very valuable process to see the children narrow down their question, give appropriate choices for the survey, and to watch how twelve-year-old children can evaluate their data. The students were asked to create a bulletin board display with their information, which included the question, the procedure used, the choices, the data (in percentages), a graph, a summary of their findings along with a critique as to how they could make their statistics more valid. I found it utterly amazing to witness the discussions that were taking place. Grade six students made some of the following statements:
Ø “Next time, I think that we should survey a larger group of people so that our data is more accurate.”
Ø “I think that we worded this question incorrectly because it didn’t give them enough choices and the answer to our question might not be right.”
Ø “I don’t think that we should allow others to see the data that we have already; it might sway their decision.”
Ø “The words that I used in my question forced people to choose a certain answers.”
What this chapter has allowed me to realize is that we are constantly surrounded with misinformation and tainted research. Having the experience and knowledge to decipher what is valid and what is “junk” is a valuable asset.
Brigitte - For the few last months, I have been involved in a project
regarding research methodology. My job
has been to find information online that relates to this wide subject. Lately I felt out of steam, not knowing what
direction to take so really feeling lost.
Taking this course
has helped me to understand the different approaches to research as well as
their various methods (survey, field, historical, etc.) It is helping me to see
the big picture finally. Chapter 16
gave me the tools to become more critical about my findings as well as gave me
some focus.
Now, I feel confident
that I can go back to my work and fill the gaps of information missing. This is a nice feeling…
Thomas - One thing I have enjoyed over that past five years is doing
my professional growth plan. I feel it
has made me become very focussed on the needs I have as a professional. Research is a driving force in
our profession from helping us reflect on our current practices to funding.
This week’s experience helped drive home the importance of
evaluating/critiquing research for the direct impact it has in every classroom.
Our Questions
After reading the reflections from last week’s
assignment, our group has decided to personalize technology so that we can get
a better understanding of who is commenting on the other end. Therefore, we are proposing that you answer
to Michelle’s question, or all questions if you like, via voice. (We are hoping
that you all have microphones, if not we understand that as well.) The instructions for doing this are as
follows:
Plug your mic into the back of your computer if you have an external mic
- Plug your mic into the back of your computer if you have an external mic
- If you have a later Mac or iMac version you probably have a built in Mic so you can just speak into your computer.
-
Restart you computer.
-
Go to the weekly assignment area, which you are
in now.
-
Go to Message – new message
-
Go to Edit – Insert – Voice
-
Click on the red square button and record your
message. When you click on the same red
button it stops recording. You can
replay your message to listen to it by pushing the rewind button that is next
to the red square button.
-
Note:
Once you have pushed the stop button it automatically saves it as an
attachment to your email. To see this
attachment or to delete it you have to pull down the record bar, so that you
can see under the subject heading the area that says attachment. If you wish to delete a recording you just
highlight it and push your delete key.
-
You may include some voice and type some of your
message as well.
-
If you have any problems email Michelle at her U
of C First Class address.
Michelle – Can you remember a time when you had an
experience where you had to decipher “good from bad” research. What did you learn from your experience?
Brigitte - How would you describe the “path” of critiquing
literature and writing reports? (Ex. a
linear journey? Cyclical? Circular?)
Explain your answer.
Tom – There has been a lot of negative talk that soon
teachers in Canada will face the same fate as our US counterparts, by being
required to attend/complete a certain number of hours of professional
development activities in a year. Is
this talk suppose to worry me?
Let your hair down
and let’s have fun with this one!