Readings and Interpretations I

Please note: "Readings and Interpretations" are weekly "journal" posts for my Reading and Interpreting Research on Writing seminar addressing issues raised by the reading for class and my own research.



Researcher

I’ve been working on a project for the last few months that involves me reading over a large swath of research, analyzing it, categorizing it, and making a claim about shifts in the academic and cultural milieu as they relate to (or don’t relate to) the subject of my inquiry.  My sample size was big, and I’ve never done this type of analysis before, and my Microsoft Excel skills maybe aren’t what they should be, so this has been tough for me. 

Serendipitously, I’ve come across quite a few articles lately that should help me along.  The professor whose class got me started on this project emailed me a link the other day to a blog post written by Brian McNely (Credentials, as stated on his blog: Assistant Professor, Rhetoric and Writing Studies, Ball State) in which McNely reviews a recent article titled “Innovation in Qualitative Research Methods.”  This type of study belongs to a genre of research that McNely argues was first sort of defined by Mary Sue MacNealy as “meta analysis.”  Put simply, it’s “a literature review [and this is McNely quoting MacNealy here, which I realize is kind of confusing] ‘that is conducted empirically and analyzed statistically.’”

Here’s McNely expounding on the benefits of meta analyses:

 “Exactly how a researcher moves from the total population to this purposeful sample is explicitly articulated in a meta analysis or systematic lit review; this is important because such transparency means that the analysis is replicable. A solid meta analysis clearly defines the total population and criteria for inclusion, describes how and why articles were selected or excluded, and proceeds along a clearly defined schema for analysis. Performing this kind of rigorous, systematic review of literature in a given area can be tremendously useful for other researchers; this is the primary reason I appreciate a strong meta analysis or systematic lit review.”

Yeah.  Me too.  I think establishing systematic criteria for inclusion and strictly following a clearly defined schema of a body of research should allow other researchers to: look at historical trends in research with a focused lens on one particularity as it develops and shifts (or doesn’t) over time; and 2.) perform their own meta analysis of a different body of research, following those same “rigorous, systematic” steps, to compare and contrast their own findings and create a big picture of an issue that, for whatever reason, is bothersome to a particular discipline.

Which is close to what was realized in Research in the Teaching of English’s latest issue, the theme of

which is “Reading the Past, Writing the Future.”  It’s a special issue celebrating the 100^th anniversary of NCTE by tasking “teams” of writer/researchers to look back over empirical studies pertaining to a particular grade level (studies of primary school, secondary school, and college English classes) from NCTE’s journals and “provide an impressionistic report of that research and its continuities and discontinuities over the past century in 25 pages, including references” (135).  Despite the qualifier “impressionistic,” I thought the meta analyses were fairly systematically rigorous in their methods.  Jory Brass and Leslie David Burns’s contribution, “Research in Secondary English, 1912-2011: Historical Continuities and Discontinuities in the NCTE Imprint,” includes about three-and-a-half pages of methodological explication before getting to the actual analysis.  They, like the other teams, are careful to define how they interpreted their task—how they interpret “empirical research,” “secondary education,” “historical (dis)continuities”—and the criteria they used to whittled down their sample.

Brass and Burns really seem to touch on all the important issues that McNely (and MacNealy) appreciate in meta analyses.  There’s really no way to get around them and write a responsible historical analysis.  It’s important in any work of historiography, I think, for the author to acknowledge the perspective through which his/her study is being conducted.  And, moreover, it’s important to acknowledge that key terms (especially key terms) shift meanings over time.  So one crucial component of a meta analysis is a sort of terministic tracing that explores and exposes these shifts.  Also, doing such work up front allows researchers in the future to simply say, “Using so-and-so’s analytic framework, I argue . . .” and it protects the study from arguments such as, “So-and-so worked with a fixed definition of ________, not taking historical changes in the semantic field into account.”

The big take-away here is that meta analyses are great as overviews of lingering issues in a particular field, and they function well as transferable analytic frameworks for researchers to compare and contrast historical views within their fields and generally try to paint a big picture of trends in those fields.