Methodological Memes and Mores: Toward a Sociology of Social Research

A plethora of scholarly research has been conducted on social science: on its organizational and communicative patterns, on the historical development of research standards, and on the diversity of local research practices. But this body of work on the sociology of social research does not hang together in ways that it could, and should, if knowledge is to accumulate. Contributors hail from various fields, subfields, theoretical perspectives, and methodological bents, and there is no extant subfield to join, legitimate, and reinforce their mutual interests. Thus, the aim of this review is not only to summarize themes, identify gaps, and suggest fruitful avenues for future research, but also to serve as a unifying force for scholars interested in studying social science from a sociological perspective. The sociology of social research, far from being a trite exercise in navel-gazing, is critical for the future viability of sociology, for the discipline’s legitimacy and autonomy, and for improving social research more generally. 33 Click here for quick links to Annual Reviews content online, including: • Other articles in this volume • Top cited articles • Top downloaded articles • Our comprehensive search Further ANNUAL REVIEWS A nn u. R ev . S oc io l. 20 08 .3 4: 33 -5 3. D ow nl oa de d fr om a rj ou rn al s. an nu al re vi ew s. or g by U ni ve rs ity o f A ri zo na L ib ra ry o n 08 /2 5/ 10 . F or p er so na l u se o nl y. ANRV348-SO34-03 ARI 7 June 2008 16:29 It is not at all unreasonable to view the results of social research as a dependent variable that is affected by the reality of what it is meant to study, and by other various technical matters that keep the study from being ideal, and also by the social context of the work itself. Stanley Lieberson (1992, p. 60) Sociology, which invites the other sciences to address the question of their social foundations, cannot exempt itself from this calling into question. Casting an ironic gaze on the social world, a gaze which unveils, unmasks, brings to light what is hidden, it cannot avoid casting this gaze on itself—with the intention not of destroying sociology but rather of serving it, using the sociology of sociology in order to make a better sociology. Pierre Bourdieu (2004, p. 4) Research is a human activity, not immune to social influences, and social research is no exception. This is immediately clear from investigations of research practices in the natural sciences, which document the malleable and formative aspects of scientific facts and technical procedures (Knorr-Cetina 1999, Latour & Woolgar 1979, Lynch 1985). Perhaps owing to perceived risks of studying phenomena too close to one’s self, only a small (but growing) number of social scientists have examined social research practices—the decisions that social scientists make as they go about their research, which may or may not be guided by methodological precepts. Turning a sociological eye toward our own research practices prompts questions: How, if at all, does everyday research practice diverge from formal approaches described in methods texts and research publications? How do social forces at micro, meso, and macro levels shape the way research is carried out? Why do some research practices spread and become dominant while others do not? Several characteristics of social research and its practice encourage systematic inquiry. First, social research is a ubiquitous, everyday, common activity: As of 2004, almost onehalf million individuals (491,700) in the United States alone were classified as social scientists (U.S. Census Bureau 2007). Second, the demonstrated and potential influence of social research—on the public and on policy (Calhoun 2007, Gans 1989)—demands that social research be analyzed systematically. Third, social research practice is consequential: There are long-term ethical, methodological, and policy implications. For example, Porter (1995, p. 31) shows how population numbers depend on “the methods specified for getting them,” and how socially organized and contingent categories (such as “Hispanic”) are impressively resilient once they are put in place. For these reasons, it is important to ask how research in the discipline is supported locally and socially (Lynch & Bogen 1997) and to investigate the “underbelly of social research—those thoughts, actions, constraints, and choices that lurk beneath the surface of our well-dressed research publications” (Staw 1981, p. 225). Indeed, much can be gained from studying social research from a sociological perspective. Describing, clarifying, and explaining the typically hidden steps of the research process encourage social researchers to be more methodical about matters that have been left largely to idiosyncrasy and circumstance (Kulka 1981, p. 176), thereby increasing transparency, attentiveness, and rigor. A better understanding of the social research process should also make it more learnable and transmittable (Becker 1998). In the past, the multiple small steps involved in research were disguised, learned only through experience; more recently, researchers are sharing their decisions, pitfalls, and surprises more openly (Hargittai 2008). Scrutinizing social research should also help improve evaluation efforts, which lie at the heart of scientific communication and progress (Roth 1973). It may also help break down what some see as false and unnecessary dichotomies: quantitative/qualitative, ideal/real, official/informal, and standards/practice (Peneff 1988). Methodological self-reflection, practiced by some of the most sophisticated quantitative methodologists, permits recognition of the subtleties of practice, which is laden with ambiguity 34 Leahey A nn u. R ev . S oc io l. 20 08 .3 4: 33 -5 3. D ow nl oa de d fr om a rj ou rn al s. an nu al re vi ew s. or g by U ni ve rs ity o f A ri zo na L ib ra ry o n 08 /2 5/ 10 . F or p er so na l u se o nl y. ANRV348-SO34-03 ARI 7 June 2008 16:29 and discursive elements (Breiger 2002). Because research is a part of theory (Lieberson 1992) and research is conducted by members of all substantive subfields, induced sensitivity to methodology radiates into all fields of social inquiry and thus benefits sociological analysis more generally (Lazarsfeld 1962). Addressing the social character and foundations of social research practice requires a realistic and skeptical perspective, but the goal is to improve—not debunk—social research. Rather than relativizing or discrediting scientific knowledge by reducing it to social or historical conditions, submitting social science to sociological analysis will check it and strengthen it, by enabling “those who do science to better understand the social mechanisms which orient scientific practice” (Bourdieu 2004, p. viii). This also provides a healthy antidote to philosophical studies of epistemology, which focus more on the truths of established science than on the errors of science in progress, and scientific activity as it actually is (Bourdieu 2004, p. 3). Calls for taking social research practice as an object of inquiry are not new (Lazarsfeld 1962, Lieberson 1992), and scholars heeding these calls hail from a variety of eras, specialty areas, and methodological bents. Aaron Cicourel, Harold Garfinkel, Anselm Strauss, and Doug Maynard come from an ethnomethodological tradition. Nora Cate Schaeffer and Tom Smith represent the survey research tradition. Historians (Margo Conk, Theodore Porter) and sociologists interested in the history of sociology (Charles Camic, Jennifer Platt) have also turned their attention to social research methods and statistical practice. Other scholars, including Howard Becker, Stanley Lieberson, and Andrew Abbott, are methodologists in the broadest sense. This diversity in origin certainly makes the task of describing and assessing the state of scholarship on social research practice formidable (Kulka 1981), but more importantly, it presents challenges to the formal organization of such scholars, and the subsequent recognition and legitimation of their work. Without a common language, journal, or recognized specialty area for the study of social research practice, there are few opportunities for scholars working on this topic to learn of their mutual interests, share insights, and build on each other’s work. This review is one step in the process of filling that gap. Although traditionally focused on the natural sciences, the sociology of science has become more open to the study of social science and provides a foundation for the emerging sociology of social research practice. Indeed, there is overlap in the topics and practices studied, such as measurement (Maynard & Schaeffer 2000). But there are some reasons for distinguishing the sociology of (natural) science and the sociology of social science. As I discuss below, the sociology of social science uses a broader array of methods, including not only qualitative and historical research, but also experiments and surveys. And social scientists study phenomena that they themselves participate in; thus, the choice of problem and attempted solutions are likely influenced by the social scientist’s personal experience as well as his or her audience (Cole 1994). While acknowledging social pressures that affect both the natural and social sciences, Lieberson (1989, p. 61) notes an additional pressure faced by the latter: the differential treatment received by results that clash with normative social views and results that support such views or are tangential to them. The social sciences are different enough, epistemologically, from the natural sciences (Hargens 2000) to warrant distinct empirical attention. This was corroborated in a study of fellowship review panels (Guetzkow et al. 2004), where conceptions of originality not found in the natural sciences (such as taking a new approach) were typical in the social sciences and humanities. The goal of this review is to provide a comprehensive assessment and evaluation of the state of research on the sociology of social research and to identify topics ripe for further investigation. Toward this end, I begin by cataloging the range of social research practices that have been investigated to date. This descriptive effort is intended to draw together www.annualreviews.org • Toward a Sociology of Social Research 35 A nn u. R ev . S oc io l. 20 08 .3 4: 33 -5 3. D ow nl oa de d fr om a rj ou rn al s. an nu al re vi ew s. or g by U ni ve rs ity o f A ri zo na L ib ra ry o n 08 /2 5/ 10 . F or p er so na l u se o nl y. ANRV348-SO34-03 ARI 7 June 2008 16:29 a diverse set of practices that have been studied by a diverse set of scholars, over a long period of time. Unlike continual reviews of traditional subfields, this one is not restricted to recent works. But like most reviews, it is necessarily selective. I highlight studies that take research practice—and especially variation in practice across time and space—as an interesting phenomenon that itself requires explanation, thereby underemphasizing a related body of literature on the methodological and ethical implications of research practices (see Table 1 for a catalog of both kinds of studies). I then move into a more analytical discussion of the myriad social factors found relevant to the use, transmission, and dissemination of practice. Lastly, in order to suggest future research directions, I review the methods used (and not used) to study social research practice and identify important but understudied research practices. I end the review by proposing a series of potential research questions. A PALETTE OF PRACTICES Many practices related to the survey research tradition have been studied extensively. Largely based on a concern for declining response rates, some scholars have studied the practice of subject recruitment, focusing on the impact of researcher-participant interaction. Interviewing is arguably the research practice that has received the most scholarly attention, perhaps in response to early standardization efforts. As Maynard & Schaeffer (2000) demonstrate, detailed, ethnographic styles of observation and analysis have much to contribute to our understanding of survey research and often illuminate the social and interactional underpinnings of work done in survey research centers. Several contributions to the edited volume Standardization and Tacit Knowledge: Interaction and Practice in the Survey Interview (Maynard et al. 2002) illustrate this, as does recent work by Weinreb (2006), who highlights the importance of interviewer-interviewee relationships. Table 1 Studies of research practices, by their stage of the research process and place in an explanatory model Place in explanatory model Stage in research process Research practice as an explanatory variable Research practice as an outcome of interest