Research design for quantitative analysis – Autumn 2010

 

This is the FIFTH update of a previous syllabus (November 2010).

 

WATCH THIS SPACE FOR FURTHER REVISIONS

 

THIS IS A DEMANDING COURSE. PLEASE READ THIS DESCRIPTION CAREFULLY BEFORE DECIDING TO TAKE IT. (Please also look out for revisions)

 

Research design is the process of creating a scientific plan for answering research questions through hypothesis generation, case selection, measurement, and analysis. Research design is both a formal and a creative process whose application, properly conducted, yields knowledge about how the world works. The creative aspect involves the application of strategic and tactical thinking in what Donald Stokes once memorably called Òa game against nature.Ó This course focuses on the strategic and tactical thinking that underlies good research design, particularly in the context of designs involving quanti­tative analysis. Relevant examples will be taken from the political behaviour, political sociology and international relations literatures.

The course is broken into two unequal halves. In the first two weeks the instructors will be pri­marily responsible for introducing discussions about fundamental topics on the logic of quan­titative research designs in general and as applied specifically to survey research. In the re­maining weeks, researchers taking the class will be responsible for introducing dis­cussions that focus on particular research strategies generally known as experimental, cross-sectional, hierarchi­cal, time-series, and panel (actual topics may be varied on the basis of researcher interest as there are too many topics to cover them all). Researchers will be responsible for pick­ing particular article(s) exem­plifying that research strategy and, in consultation with the instruc­tor, for choosing additional relevant readings. One hour will be devoted to the researcher presen­tation leaving one hour for guided discussion and the coverage of additional pertinent issues.

Although these remaining topics are identified in terms of data organization strategies, it is expected that through judicious choice of articles to be presented, additional issues (both strategic and tactical) can be covered at the same time – for example missing data handling, dimensional analysis, item scoring, data stacking, structural equation modeling, instrumental variable regression, path analysis, and the choice of estimation methods. All members of the class will be expected to have read each paper beforehand, so the presentations should focus on lessons to be learned, including suggestions for improvements in strategy and tactics, Researchers who present a particular article should replicate the analysis (as far as possible, given data availability) and try out suggested improvements. They should provide the STATA annotated syntax for the benefit of other members of the class. This means that original research will be conducted for each presentation (but researchers may pick research strategies and articles relevant to their thesis research).

Thus the seminar is intended for researchers who have completed the Intermediate Data Anal­ysis Seminar (or reached the same level in other ways) so it is not necessarily suitable for researchers in their first year. It builds self-consciously on the Core Methods Seminar and assumes careful reading and comprehension of the Schmitter chapter on research design, along with the Chwas­zcza, Heritier, Mair and Franklin chapters in the departmentÕs core methods book. Moreover, the first chapter of Van EveraÕs Guide to Methods for Students of Political Science should be carefully read before the first meeting of the class. Many of the readings will be available in the course dropbox folder.

Students who find themselves challenged when it comes to econometrics should read the first five chapters of KennedyÕs Guide to Econometrics (not the small print). Anyone taking this course will probably want to own a copy but chapters relevant to the class have been scanned and uploaded to the Class Materials web page (see below). If you have trouble reading equations, see my ÒEquations for DummiesÓ in the class dropbox.

 

 

Readings

 

Readings are split between three repositories. Some are in the dropbox, others are available as electronic publications through the EUI library (links below). Other links take you directly to a folder on my homepage at Class materials

 

To access this folder you will require a username and password. In this (dropbox) version of the syllabus I can tell you that the username for the folder is ClassStuff (capitals optional) and the password is studen1 (capitalization matters - note that the final character is the number 1 not the letter t). Readings added this year will appear first in the appropriate dropbox folder (one for each week) and will later migrate to the ClassStuff folder in order not to overflow the dropbox.

 

Preliminary readings are listed on the syllabus insofar as they are already in the dropbox or on the web. Updating the readings may result in readings being removed as well as added.

 

 

Stata license renewals

 

Many of you have Stata licences that will expire this month. As and when your licenses expire you will need to migrate your work to the version of Stata (much more powerful if a bit slow to react) accessible on a server known as the Citrix Farm or Citrix Cluster. Please read my usage notes in the dropbox and start using the Citrix version of Stata BEFORE your license runs out. If you encounter difficulties you want leisure to deal with these while you still have another means to access Stata.

 

 

 

Seminar Topics

 

Oct 4: Mark Franklin and Fabrizio Bernardi: Why design research (why not just do it)?

Generic problems in quantitative research. Course structure and organization.

            Social research as a 'game against nature' – a game of strategy and tactics

                        Strategic thinking in research design: the choice of method

                        Tactics of research design – measurement and analysis

                        Inductive and deductive reasoning in hypothesis generation

Statistical inference: getting a handle on the chances of being wrong

            The data box and the opportunities it presents

                        Cross-sectional, longitudinal and hierarchical designs

            The fundamental problem of causal inference – our inability to observe

counterfactuals

Selection bias; Omitted variable bias; Case selection and sampling theory

The level of analysis and associated fallacies: turning a problem into an

opportunity

The problems of identification and endogeneity

Readings:        King et al. (1994) Ch1-6 (Library reserve);

Hainmueller and Kern 'Incumbency and Spillover' (Electoral Studies 2008);

Manski 'The Identification Problem' Intro and Ch 1;

 

Oct 14 (Week 1, Thursday at 3pm): Mark Franklin and Fabrizio Bernardi: Measurement problems

            Identifying error: reliability, validity and specification assessment in practice

            The consequences of error: bias, specification error, random error

Robustness and generalizability of findings: how widely they apply

The special role of survey research in overcoming error

            Problems of survey research

                        Response bias

                        Question wording

                        Questionnaire and interviewer effects

                        The endogeneity problem in survey research

            Panel and 'rolling thunder' surveys

            Linking surveys with other data: quasi-panels and hierarchical linkages

The central roles of falsifiability and replication; the edifice of knowledge

Readings:        Bradburn, Sudman and Wansink (2004) Library reserve;

Glenn Firebaugh Seven Rules for Social Research Chapter 5

Schlotter, Schwerdt, Woessmann, 2010

Raghunathan 'What do do with missing data'

Wretman (2005) Allowing non-response may give a better estimate

Franklin and Wlezien (2002) selections downloadable from Class Materials.

 

Oct 18 (Week 2): Mark Franklin and Fabrizio Bernardi: Getting serious with Stata

                        Reshaping and restructuring for different purposes; changing the level of analysis;

creating proximity and affinity measures; handling missing data.

Readings:        Franklin 'Getting Out the Vote' (EJPR 1991)

van der Eijk at al ÒElectoral UtilitiesÓ (Electoral Studies 2006)

 King, Tomz and Wittenberg (2000) 'Clarifying presentations'

Stata manual (Online from your copy of Stata)

(More to follow)

 

Oct 25 (Week 3): Goerge and Pedro: Hierarchical Linear Modeling (HLM, MLM)

             Readings:       Beaudonnet et al., "Advice for Comparativists" in dropbox

Brambor, Clark and Golder 'Interaction Models'

                                    Steenbergen and Jones 'Multi-level data structures' (2002)

                                    Franklin, 'Choosing Europe? Data structure' (from van der Eijk and Franklin 1996)

Pardos and Dinas (2010) Electoral Studies in dropbox For replication

 

Nov 4 (Week 4, Thursday at 5pm): Kathrin and Emre: Time Series Cross Section (TSCS) models

              Readings:      Franklin 'Time Series Estimation Issues'

                                    Kennedy Ch8, 9 and 19 + notes downloadable in several files from Class Materials.

Beck and Katz (APSR 1995)

Beck (Oxford Handbook 2008)

Wilson and Butler (2007) Political Analysis

Banducci et al. (2009) JEPP in dropbox For replication

 

Nov 8 (Week 5): Juana, Marit and Anna: Models for panel data

               Readings:     Peracchi 'European Union Household Panel'                          

Watson 'Panel Attrition'

Woolridge Ch14 in dropbox

Gerlach and Stephen (1996)  For replication

 

Nov 15 (Week 6): Silvia and Andrea: Models for event history data

              Readings:      Agresti and Finlay (event history etc.)

Fabrizio Bernardi, "Introduction to Event History Analysis using STATA

            In dropbox

Bernardi (2001) Quality and Quantity

Box-Steffensmeier and Zorn (2002) For replication

Box-Steffensmeier and Jones (1997)

 

Nov 22 (Week 7): Daria and Pelle: Measurement models and latent structure analysis

              Readings:      Coombs 'Theory of Data' (1966)

                                    Kaufmann et al. (2002) in dropbox For replication

                                    Jacoby (1991) – In three parts, downloadable from Class Materials

                                    Poole (1998a) and Poole (1998b), downloadable from Class Materials

                                    Scheuer How Europeans See Europe

                                    van der Eijk handout

                                    (More to follow)

 

Nov 29 (Week 8): Alexis, Jenny and Reuben: Propensity score matching

              Readings:      Becker and Ichino 'Estimation with propensity scores' (2002)

                                    Sekhon 'causal matching' (2004)

                                    Sianesi 'Propensity score matching (2006)

Arceneaux, Kevin et al. (2010) "A Cautionary Note on the Use of Matching"

Sociological Methods and Research (39:2) 256-282. For replication

 

Dec 6 (Week 9): Laurie: Interaction models

            Readings:        Ray, Leonard (2004) "Don't Rock the Boat: Expectations, Fears, and Opposition to

      EU Level Policymaking" in European Integration and Political Conflict, G. Marks

      & M. Steenbergen eds. Cambridge: Cambridge University Press. For replication

Brambor, Clark and Golder 'Interaction Models' And the webpage:

     http://homepages.nyu.edu/~mrg217/interaction.html

                                    Braumoeller, Bear. 2004. ÔÔHypothesis Testing and Multiplicative Interaction

                                         Terms.ÕÕ International Organization 58:807–820.

                                    Norton, Wang and Ai (2004) 'Computing Interaction Effects' (supplementary)

 

 

Dec 9 (Thursday at 5pm in the Emeroteca): Discrete choice models and voting utilities

            Readings:        Alvarez and Nagler (1998)

                                    Whitten and Palmer (1996) For replication

                                    Van der Eijk et al. (2006)

 

 

 

ADDITIONAL TOPICS FROM PREVIOUS SEMINARS

 

Research designs for experimental and quasi-experimental data

             Readings:       Bruter: "Symbols, Media and Mass European Identity"

                                    Wright, G., Schaffner, B. (2002) "The influence of party"

                                    Willer, D., and Walker, H. (2007) Building Experiments

 

Research designs for cross-sectional survey and public record data

            Readings:        Cohen and Cohen, 3rd ed. Ch 3

                                    Newton and Norris "Confidence in Public Institutions"

                                    Van der Eijk and van Egmond "Effects of Low Turnout"

 

 

References  (these will expand as readings as topics are prepared)

 

Achen, Christopher (2006) "Two-Step Hierarchical Estimation: Beyond Regression Analysis" Political Analysis 13:447-456.

Beck, Nathaniel (2007) "Time-series-Cross-Section Methods" Annual Review of Political Science.

Becker, S. and A Ichino (2002) Estimation of average treatment effects based on propensity scores. The Stata Journal

Bradburn, Sudman, & Wansink (2004). Asking questions: The definitive guide to questionnaire design (revised edition).

Brambor, Thomas, William Roberts Clark and Matt Golder (2006) "Understanding Interaction Models: Improving Empirical Analyses," Political Analysis 14:63-82.

Bruter, Michael (2007)  "Symbols, Media and the Emergence of a Mass European Identity" paper presented at the APSA.

Cohen and Cohen (2003) "Multiple regression/correlation with two or more independent variables", in Cohen et al., Ch 3.

Coombes, Clyde (1966) "An Overview of a Theory of Data" in A Theory of Data. Ann Arbor MI: Thesis Press.

Dillman, Don A. Glenn Phelps, Robert Tortora, Karen Swift, Julie Kohrell, Jodi Berck and Benjamin L. Messer (2008) Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet, Social Science Research, in press.

van der Eijk, Cees (undated) handout on latent structure analysis.

van  der Eijk, Cees and Mark Franklin (1996) Choosing Europe? The European Electorate and National Politics in the Face of Union. Ann Arbor: University of Michigan Press.

van  der Eijk et al. (2006) "Rethinking the dependent variable in electoral behavior - on the measurement and analysis of utilities" Electoral Studies (25) 423-446.

Fowler, F. J. (1995). Improving survey questions: Design and evaluation. (Read chapter 4: Some general rules for designing good survey instruments).

Franklin (1991) "Getting out the Vote: Social Structure and the Mobilization of Partisanship in the 1989 European Elections" European Journal of Political Research (19) 129-148 .

Franklin (2004) "Estimation Issues" from Voter Turnout and the Dynamics of Electoral Competition in Established Democracies Since 1945. New York: Cambridge University Press, pp. 126-9.

Franklin and Wlezien (2002) The Future of Election Studies. Oxford: Pergamon.

Grofman, Bernard and Peter van Roozendaal (1997) 'Modelling Cabinet Durability and Termination', British Journal of Political Science 27(3), 419-51.

Hainmueller and Kern (2008) "Incumbency as a source of spillover effects in mixed electoral systems: evidence from a regression-discontinuity design" Electoral Studies in press.

Holbrook, Allyson L., Melanie C. Green and Jon A. Krosnick (2003) Telephone versus face-to-face interviewing of national probability samples with long questionnaires: Comparison of respondent satisficing and social desirability response bias, Public Opinion Quarterly 67(1), 79-125.

Inglehart (1985) "Aggregate stability and individual-level flux in mass belief systems: the level of analysis paradox" American Political Science Review

Inglehart and Abramson (1994) ÒEconomic Security and Value ChangeÓ American Political Science Review

Jacoby (1991) Data Theory and Dimensional Analysis

Kennedy (2003) A Guide to Econometrics Chapters 1-5 (not the small print)

King, Keohane and Verba (1994) Designing Social Inquiry

King. Honaker, Joseph and Scheve (2001) "Analyzing Incomplete Data: An Alternative Algorithm for Multiple Imputation," American Political Science Review, 95:1, pp. 49-69.

King, Gary, Michael Tomz and Wittenberg Jason (2000) 'Making the Most of Statistical Analyses: Improving Interpretation and Presentation', American Journal of Political Science 44(2), 347-61.

de Leeuw, Edith D. (2005) To mix or not to mix data collection modes in surveys, Journal of Official Statistics 21(2), 233-55.

Manski (1995) Identification Problems in the Social Sciences

Newton and Norris (2000) "Confidence in Public Institutions" in Pharr and Putnam, eds. 52-73.

Norton, Edward C., Hua Wang and Chunrong Ai (2004) 'Computing Interaction Effects and Standard Errors in Logit and Probit Models', The Stata Journal 4(2), 154-67.

Peracchi, Franco (2002) "The European Community Household Panel: A Review" Empirical  Economics 27:63-90.

Poole, Keith (1998a) "Recovering a Basic Space From a Set of Issue Scales", American Journal of Political Science, 42: 954-993.

Poole, Keith (1998b) "How to use the black box" unpublished manuscript.

Raghunathan (2004) "What Do We Do With Missing Data? Some Options for Analysis of Incomplete Data" Annual Review of Public Health, 25: 99-117.

Schafer, Joseph and John Grabam (2002) "Missing data: our view of the state of the art" Psychological Methods 7:147-177.

Sekhon, J. (2004) The 2004 Florida optical voting machine controversy: a causal analysis using matching. Harvard Unpublished working paper.

Semyonov, Moshe, Rebeca Raijaman and Anastasia Gorodzeisky, "ForeignersÕ impact on european societies public views and perceptions in a cross-national comparative perspective", International Journal of Comparative Sociology 49:5-29.

Sianese, B. (2006) Propensity score matching. ESRC Research Methods Festival unpublished paper.

Steenbergen, Marco and Bradford Jones (2002) "Modeling Multilevel Data Structures" American Journal of Political Science 46:218-237.

Van Evra (1997) Guide to Methods for Students of Political Science

de Vaus (2002) Surveys in social research

Watson, Dorothy (2003) "Sample Attrition between Waves 1 and 5 in the European Community Household Panel", European Sociological Review 19: 361-78.

Willer, D., and Walker, H., 2007. Building Experiments: Testing Social Theory. Stanford: Stanford University Press.

Wretman, Jan (2005) Allowing nonresponse may give you a better estimate, Journal of Obnoxious Statistics 1(1), 76-78.

Wright, G., Schaffner, B., 2002. "The influence of party: evidence from the state legislatures". American Political Science Review. Vol 96.2: 367-379. (Available on jstor).