Choices in quantitative data analysis

Possible choices researchers might made in data analysis. An excerpt from Table 1 of Wichert et al. (2016)

  • A1: Choosing between different options of dealing with incomplete or missing data on ad hoc grounds
  • A2: Specifying pre-processing of data (e.g., cleaning, normalization, smoothing, motion correction) in an ad hoc manner
  • A3: Deciding how to deal with violations of statistical assumptions in an ad hoc manner
  • A4: Deciding on how to deal with outliers in an ad hoc manner
  • A5: Selecting the dependent variable out of several alternative measures of the same construct
  • A6: Trying out different ways to score the chosen primary dependent variable
  • A7: Selecting another construct as the primary outcome
  • A8: Selecting independent variables out of a set of manipulated independent variables
  • A9: Operationalizing manipulated independent variables in different ways (e.g., by discarding or combining levels of factors)
  • A10: Choosing to include different measured variables as covariates, independent variables, mediators, or moderators
  • A11: Operationalizing non-manipulated independent variables in different ways
  • A12: Using alternative inclusion and exclusion criteria got selecting participants in analyses
  • A13: Choosing between different statistical models
  • A14: Choosing the estimation method, software package, and computation of SEs
  • A15: Choosing inference criteria (e.g., Bayes factors, alpha level, sidedness of the test, corrections for multiple testing)