Setting Statistical Assumptions When Setting Up Projects

Setting defaults in a project

The default statistical assumptions in Q:

• Are conservative. In particular, the False Discovery Rate Correction is automatically applied on all tables.
• Focus on using statistical testing as a way of highlighting results that are in some sense exceptions, and displays these results using colors and arrows (e.g., )

The defaults can be modified using Edit > Project Options > Customize > Statistical Assumptions. Common modifications are:

• Changing Show significance to Compare columns (see Ways of Showing Statistical Significance).
• Changing the Overall significance level from 0.05 to 0.10 (i.e., from the 95% level of confidence to 90%).
• Set the Minimal sample size for testing to 30.
• Changing both Multiple comparison correction settings to None.
• Either:
  • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), which means that Q will only use UPPERCASE letters when showing significance.
  • In the Significance levels and appearance grid, copy one of the uppercase Column letters fields (i.e., A,B,C,...,) and paste it over the top of each of the lowercase fields (i.e., a,b,c,...,), except those for the rows with a Cutoff p-value of 0.005, 0.01 and 0.05. In conjunction with changing the Overall significance level, this means that Q shows lowercase letters for results significance at the 0.10 level and UPPERCASE for 0.10 and higher.

If wanting to replicate results from other programs it will be necessary to adjust more options (see Results Are Different to those from Another Program). However, some caution should be undertaken when modifying the settings, as Q's default statistical tests are, in general, safer than those in more traditional survey programs (in particular, they deal better with weighting and edge cases, such as if testing against a cell containing 0% or 100%).

Applying the defaults to other projects

Modifications that are made to Statistical Assumptions can be saved as Project Templates, and automatically applied to new projects (see Project Templates).

Over-riding Q's statistical testing

Rules can be used to modify how Q performs statistical tests. For example, Significance Testing in Tables - Independent Samples Column Means and Proportions Tests can be used to perform tests that assume samples are independent, even when the underlying data are not independent. Considerable caution should be undertaken when performing using rules to over-ride Q's inbuilt statistical tests:

• The rules are less sophisticated than the in-built tests (i.e., they have fewer checks and balances).
• Rules are essentially modifications of the core functionality of Q and can have unintended consequences (e.g., if inadvertently applied to the wrong table, or, if using them in ways not anticipated when they were created).

See also

• Setting Up Your Data in Q for an overview of how to set up data in Q
• File Formats Supported by Q
• Manipulating data files
• Basic Workflow For Checking and Cleaning a Project
• Constructing Variables to Make Analysis Easy
• Weighting (Sample balancing)
• Advanced Data Tidying
• Exporting, Copying and Printing
• Updating Projects with New or Revised Data
• Converting Other Files Types into SPSS or CSV Data Files)

Further reading: SPSS Alternatives