# Which t-test to use for a two-group pre- post-test design?

I have data on 26 participants (13 from computing and remaining 13 from non computing) who have participated in my research. Each participant is treated with a lab module (Hands on Robotics Session). Now each participant will be evaluated using a rubric on scale of 1 to 4. This experiment has both pre and post test.

For my research i want to evaluate the following questions:

### Research question 1

• Null Hypothesis: students do not learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.
• Alternate Hypothesis: Students learn about computational thinking (programming basics and algorithmic thinking) with the help of robotics.

To evaluate the above question, the categories i will be considering are Plan, Implementation and Knowledge gained on a scale of Excellent, Good, Fair and Poor.

I think i should use DEPENDENT T-TEST FOR PAIRED SAMPLES. Am i correct?

### Research question 2:

• Null Hypothesis: Participants background (computing and non computing) has no effect in learning about algorithmic thinking with help of robotics
• Alternate Hypothesis:Participants background (computing and non computing) has an effect in learning about algorithmic thinking with help of robotics

I will also evaluate question 2 on a scale of Excellent, Good, Fair and Poor, but with respect to background.

### My Statistical Question

Which T test should I use for each of my research questions?

-

– chl♦ Jun 20 '12 at 7:59

## 3 Answers

Since you are interested in measuring an increase pre- and post-test, it seems to me that you should use a paired test.

An issue here is that your variables are likely non Gaussian since they are among 4 categories. If your sample size is big (very roughly larger than 50), then it is no big deal. Otherwise, I would use the Wilcoxon signed rank test which is a non parametric analog of the paired t-test.

-

2
very roughly is right. What if $P( {\rm category \ 1} ) = 1 - 10^{-200}?$ ;) – Macro Jun 18 '12 at 23:40

1
Aha (+1), then I am f... But how likely is that? Close to $10^{-200}$ I'd say :-) – gui11aume Jun 18 '12 at 23:45

1
It's also worth noting that this would also require an assumption that the categories comprise an interval scale. It also seems (unless I misread) that, in question 1, the role of the null and alternative are reversed from the way they normally are in a paired $t$-test; the null appears to be that the module does help. Finally, question 2 appears to require a two independent samples test, where the paired differences are the outcome variables. – Macro Jun 18 '12 at 23:51

1
I do not subscribe to using the Wilcoxon signed rank test alternative. If you do a simulation study, you can still show the power of the t-test is good --comprable to the signed rank test-- in moderately skewed distributions. If the researchers are interested in testing for a geometric difference in responses, then a log transformation would be justified. – AdamO Jun 18 '12 at 23:54
@marco: Thanks for pointing out that my formation for null and alternate was wrong and thanks for your suggestion on the second question. – Dumb_Shock Jun 19 '12 at 0:59
show 1 more comment

I think you are having problems with the nature and elaboration of your alternative hipotesis. Your investigation/reserach hypotesis (and all the theoretical model associated with it) is your alternative hipothesis. Since the variable is clearly ordinal, i´d prefer a Wilcoxon rank test. But anyway i suggest to work out a little bit more your hipothesis.

-

gui11aume beat me to it. He said exactly what I had in mind when I read the question. Since scaled measurements are not going to fit a normal error distribution very well a nonparametric paired test is the best way to go in my view and for that I would have recommended the Wilcoxon signed rank test. On the other hand if you were comparing scores by domains where many responses are summed, normality is not a bad assumption and the t test is fairly robust anyway. So in that situation a paired t test might be okay.

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 3, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": false}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9346959590911865, "perplexity_flag": "middle"}