All experiments and trials should be reproducible! No questions about it. A reader should be able to read your materials and methods section of your report and/or your journal publications and be able to replicate your experiment or trial. Think of this as the recipe for your trial. You can read and follow recipe instructions, as you should also be able to follow the materials and methods section of a trial and replicate it.
But this isn’t always true in publications. There are a number of thoughts as to why this may happen:
- word limitations – should the words be used in the materials and methods, or should we reserve them for our Results and Discussions?
- uncomfortable talking about the statistical analysis – lack of knowledge or too much knowledge?
- lack of confidence – so lets skip it, nobody is going to read it?
Examples – can you replicate the following studies?
Read through the Materials and Methods section of these papers and decide whether you have enough information to replicate the study.
- Mist Blowing versus Other Methods of Foliar Spraying for Hardwood Control (1968)
- Seedling year management of Alfalfa-grass mixtures established without a companion crop (1969)
- Leaf and stem nutritive value of timothy cultivars differing in maturity (1996)
Goals of writing:
Our readers should be able to determine:
- that the analysis fits the objective stated for your experiment or trial
- that your research methodology and data collection processes match the analyses
- that your data management processes ensure data quality
- Objectives clearly stated
- Materials and Methods:
- Identify your target population
- State your treatment effects
- Identify how you selected your experimental units and your sample size
- State your experimental design, be sure to include number of replicates
- Identify your analysis variables:
- Are you using your raw data?
- Are you using derived variables? If so, what are they?
- Are you using transformed variables? If so, how and why were they transformed?
- State your statistical model – don’t be shy about this!
- State your method of analysis – if you say something like “model was produced using stepwise regression” – this is a flag for a statistical reviewer!! Provide your reader with clear directions
- State your p-value
- State the software that was used.
Additional items to consider and add to your description
- If you have several trials, consider combining them rather than reporting several single trials.
- Remember that you must account for your error variances if they are heterogenous.
- If the data was transformed for your statistical analysis, back transform your results for presentation in the publication
- When your Null hypothesis is NOT rejected (in other words, when there are no differences observed), report the observed p-value. As an example p=0.35
- No Significant difference is correct and accepted in publications. Do NOT say the difference was non-significant
- Report means with their standard errors.
Can you replicate the following trial?