What should a graduate course in experimental design cover?

I have been asked to propose a course in experimental design for advanced graduate students in agronomy and ecology. I have never taken such a course, and was surprised to find that the course might be more aptly named “Beyond one-way ANOVA”, and that it covers material that I learned in an advanced graduate course on statistics for agricultural field experiments (e.g. RCBD, Latin Squares, Contrasts, repeated measurements, and covariates). Perhaps I am confused by the name “Experimental Design” rather than “Analysis of Experimental Results”.

I have some ideas about what such a course should contain and would appreciate feedback on how this might be integrated into a statistics curriculum that meets the needs of the students while presenting modern alternatives to named lists of designs and their associated tests.

For example, I can’t imagine teaching students to use linear and quadratic contrasts with ANOVA that enforces categorization of continuous variables when I could teach them to compare regression models with linear and quadratic functions. In the second case, they would also learn how to deal with factors that are not experimentally defined discrete values. If anything, I might compare the two approaches.

If I were to teach a course in “Experimental Design” I would really like to emphasize fundamental concepts that are independent of the statistical model being applied, and that would translate more broadly to other problems. This would enable students more flexibility to use modern statistical approaches.

Some of the relevant concepts that do not appear to be covered in the existing course include:

  • hierarchical and mixed models (of which I understand ANOVA and relatives as one example)
  • model comparison (e.g. to replace contrasts)
  • using spatial models instead of blocks as ‘factors’
  • replication, randomization, and IID
  • differences among hypothesis testing, p-hacking, and pattern recognition.
  • power analysis through simulation (e.g. recovery of parameters from simulated data sets),
  • pre-registration,
  • use of prior knowledge from published studies and scientific principles.

Are there any courses that currently take such an approach? Any texts books with such a focus?


Here is a list of some books that I like and which would be good material for such a course:

  • David Cox: Planning of Experiments, Wiley classics, 1992. This is non-mathematical, but not easy! A profound discussion of basic concepts behind design.

  • D. R. Cox & Nancy Reid: The Theory of the Design of Experiments, Chapman & Hall, 2000. More mathematical, but still with focus on basic concepts

  • Rosemary A. Bailey: Design of Comparative Experiments, Cambridge UP, 2008. From the foreword: “My philosophy is that you should not choose an experimental design from a list of named designs. Rather, you should think about all aspects of the current experiment, and then decide on how to put them together appropriately …”.

  • George Casella: Statistical Design, Springer, 2008. Another book looking at old topics with fresh eyes!

  • You could do worse than look at George E. P. Box, J Stuart Hunter and William G. Hunter: Statistics for Experimenters: Design, Innovation and Discovery (second edition, Wiley, 2005) for inspiration.

I would avoid older books looking like a catalog of named designs, and go for one of the above based on fundamental principles. One such book I would avoid is the popular (why?) Douglas C. Montgomery: Design and Analysis of Experiments.

 EDIT 2017   

Another topic which could be included is optimal experimental design, with concepts such as D-optimal designs or A-optimal designs. There is now a plethora of books, so difficult to advice, some possibilities:
Optimal Experimental Design with R
Optimal Crossover Designs
Optimal Experimental Design for Non-Linear Models: Theory and Applications
Optimal Design of Experiments: A Case Study Approach

There is a lot of development in this area in R, so have a look at https://CRAN.R-project.org/view=ExperimentalDesign

Source : Link , Question Author : Abe , Answer Author : Community

Leave a Comment