Experiments are commonly used in economics to test whether certain factors affect, for example, the behavior of consumers or the productivity of workers. When planning an experiment, several important decisions have to be made, such as how many subjects should participate. If this number is too small, there is a risk that the study cannot uncover an existing effect.
Another important question concerns the choice of design, whether participants take part in all experimental conditions (a so-called within-subjects design) or whether each participant takes part in only one experimental condition (between-subjects design). Both decisions imply tradeoffs. Between-subjects designs require more participants than within-subjects designs. But it is generally not known how much more participants are needed.
In a new IZA Discussion paper, Charles Bellemare, Luc Bissonnette and Sabine Kröger demonstrate how ex-ante power calculations can be conducted in a flexible way using simulations in order to compute the minimal number of participants. They show the trade-off between the choice of design and minimal number of participants needed. In their application of labor market gift giving field experiments, they find that between-subjects designs require 4 to 8 times more participants than within-subjects designs to reach the same power to detect an effect. The authors offer a STATA software package that runs the simulations and can be used to compute the minimal number of sample size needed to detect an effect for a variety of situations for different design choices.