Brief Guidelines for Monte Carlo Studies

(Brief Guidelines for Monte Carlo Studies)

1. A Monte Carlo study is an experiment whose methodology should be explained clearly enough that another researcher can replicate the results of the study. The goals of the study should motivate the design of the experiment.
2. The design of the experiment (e.g., factors, levels, etc.) dictates the statistical analysis. Efficient use of resources usually requires comparing methods with the same generated data. This blocking factor should be taken into account in the analysis.
3. Interpreted results should be justified by proper statistical comparisons. Presenting a table of means and standard errors may enable the reader to do a conservative, approximate t-test to test equality of means (assuming positive correlation due to blocking), but a good exposition shows the details for at least one example. Details of the analysis should be included for more complex comparisons.
4. Graphs can present more information in less space than tables and should be easier to interpret. Both tables and graphs should be as self-explanatory as possible. All tables and graphs should include some indication of their accuracy; usually a summary of standard errors of entries or plotted points (average, range, etc.) will suffice. The number of significant digits presented in tables should be supported by their accuracy and should rarely exceed 2 or 3.