Non-Standard Errors

In statistics, samples are drawn from a population in data-generating process (DGP). Standard errors measure the uncertainty sample estimates of parameters. science, evidence is generated to test hypotheses an evidence-generating (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams six on same sample. find sizeable, par with standard Their size (i) co-varies only weakly team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) underestimated by participants.