How do you choose a reproducibility metric?
Mesnard and Barba produced a brilliant reproducibility paper, showing how hard it can be to get many details correct. In contrast to some previous reproducibility efforts they don't have a single number, or curve, that they want to reproduce in advance. Is this the best approach?
more ...