Publish your scientific materials when you publish your paper

Last week, for the first time, I published a paper for which I also uploaded all the electronic materials needed to replicate the study (for a moderately experienced vision scientist). You can now read about the fact that not only motion causes the silencing illusion of Suchow and Alvarez (I know you'll all be fascinated). But then you can also download the PsychoPy scripts to run the study, as well as analyse the original data and generate the plots. It may be surprising to the non-scientists out there that this is newsworthy but, in fact, almost nobody does this yet. I know! Unbelievable, right? Although scientists are mostly not shy about their findings, most are very shy about providing all the guts of their research, warts and all.

Some time ago I posted on the idea that we could do with an easy-to-use repository to which we could upload materials from experiments in psychology. There are numerous benefits for science in general, expanded on in the post above; we can create direct replications of other studies, we can spend more time thinking about scientific issues and less time rewriting basic stimulus code. I didn't express in that post that there's also a massive benefit for the publishing scientist, who is shy and territorial and doesn't want to lose the "competitive advantage". The perceived competitive advantage is that in this experimental topic (s)he's already written experiment code that nobody else has. But I believe it's massively outweighed by the benefit that when we publish our code we encourage more people to read our work and base their study on ours (to a geeky scientist that's the biggest complement you can pay). Let me put it another way: if your study was easy to program then somebody else can do it in no time (so no loss in giving them the code) and if the study was hard to program then they might never run the extension of it (so it's really important to give them the code so that they do).

Happily, unbeknownst to me when I wrote that post, the necessary repository was being created and you can now join up and use it at OpenScienceFramework.org. It's free, it's easy, it's permanent and it's easy on the eye. But it provides some great additional features. You can use it, before publishing your work, to share materials with your collaborators in a secure (private) repository and makes the materials available publicly later if you wish. The repository then has version control built-in so you can track changes to the materials, without needing to know about the underlying technologies. You can also use OpenScienceFramework to register in advance (again, privately) your intention to conduct a study and the expected outcomes in order to demonstrate which of your conclusions/analyses stem from genuinely a priori hypotheses.

Basically, this is a great resource that behavioural scientists should all be looking seriously. Many thanks to Brian Nosek, Jeff Spies and the rest of the OpenScienceFramework team. It is still in beta, so the creators are still taking feedback and adding features.

I'll be writing all my experiment code from now with the expectation that it will be published on OSF, and therefore writing it carefully with clearer-than-ever notes. For me this recent study is the first of (hopefully) many where you will be able to download the entire set of materials and data for the publication.