Recently, one of the students here at U. Sydney and I had the frustrating experience of trying to reproduce a numerical result from a paper, but it just wasn’t working. The code used by the authors was regrettably not made publicly available, so once we were fairly sure that our code was correct, we didn’t know how to resolve the discrepancy. Luckily, in our small community, I knew the authors personally and we were able to figure out why the results didn’t match up. But as code becomes a larger and larger part of scientific projects, these sorts of problems will increase in frequency and severity.
What can we do about it?
A team of very smart computer scientists have come together and written the science code manifesto. It is short and sweet; the whole thing boils down to five simple principles of publishing code:
- Code
- All source code written specifically to process data for a published paper must be available to the reviewers and readers of the paper.
- Copyright
- The copyright ownership and license of any released source code must be clearly stated.
- Citation
- Researchers who use or adapt science source code in their research must credit the code’s creators in resulting publications.
- Credit
- Software contributions must be included in systems of scientific assessment, credit, and recognition.
- Curation
- Source code must remain available, linked to related materials, for the useful lifetime of the publication.
If you support this, and you want to help contribute to the solution, then please go and endorse the manifesto. Even more importantly, practice the five C’s the next time you publish a paper!
Also, arxiv.org lets you attach code as an ancillary file. Many journals do too.
Oh, I didn’t realize that you could do this separately. In my recent tomography paper, I just bundled the code with the latex source in a zip file.
I think that’s almost the same. I’ve never done it before, but it looks like the only difference is that extra files are in an “anc” subdirectory and then that creates a link on the abstract page saying “ancillary files for arXiv:xxxx.xxxx”.
Meanwhile, the commercial publishing industry is also innovating, with ads appearing in J. Math. Phys. articles. (Although to be fair, they’ve hosted supplementary files for awhile too.)