Funders, publishers, research institutions and many other groups are increasingly keen that researchers make more of their data more open. There are some very good reasons for doing this, but many researchers have legitimate concerns that must be dealt with before they can be convinced. This is the first in what I hope will be a series of posts exploring arguments against sharing data.
“We really want to share our data more widely, but we’re worried that it’s going to give the crackpots more opportunity to pick holes in our findings.”
A PhD student asked me something like this recently, and it’s representative of some very real concerns for a lot of researchers. While I answered the question, I didn’t feel satisfied with my response, so I wanted to unpack it a bit more in preparation for next time.
It seems to me that there are three parts to this. No-one likes to:
- Have their time wasted
- Be wrongfully and unfairly discredited
- Have genuine flaws found in their work
Having genuine errors challenged is a very useful thing, but spurious challenges (i.e. those with no valid basis) can be a stressful time-sink. Such challenges may be made by someone with an interest in seeing you (or your results) discredited; they may also be made by someone who simply fails to understand a key concept of your research1. Either way, they’re a nuisance and rightly to be avoided.
Perhaps the scariest aspect of this is the possibility that your critics might actually be on to something. No-one really enjoys finding out that they’ve made a mistake, and we naturally tend to avoid situations where an error we didn’t know was there might be brought to light.
If all this is so, why should you share your data? Ultimately, there will always be crackpots, or at least people with an ax to grind. Publishing your data won’t change this, but it will add weight to your own arguments. Firstly it says that you’re confident enough in your work to put it out there. But secondly it gives impartial readers the opportunity to verify your claims independently and come to their own judgement about any potential criticism. It’s much harder for the “crackpots” to pick holes in your work when your supporting evidence is available and the validity of your argument can be easily demonstrated.
There’s also a need to accept, and indeed seek out, valid criticism. None of us is perfect and everyone makes mistakes from time to time. When that happens it’s important to find out sooner rather than later and be ready to make corrections, learn and move on.