A New Study says New Studies Cause Anxiety

Trusted leaders in their field, if they are to remain trusted, must demonstrate they are on top of the latest trends and research in their fields. This is, of course, no easy task given the volume of insights and research generated by companies, universities, think tanks and others. (In just the United States, it is estimated that more than 2.5 million articles get published in medical, scientific, technical and other journals.) Further complicating matters is that it’s not uncommon for research to lead to different, and at times conflicting, results.

Under these trying, and sometimes confusing, conditions, how does one stay up-to-date with—and confident in—the research that shapes their points of view? Here are some helpful tips.

Always Remain Skeptical

Just because an alleged fact, data point or insight is said to be backed up by research doesn’t mean that the research was done well and in keeping with accepted best practices. Nor does it mean that the conclusions drawn by it flow sensibly¾and defensibly¾from the data. In other words, approach every piece of research and its conclusion with some degree of skepticism.

Consider the Source

You should ramp up your skepticism when there’s reasons to believe that the organization conducting or sponsoring the research has a vested interest in the data arriving at one conclusion versus the other, as when an organization advocating for coffee consumption publishes results that claim¾surprise!¾coffee is good for you. Research you can trust will reveal any potential conflicts of interest and, very importantly, the source of funding. When biased, well-funded organizations are behind the research, you should spend extra time in the study and its result before awarding it your trust.

Research conducted “in house” typically deserves more scrutiny than that conducted by outside, independent research organizations. The best independent research companies—such as the UK’s Nielsen or Japan’s Intage—refuse to have their good names associated with research that doesn’t adhere to the highest standards or that publishes questionable results.

Can You Follow It?

Good research, even on technical matters, should contain clear information about their methodology, their calculations, and other relevant research. If this information isn’t made available, or leaves you confused, you likely want to check with other sources. Remember, a really encouraging sign of a study that you can trust is one that makes it clear how it could be repeated allowing another researcher to do just that and confirm (or challenge) the results.

The Power of the Peer Review

High-quality research is almost always peer reviewed, meaning that highly qualified, objective professionals reviewed the research to ensure that it’s capable of withstanding intense scrutiny. In fact, the journals with the best reputations will only publish peer-reviewed research.

Beware of the Wow

If an article is touting some truly novel, unexpected, jaw-dropping conclusion, invest extra effort in validating the study’s legitimacy. Sometimes, research results get boiled down to attention-getting headlines (e.g., “Reading Actually Makes You Dumber”) as a means to generate attention (and website clicks), even though the actual study and its finding are, at closer inspection, more nuanced.

Realize the Limits of General Interest Media

Speaking of hyperbolic headlines, one must cautiously approach research summaries covered in general interest media. These reporters are often not researchers nor scientists, so they rely on summaries provided by the study’s investigators or underwriters, who may bring certain biases to the research. In addition, not all reporters are equipped with the training necessary to scrutinize a study’s methodology and its results. When you can, look to the trade and scientific journals for their take on the research. That’s not to say that the reporting is inaccurate or intentionally false—just that, when distilled for a broad audience the necessary nuance to any study’s conclusion may be diluted or misunderstood.

Whether by design or by accident, faulty research is far from a rare occurrence. The tips above will help you confidently decide what research to embrace and what research to run from.