Searching for and Evaluating Scholarly Articles

 If you are looking for scholarly articles on a topic, Google has a dedicated search engine, You can set up alerts to learn of new articles on topics or authors of interest. Although one can expect and will usually receive, higher quality information from authors extremely knowledgeable in their respective areas of interest, there are certain realities that should be understood. Refereed articles are articles that have been reviewed by several authors knowledgeable in the topic prior to publication. Prior to the internet era this refereeing was needed because paper publication was costly and journals needed to be selective. Typically, there were large delays between the submission of the article, its acceptance, and its eventual publication. There were also journals that did not use referees, that would publish articles for a fee.

With the advent of the internet, the cost of publishing articles and the time to publish articles have been drastically reduced. Yet the archaic artifacts from the print era persist. An author can disseminate an article as soon as she deems it worthy. Why delay the dissemination of information? Some argue that refereeing is still necessary. I’ve participated in this review process both as an author and also as a reviewer. I’ve had articles accepted, and I’ve had articles that I thought worthy that were rejected, but subsequently published by another journal and another review process. As a reviewer I’ve seen articles that I thought worthy of publication be rejected. I’ve also seen articles accepted that, while adequately done, made a questionable contribution to substantive knowledge. Published statistics on the review process are not impressive. Statistics on agreement among reviewers have typically been low. There is also a bar that the editor needs to set to assure that accepted articles have enough allowable pages to accommodate them. I find it odd that academics tend to be impressed by high rejection rates rather than forlorn about research that has gone unpublished. Academics also are keen on refereed journals. Personally, I can quickly ascertain whether an article is worth my time and I don’t need some reviewers editing or censoring the information that is available to me. I think one of the primary reasons academics are keen on refereed journals is that they can use the number of publications in refereed journals in making decision about whom should be awarded tenure. Otherwise, they might actually have to read articles written by tenure candidates.

I see little justification for the traditional institutions for publication. Research can be disseminated quickly via the internet and judgments made regarding the value of the research and on whether it should be ignored or put to good use. The problem is that big moneyed interests are involved. They are the publishers and the professional associations that sell publications. Typically authors and reviewers are not paid. Their efforts are pro bono. The editor might be given an honorarium, but the amounts of small. But the journals and professional books are expensive. And there is no need for them to be. They provide little of added value.

I am able to get online access to journals published by the professional organizations to which I belong for nominal rates. Others are typically quite expensive, as I’m sure some of you can attest. There is a quality online journal that anyone can access for free, PloS 1, Public Library of Science,, this is a referred journal. We should be able to access any research done with taxpayer support online for free. And I, personally, would be willing to forgo the referring requirement.

You should also be wary of biases in different academic disciplines. I’ll provide a couple of examples from my discipline, psychology. When I was a graduate student, there was a large controversy as to whether humans could learn to control their own autonomic processes, heart rate for example. Now it was well known and well documented that Buddhists proficient in meditation could do so. However, this did not constitute the appropriate evidence hard-nosed psychologists required. They wanted to see it done by some student fulfilling a course requirement in a one or two hour psychology experiment.

Shortly after I received my Ph.D and began working at the Army Research Institute for the Behavioral and Social Sciences, I tried to replicate an experiment that I had read in the psychological literature. Although I was able to do so, I was only able to replicate the finding in the group that had General Technical (IQ) scores comparable to college students. The vast majority of reseach in cognitive psychology is based on research done with college students. Although one of the fundamental requirements for generalizing statistical results is that the population to which one is generalizing have been represented in the sample participating in the experiment. I have yet to see a finding in a psychology with the caveat that the results should be restricted to those representated in the statistical sample, college students, for example. I am a working statistician and I am constantly amazed how statistical requirements vary from discipline to discipline when the underlying statistics and their assumptions remain unchanged.

© Douglas Griffith and, 2012. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and with appropriate and specific direction to the original content.

Tags: , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: