Posts Tagged ‘Testing’

The Importance of Testing

September 17, 2015

Complaints are being received from teachers that testing is interfering with the education of students because they have to teach to the test.  There are two points to be made here.  First of all, testing is necessary to measure whether anything is being learned.  The second point is that testing rather than interfere with learning, can enhance learning.  These points were effectively made in a Scientific American Article that can be found at
http://www.scientificamerican.com/article/researchers-find-that-frequent-tests-can-boost-learning/

An example of one of these effective teaching techniques was provided in the article.  The teacher posted a multiple choice question on a smartboard screen.  The students clicked in their answers which were posted on the bottom of the smart board screen.  So the students needed to retrieve information to make their selections.  The teacher received feedback on the knowledge of the class, and was able to provide feedback for the wrong answers.  When every student provides the correct answer, the class members raise their hands and wiggle their fingers in unison, which is an exuberant gesture that they call “spirit fingers.”

There is ample evidence from research in cognitive psychology that retrieval practice increases learning.  Whenever we retrieve a memory, the memory representation changes, and its mental representation becomes stronger, more stable, and more accessible.  If material is simply reread, this retrieval practice does not occur.  Retrieval strengthens and has additional benefits noted by cognitive psychologist Jeffrey Karpicke.  He notes that as our memory is necessarily selective, the usefulness of a fact or idea—as demonstrated by how often we have reason to recall it—makes a sound basis for selection.   He said that “our minds are sensitive to the likelihood that we’ll need knowledge at a future time, and if we retrieve a piece of information now, there’s a good chance that we’ll need it again.  The process of retrieving a memory alters that memory in anticipation of demands we may encounter in the future.”

Karpicke argues that retrieving is the principal way learning happens, “Recalling information we’re already stored in memory is a more powerful learning event that storing that information in the first place.  Retrieval is ultimately the process that makes new memories stick.”  Not only does retrieval practice help students remember the specific information they retrieved, it also improves retention for related material that was not directly tested.  When we are sifting through our mind for the particular piece of information we are trying to recollect, we call up associated memories and in doing so strengthen them as well.

I remember from my college day the yellow marked sections whenever I had a previously owned text.  I made it a point to never rely upon those yellow marked sections.  It was my guess that when studying for a test, the previous user simply reread the highlighted section.  I never did that.   I always tried to recall the gist of the material, and then I checked my recall.  If just rereading highlighted sections was done, my guess is that the best result would be a C.  My goal was an A, and I often received them.

There are hundreds of studies hat have demonstrated retrieval practice is better than virtually any other method of teaching, including doing concept maps.

Research using fMRI has shown that calling up information from memory versus simply restudying it, produces higher levels of activity in particular areas of the brain. These regions are associated with the consolidation, or stabilization, of memories and with the generation of cues that makes memory readily accessible for later recall.  Research has demonstrated that the more active these regions are during an initial learning session, the more successful is recall weeks or months later.

So this testing versus learning complaint is a pseudo issue.  It is not an issue of teaching to the test.  Rather it is a matter of developing teaching plans that require students to actively recall information rather than to simply reread material that will likely be on the  test.  This is a pseudo complaint.  If done properly it is a win win issue.

However, according to the Scientific American article there is a feature of standardized tests that prevents them from being used more effectively as  occasions for learning, and that is that the questions they ask tend to be of a superficial natures, which tends to lead to superficial learning.  There is a tool called Webb’s Depth of Knowledge, created by Norman Webb, a senior scientist at the Wisconsin Center for Education Research.  This tool identifies four levels of mental rigor:
DOK1 (simple recall)
DOK2 (application of skills and concepts)
DOK3 (reasoning and inference)
DOK4 (extended planning and investigation)
Most questions on state tests were DOK1 or DOK2.

So rather than complain about testing, the complaints should be on the DOK required on the tests.  The deeper the depth of knowledge, the better the test, which leads to more effective learning.

© Douglas Griffith and healthymemory.wordpress.com, 2015. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.

Risk Intelligence

November 11, 2014

Risk Intelligence: How to Live with Uncertainty is the title of a book by Dylan Evans. This title should immediately grab you attention as we must live with uncertainty. Actually our cognitive processes lead us to believe that we live in a world with much more certainty than there actually is. The first chapter begins with a quote by Thomas Jefferson: “He who knows best, best knows how little he knows.” And there is also the human tendency to think we know more than we know. I’ve found that in continuing my education through the Ph.D plus decades of additional experience and learning that I know much less than I thought I knew when I graduated from high school! I’ve discovered vast new landscapes of ignorance.

Given the ubiquity of risk, we might well as is there such a thing as risk intelligence and can it be measured? Given the title of the book, you will probability not be surprised to learn that the answer to both questions is yes, Evans book contains a Risk Intelligence (RQ)Test or you can go online to projectionpoint.com. There are also expert RQ tests targeted for specific areas of expertise.

A fear I had when I first encountered RQ tests was whether they would result in debates and research on the issue of whether RQ is innate or learned. Fortunately, RQ can be developed through education and training. Perhaps a good first step would be to read the book. Moreover, it has the additional merits of being interesting and entertaining.

© Douglas Griffith and healthymemory.wordpress.com, 2014. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.

To Get It Right, Get It Wrong First!

March 22, 2010

A recent article, “The Pluses of Getting it Wrong” by Henry Roediger and Bridgid Finn has profound implications for students, in particular, and education, in general.1 They present research that makes the case not only for difficult tests in school, but also for testing before any instruction takes place. Students who make an unsuccessful attempt to answer a test question before receiving the correct answer the material remember the material better than if they simply study the information. One can certainly ask, how can this be?

One possibility is that asking questions before studying the material focuses the students’ attention on critical concepts. This could be beneficial, but might not the same benefit be achieved by allowing students to preview the questions without having to answer them? This issue was addressed by comparing three groups in a study. One group, which you might call the standard control group, was allowed to study the material in advance of the first test. A second group previewed the questions before studying the material. The third group not only saw the questions, but was also required to attempt to answer them. All groups were allowed to study the material again and were given a final test.

The third group, the one that not only previewed the test questions, but were also required to attempt to answer them, performed the best. The group that previewed the questions came in second, and the standard traditional group performed the poorest. So testing in advance not only facilitates the identification of key concepts, but the attempt to answer the questions provides additional benefit. This might activate memory circuits that facilitate learning.

A previous blog post “The Benefits of Testing” also cited the work of Roediger. Testing before studying resulted in better recall. Roediger has used his results and the results of others to modify his teaching. Every class begins with a test on the material of the day. When this test is completed he proceeds to cover the material. This results in better retention, long term retention, in particular.

When or whether the educational establishment acts upon these findings remains to be seen. However, the industrious student can use these results to improve the effectiveness of her own study. If there are questions in the back of a chapter, attempt to answer them before reading the chapter. If there are no questions, then read headings and try to construct questions based on the headings and then attempt to answer them before reading the chapter. Then read the chapter.

1Roediger, H. L. III & Finn, B. (2010). The Pluses of Getting It Wrong, Scientific American Mind, March/April, 39-41.

© Douglas Griffith and healthymemory.wordpress.com, 2010. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.

The Benefits of Testing

February 9, 2010

The distinguished psychologist Roddy Roediger was invited to give the keynote address for the 50th Anniversary meeting of the Psychonomic Society. The title of the address is “The Critical Role of Retrieval in Enhancing Long-Term Memory: From the Laboratory to the Classroom.” A streaming video of this keynote address came be found at

psychonomic.org/annual_meeting.html

Roediger begins his address by stating the implicit bargain that is usually made between teachers and students. Students don’t like taking tests and teachers don’t like giving them. Not only does the teacher need to construct the test, but she also needs to grade them, a time consuming task. So testing and exams are usually kept to a minimum. Moreover, testing is used to measure learning and the assumption has been that little or no learning takes place during testing. Roediger’s address should disabuse anyone of this notion.

Roediger presents a series of studies that vary the respective number of study and test trials. Little difference was observed during learning. But on retention tests that were given two days later, retention was solely a function of the number of test trials. He presents a series of studies varying the materials and the nature of the tests, but they all basically hammer home the same theme. Not only does learning occur during testing, but more learning occurs during testing than during study. One study done with a group of middle schoolers showed that repeated testing had the result of raising the average grade from a C+ to an A-.

It is interesting to examine the subjective ratings of students and test participants. They feel that they are learning more during study than during testing. When students keep re-reading highlighted material in a textbook, they get the filling that they really know the material and their confidence goes up. However, when a student tries to recall material from memory and fails, confidence is lowered. Yet the looking up of the material that was forgotten is more beneficial and the student has a more realistic appraisal of what is known and what needs to be studied. In the end, this latter experience is more beneficial.

The actual attempt to remember information forces the person to access the correct retrieval routes to that information. If the information is found, then that retrieval route is strengthened. When it is not found, the information is restudied and the retrieval route relaid. More effort is involved in testing than simply studying material, and there is evidence that this increased effort is also beneficial.

So what are the lessons to be learned here? First of all, cramming is not recommended. Even if you learn enough to pass the test, the information will quickly be lost. So its availability on a final exam or later in life is questionable.

Secondly, test yourself and recited the material frequently. This testing should be even more effective if spread out over time.

And what, if any, are the implications for the education system? Break the silent bargain between teachers and students and test more frequently. Roediger and his colleagues have taken to the practice of having a ten minute test at the end of every lecture. This practice not only forces students to keep up, but it also leads to better lifelong learning.

© Douglas Griffith and healthymemory.wordpress.com, 2010. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.

Transactive Memory1 and Educational Testing

December 30, 2009

The most common criterion for learning in our educational system is whether you can remember certain information. Sometimes recognition memory is tested, as in true false or multiple choice tests. At other times recall memory is tested, as in fill in the blank or essay tests. These tests are carried up the educational hierarchy all the way to comprehensive written tests for Ph.D. qualifying exams. Open book exams are the exception and not the rule. And the use of crib notes can get a student into serious trouble.

Educators have tended to regard the proliferation of transactive memory (the internet, for example) as a threat to education. They fret about students plagiarizing text from the internet and their inability to recognize or identify this plagiarism. This blog posting will argue that the abundance and availability of transactive memory should be regarded as an opportunity rather than a threat.

When I taught introductory or lower level courses in college, I placed heavy reliance on multiple choice tests. The main considerations here were time and resources. Given an abundance of students and no teaching assistants, practical considerations dictate multiple choice tests. When I needed to construct make up tests for students who had missed scheduled tests for legitimate reasons, I made up essay tests. It is not practical to construct multiple choice tests for one or several students. Usually I was appalled when I graded these tests. Part of the problem often was poor composition skills, but the conclusion I drew was that the students had but the flimsiest grasp of the material. So students seemed to be learning much less than what I had inferred from their multiple choice test performance.

Now consider this new type of test in today’s world of ubiquitous transactive memory. Students would arrive at the exam with their laptops and would be given full internet interactivity. There would be no restriction on any materials they had prepared for the exam. They would be given a problem, perhaps more than one. And it is possible that these questions would be taken from a set of potential exam questions that the students had been given in advance. They would be required to answer the problem or problems to the best of their ability using all the resources at hand. The premise underlying this type of test is that the critical test of knowledge is how well you can use it rather than whether you can recall it by rote. Using the knowledge of others is not a problem as long as credit is given. Failure to provide sources would be heavily penalized.

What do you think of this new type of test for the 21st Century?

1Transactive memory, as presented in previous blog postings, is memory external to our personal selves. So this is memory resident in our fellow humans and in the vast expanses of technology, for example libraries and the internet.

© Douglas Griffith and healthymemory.wordpress.com, 2009. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.

What Do We Need to Know

November 22, 2009

This is the question we ask whenever we encounter new information, be it an article, a website, or whatever. If it is of no interest the answer is simple, no we do not need to know this and we proceed no further. The question becomes more difficult when the answer is “yes.” Then the question becomes “how well do we need to know it?” If it is of extreme importance or interest, then one decision might be to commit it, or the gist of it, to memory on the spot. Rarely do we encounter something of this importance or interest, but if we did commit it to memory we would need to devote some attention to its maintenance. Otherwise it could become lost from or inaccessible to memory. So it might also be a good idea to store it in some sort of transactive memory, either to save the file or to tag or bookmark it. If it is also of interest to an acquaintance you could also tell them about the item and why it is so important to you.

In most cases you would either save, tag, or bookmark the item. Should you fail to do so, at a later time you might recall there was something of interest or importance, but be unable to find it. So you need to take recourse to transactive memory frequently or you will be in a state of having a wealth of memories, but being unable to access it.

Essentially, you need to decide what level of effort the information affords. You cannot remember everything and to a large extent what you do remember depends on the amount of effort you expend. Although you could commit a great deal of information to memory, you would do this at the cost of remaining ignorant of other information (to say nothing of the free time lost). Some idiot savants commit enormous amounts of information to memory (remember Dustin Hoffman in the movie “Rain Man?”), but these people are often socially inept. So you want to learn thing, have social relationships, and enjoy life. And you do this by relying on transactive memory

This is an interesting question because it is asking what does it mean to “know” something? In most tests taken at school the standard is whether the information can be retrieved from memory. Sometimes, as in multiple choice or true false tests, the criterion is whether the information can be recognized. In fill in the blank or essay questions, the criterion is whether the information can be recalled. Usually one of the requirements for a Ph.D. that needs to be passed before you can do a dissertation is a comprehensive exam. Usually this exam is written and is an enormous closed book test on the relevant material in the subject in which you are trying to earn a Ph.D. That was true in my case in which I had to answer question without the aid of external supports (no lifelines!)

A reasonable question is whether this is the only criterion for knowing. Suppose you know where to find the specific material. So you know what the material is about and where it fits into some general scheme of knowledge. Does this not also imply that you have some knowledge about a topic? Does not having information in transactive memory and being able to access it also count as knowledge?

© Douglas Griffith and healthymemory.wordpress.com, 2009. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Douglas Griffith and healthymemory.wordpress.com with appropriate and specific direction to the original content.