25 People Who Influenced My Thinking On Assessment

A few months ago, I had the privilege to share some of my ideas about assessment with an audience that is very data savvy.  Without realizing it, teachers are the product of interactions with present colleagues, former teachers and past students.  Beyond the ideas I've shared, I hope this presentation inspires you to think about who has had the greatest impact in your teaching.

In the original talk for MISA, I shared stories highlighting 50 people who have influenced my thinking on the topic of assessment and evaluation. I'm really happy with the look of the original slidedeck which is available below, but it's difficult to argue with the nice editing job the creatives did with the presentation.

Assessment in a networked world from Rodd Lucier

UnPlugd 2012: Assessing the Unmeasurable

This summer, 39 intrepid educators embarked on a journey that brought them from the skyscrapers of downtown Toronto, to the shores of Kawawaymog Lake in the region of Algonquin Park. With the promise of a wide range of engaging experiences, UnPlug’d 2012 brought together co-learners from Australia, USA, and Canada to share stories about what matters most in education and to collaboratively author a collection of letters that will be published later this fall.

When it comes to digesting a good meal, a sense of satiation triggers positive feelings. In a similar way, participants engaged in our unique unplugged learning adventure, left UnPlug’d with a feeling of fullness, refueled by deep conversation, nourished by rich natural meals, and refreshed through recreation amid the splendor of Ontario’s near north. While we can measure the success of our event by pointing to the product that we produced, many of the hidden benefits our event, are intangible and immeasurable.

How can you summarize the effectiveness of a collaborative writing venture that brings to life the narrative experiences of a diverse group of co-authors? How can one measure the value of a journey in a private rail car filled with kindred spirits? How do you put a value on the achievement of emboldened colleagues who succeeded in their cross-lake swim to an iconic tree on a nearby island? How might you assess the experience of once distant friends sharing their passion for running by connecting for a morning run amidst the wonder of Algonquin region? How can anyone measure the magic of an evening paddle to a sauna, swim and campfire, when capped by the shooting stars of the Perseid meteor showers? Is it even worth considering what it means to hold in your hands a guitar that is made up of historic artifacts from across Canada or what it feels like to hear that same guitar played at a campfire or at a private indoor gathering?

While educators had to take up the invitation to participate in UnPlug’d 2012, it was through generous sponsorships and donations from a number of Canadian and US EdTech companies, that we were able to set the conditions for a rich and memorable event. We are indebted to many partners who supported our mission to dig into our collective memory, to share experiences with distant colleagues, and to edit and publish our most impassioned ideals.

UnPlugd’d participants continue to talk, share, and collaborate by leveraging social media tools, and by gathering for face-to-face meetings and professional development events. Through open conversations on Twitter, and through deeper reflection on personal blogs, participants continue to reflect on their experiences, to consider adaptations to their professional practice, and to lead by learning in public. As our multimedia publications near completion, the dividends from UnPlug’d 2012 are sure to impact connected teacher-learners from Canada and beyond. We invite you to consider coming along for the ride at unplugd.ca.

Image credits: Rodd Lucier, Lisa Neale, Todd Lucier, 

Many more images are available in the UnPlugd12 Photo Pool
This article was originally written to appear in the MindShare Learning Report.

Why I Love the Game

This past week, I had the great pleasure of taking my 8 year old onto a real golf course for the first time. Over the past few summers, he has developed a swing on the range, and has learned some golf etiquette at the local pitch and putt; but this was his first chance to tee it up for real. As we worked our way around the course, I found myself reflecting on the way golf is naturally differentiated for players of varying skills. By the time we'd finished, I had 9 holes worth of reflections to share.

Hole #1 Differentiated Challenge
I usually play the blues, hitting my tee shots from tees just in front of those reserved for professional golfers. Students on the golf team at our school, usually play from the whites; my wife starts at the reds; and the new golfer in the family, played his first round from the yellow tees. For each hole, the game of golf is differentiated, allowing player with varying levels of experience, to choose an apt challenge. I found myself thinking that school could be a much more engaging place if we could provide custom starting points for each of our students.

Hole #2 Modern Tools
While design and innovation lead to the introduction of new clubs and new learning technologies every year, the goals of both golf and school remain relatively unchanged. Many golfers find comfort in well-worn tools, like a familiar wedge or putter that just feels right; while teachers and students may be comforted in the familiarity of pencil or chalk. Those who ignore innovations in club design or developments in learning technologies, may struggle to produce their best work. It remains my opinion that hickory shafts and chalk, belong in the same place... the museum archives. We've got more effective tools to leverage.

Hole #3 Acing the Test
Through a combination of luck and skill, I was rewarded with my only hole-in-one on May 26, 1990. It was the 17th hole, a par four at Oakwood G. & C.C. where my tournament ace, keyed my foursome's victory. I remember many details from the experience, including the fact that I had to delay hitting my shot until two young golfers walked through our fairway; that I used an orange Top Flight golf ball on the tee; that my 5 wood drew in a right-to-left arc, landed on the green and rolled into the hole. Which leads to the question: What do you remember most vividly about the last test that you aced? The perfect performance on a written test may be more common, but is it far less memorable. How critical can either ace be, when you realize that given the opportunity to replay a hole you've mastered, or re-take a previously aced test, most of us are unlikely to again realize such perfection.

Hole #4 Practice as an End in Itself
As in learning, there is work to be done if you are to achieve to your best potential on tournament day. Whether practice takes place on the range where different aspects of the game are practiced in isolation; or on the course where you practice skills in the context of the game, most players find enjoyment in the practice. And the enjoyment happens in spite of the fact that many players, spend significant time on the weakest parts of their game. Knowing that every student in a classroom has a unique skill set, I'm left to wonder: Why it is common for every student to practice their way through a common set of experiences? How might the teacher discover which types of practice would be most apt for a given individual? Can choice, context, and varied practice lead individuals to enjoy the practice of academics?

Hole #5 We Play By Rules
The rules in golf may seem unfair at times, but wherever possible, you 'play it as it lies'. It's a game that relies upon the honesty of the competitors, even when the rules don't seem to make sense. In the world of school, many students look for ways to bend the rules or to outright ignore those that may seem unjust. The parallel seems to be that both school and golf tend to outlaw tools that make the 'game' too easy. Golf balls that go too straight off the tee, or those that travel too far in the air, or clubs that provide the advantage of extra spin are deemed to be illegal equipment. In the classroom, calculators were once seen to provide an unfair advantage; but today, tools like Wikipedia are deemed untrustworthy, while smart phones that provide access to the 'sum of human knowledge', are banned from exam rooms.

Hole #6 Handicaps Level the Playing Field
In golf, once you've been involved in enough 'assessments', your performance becomes predictable. The resulting golf handicap gives each competitor an equal opportunity to win an event. In a handicapped golf tournament, one has to turn in a performance that is better than his or her normal performance, in order to be rewarded. In contrast, handicaps in the classroom appear as challenges that result in an imbalance. Adaptive technologies, varied learning strategies, and universal designs may offset a learner's identified disability, but most of the time, learners are assessed using identical performance scales. As a result, the education system tends to reward the same kids over and over again, just for doing that they've been successful at doing throughout their school careers. As more and more educators embrace differentiated assessment, the practice may be seen as one to reward learners with 'assessment handicaps'.

Hole #7 The Team 'Scramble'
Occasionally, golf and school offer opportunities to learn and play as the member of a team. In golf, the scramble tournament allows a team of competitors to take advantage of the best shots of colleagues. Whether a long ball specialist or an expert putter, being a part of a foursome is most rewarding when your teammates brings different talents to the course. In the classroom, the best parallel I can think of, is when rich performance tasks allow learners to play unique roles in designing solutions to compelling problems. At their very best, both learning and golf are social experiences that bring out the best in the participants.

Hole #8 Data Driven Assessment
In golf, knowing exact yardages to fairway bunkers, water hazards, and pin placements, gives the player a tremendous advantage in making appropriate club selection. Whether using a laser range-finder, a GPS tool, or an iOS solution, players with the right information, are far more likely to make wise decisions on the course. In the classroom, it is the pre-assessment that offers a teacher similarly useful data. Knowing what your students know, and what they need to know, the informed teacher is more likely to plan an appropriate and productive range of learning activities. So, why is it that educators are far more likely to give tests only at the end of a unit of study? Experienced teachers know that gauging the strengths and weaknesses of their students, allows them to play 'target golf'.

Hole #9 What Did You Score?
Ultimately, golf is a game against yourself, and perfection is unattainable. Just as report cards attempt commonly boil down and individual's achievement to a number, so too does your scorecard . Whether or not you have rich stories to share about amazing experiences on the course or in the classroom, that final grade how success is ultimately measured. Yet, after every round, we take the time to celebrate our on course adventures with fellow competitors. Wouldn't it be wonderful if we paid more attention to the affective experiences of learners? We commonly share assessment data, and report our findings to parents, but what about sharing the anecdotes that make learning in your classroom more than 'a good walk, spoiled'?

Photo Credits: danperry.com, kazamatsuri, VancityAllie, jc_091447

Who Took the Easy Way Out?

Cheating... on an exam?!

After years of giving exams to students, an instructor at University of Central Florida, was shocked to discover that students cheat on exams.

Is it really all that hard to imagine that students would take the easy way out, when that appears to be exactly what the professor did? Instead of creating a novel and authentic way to assess student learning, Richard Quinn used his 20 years of teaching experience, to draw his exam from the question bank supplied by a textbook company. Instead of sharing his own work with students, he administered an exam created by a publisher, leaving me to wonder: Did the test acknowledge the source for the exam questions?

When I study at an institute of higher learning, I prefer to take courses that are not the same year after year; courses that recognize the realities of today's hyper-connected world; courses that don't place such a huge emphasis on a written exam to demonstrate the stickiness of course content.

In watching the 'lesson' below, I can't help but sense the emotional vibrations from the teacher: disappointment, frustration... disillusionment. As the audience for this 'lecture' learns that statistical variations, forensic analysis, and data tracking have narrowed the pool of suspected cheaters to about a third of the class, I'd have been just as interested to read the faces and body language of the 400 students.



The rant appears to have led to 200 confessions. Self-identifying cheaters will be allowed to complete the course and graduate, provided they take a four hour course in ethics. I can't help but wonder if/when any teachers at the school will see themselves as culpable.

At the close, we learn that the instructor was too distraught to load the slides for Chapter 8 and Chapter 9. Whether or not the slidedecks were also provided by the publisher, Mr. Quinn's closing comments indicate that he has much to learn:

"The days of finding a new way to cheat the system, are over!"

I suspect that the system will need far more than a new exam in order to make learning relevant, and cheat-proof. What do you think?

For more food for thought, visit a few true stories of students from the University of Windsor.

Classrooms of Tomorrow

Yesterday, I had the opportunity to lead the academic staff of Regina Mundi College, through a morning of learning, reflecting and designing. With the staff still getting to know me through the first ten weeks of school, it was a unique opportunity to revel in my passions for emerging technologies along with local colleagues.

The Lesson
I began the workshop by highlighting Ten Trends sure to affect teaching and learning in the years to come:

Ten Trends Rmc

View more documents from Rodd Lucier.

Peppered throughout the morning, were unexpected bonuses that I like to call 'soft returns': resources that can immediately impact classroom teaching & learning. Tools you might take for granted such as Wordle, WolframAlpha or Mr. Robbo's blog, are a few of the examples that rewarded participants for engaging with the 'formal lesson'.

The Culminating Task:
To demonstrate an understanding of highlighted emerging trends, teachers were asked to consider the following questions:

Which trends are most likely to impact your classroom?
How will your classroom change?
What tools will you need to address these trends?
What will you need to learn?
What will you need to un-learn?

Rather than answering the questions in a journal, or writing a test, teachers were grouped and tasked with designing a 'Classroom of Tomorrow'. To highlight the potential of collaborative design tasks, members of each group were invited to take on roles with entry points differentiated to meet the needs of a diverse 'classroom':

Team Leader: ensure all have input
Architect: sketch the classroom
P.R. Specialist: communicate design decisions
Espionage Expert: sample the ideas of other teams
C.F.O.: calculate a budget for proposed design
Timer: reinforce timelines for design and presentation

Showing What You Know:
Participants came to realize that it was possible to demonstrate an understanding of the 'course content' through an engaging activity. If such a task were to be used for assessment purposes, teachers were reminded that each individual should be required to explain the group's design choices, in the context of course expectations. It was also emphasized that any rubric for such a task, might de-emphasize the artistic presentation, in favour of a focus on design thinking and understanding.



The Wrap-up:
In completing this design task, my teaching colleagues transformed into students before my eyes. Letting their true colours show, we staff members unwittingly took on the characteristics of just about every type of student you can imagine. The animation and willing participation of my colleagues was beyond my greatest expectations.

To conclude our morning, an eloquent Ontario teenager, and 'wired' high school student, Patrick Quinton-Brown joined us via Skype. Patrick is a student trustee with the Durham District School Board, and Director of Communications with OSTA-AECO whom I met six days earlier while sitting on a panel at the People For Education annual conference in Toronto. Having heard him speak about the role technology played in his life, and about how the restrictive classroom environment often impeded his learning, I knew he'd be the perfect guest to wrap our morning.

In arranging an appearance, 'live' from his home school, Patrick was instrumental in helping me to put an exclamation point on the need to transform our classrooms. Indeed, my colleagues were able to experience first-hand, the Classroom of Tomorrow!

Photo Credit: Rodd Lucier

The Origins of Grading

Have you ever wondered why we as educators have to assign grades to students and their work? Take a listen, and learn how William Farish has had a lasting impact on the work we do with learners.



Related Links:
William Farish: The World's Most Famous Lazy Teacher
William Farish (Wikipedia)
Grading Systems (Wikipedia)
Senior Wrangler (Wikipedia)

Is grading a practice that has more benefits or more detriments?

Image Credit: superia85

The Open Exam

Does your course have a culminating exam to assess student knowledge and understanding? Would you like to make such a test a more authentic endeavor?

This 6 minute podcast invites teachers to consider offering 'Open Exams'...



Have I made an effective case?
Do you have experiences with open tests that you can share?

For more of my thinking about exams, visit "When Are Exams Over?".

Photo Credit: Night Owl City

When Are Exams Over?

This evening, I learned that the exams my daughters wrote last week, will not be returned. For the sake of maintaining the security of the exams, which I presume will be used again and again, exam-writers will never see how their responses were graded.

Am I alone in shaking my head at this reality?

Why I Dislike Exams

Too often, exams test 'facts' which any student would be able to find via Google.

Too often, exams are completed only with paper and pencil.

Too often, exam results comprise a significant proportion of a student's final grade.

Too often, exams are most stressful to those least likely to benefit from the experience.

Too often, exams are ineffective in their attempts to assess learning that took place throughout an entire course.

Too often, the knowledge and understanding demonstrated through exams, cannot be repeated in days following the exam.

Too often, exams test language skills, rather than course specific expectations.

Too often, final grades are the only feedback students get on their exam experience.

Too often, exams measure knowledge and understanding, completely devoid of meaningful context.

I've heard the argument that we need exams to prepare students for university... Yet, in some places policies exist to exempt the very students who might one day have to write post-secondary exams. Are exemptions a way to reward high achievers, or a way to penalize low achievers?

As observant teachers grow in confidence, assessing the learning of their students through rich day-to-day experiences, perhaps the authority long held by written tests, will be gradually diminish...

Photo Credit: Jon Oakley; COCOEN

Exams as Circumstantial Evidence

While our high school students prepare for final exams, I'm left to ponder how these snapshots came to yield so much weight.

In a discussion with colleagues earlier today, I suggested that the written test is best considered as 'circumstantial evidence' of student learning. More often than not, and exam indicates little more than which student studied, and to what degree they paid attention to the right material.

Rather than rely on the information provided on a written test, I await the day that teachers rely on anecdotal evidence as the best measure of student learning. Certainly an educator's first-hand observations taken in documentation of rich performance tasks, or culminating projects would be more telling than first draft handwritten exam responses. Teacher-student discussions in the midst of such experiences allow all types of learners to 'show what they know'.

It was back in April when members of the blogosphere publicly debated in response to Will Richardson's query "When Are We Going to Stop Giving Kids Tests That They Can Cheat On?" The shortest blog post I've read this year, generated great discussion on this topic.

As experts in learning, it is teachers themselves who need to rally against the traditional exam. While it will take a special teacher to even open this conversation in a staff room, the discussion following Will's post is a terrific place to begin the dialogue.


For a humourous example of how I survived oral exams in university, you might be interested in re-visiting the "Tale of the Tell-tale Toes" episode of the Teacher 2.0 podcast. New today: Choice Matters.

Photo Credit: ccarlstead; beatsrhymesnlife

Expelled for 'Studying' on Facebook

First year computer engineering student Chris Avenir is facing expulsion after a teacher at Ryerson University discovered a Facebook group set up to share tips and answers to an online testing system.

Will this action result in a culture of fear among students who participate in study groups online?

I agree with Jesse Hirsh who is quoted in on online Canwest article:

"The online culture is outpacing the curriculum and education system," said Jesse Hirsh, who has been studying social websites for more than a decade. "These students are being smart and using the Internet the way that it should be used. This is the future of education."

I find it ironic that The student is studying 'Computer Engineering" and the administration at Ryerson is upset that he and well over 100 of his classmates are leveraging technology to share strategies on beating the technology-based assessment system used at the Toronto school.

In stark contrast to traditional testing and exams, "Studio Learning" first caught my attention in November of 2004, when I read an article on 'New Jersey Institute Of Technology Pioneers New Way To Teach Engineers'. At the time, I was surprised that tens of thousands of dollars were being granted to a 'novel teaching method' that I'd been employing with elementary students for years! Read bioengineering associate professor Richard Foulds' take on the effectiveness of his approach.

Rather than come down on students who have designed creative solutions to 'beat the system'... It's time that universities join others in developing rich assessment opportunities that engage students in solving authentic problems.

Photo Credit: Grant Hutchinson
Teacher 2.0 Pocast: Cheating on Facebook

Bum in the Seat?

Recently, in a small group e-learning workshop, a colleague of mine was reinforcing the need for teachers to use what she called "bum in the seat" assessment. The intent of this 'pictorial' cue was to reinforce for teachers the fact that in evaluating student learning, we must be sure that students are demonstrating knowledge & understanding, thinking, communication, and application in the presence of the teacher.

My problem with the term "bum in the seat", is that it leads educators to envision the student meeting curriculum expectations independently, while sitting at a desk. Exacerbating the problem is the fact that educators predominantly rely upon the written word as the valid way for students to demonstrate their learning, in spite of the fact that modern curriculum documents require that teachers assess expectations that point to skills like demonstrate, collect, identify, predict, compare, interpret, create, express, generate, publish, establish...

Curriculum documents in Ontario also ask teachers to engage community resources and to provide authentic audiences for student work. Students, thinking both logically and creatively; and producing content in written, oral and multimedia forms, are required to produce wide variety of cross-curricular products. Undertaking such a wide range of prescribed learning experiences is not possible with students sitting in desks.

As educators, let's get in the habit of assessing what students DO, instead of what students WRITE. Take time to observe with a critical eye, and to document what students accomplish against established standards. It would help if provincial assessments undertaken by the Education Quality and Accountability Office (EQAO) in Ontario, would harness something other than written tests in assessing reading; but don't let it stop you from modeling the types of assessment practices that are clearly stated in the front matter within our curriculum documents.


Note: This image is a model of a desk designed and constructed at school by students as part of a rich performance task. You can't demonstrate this type of thinking and learning by writing a test!

70 Years Ago... are we there yet?

A few weeks back, a colleague shared his mother's report card with me. From Nova Scotia in the 1937-38 school year, the introduction on the report demonstrates much wisdom. With thanks to Vince and his mother, I invite you to click the image for the enlarged view; I'll let these 70 year old words serve my purpose today.


An interesting aside to this report, is that the teacher and the student have continued to maintain a collegial friendship that still endures! Though an anecdote, this type of relationship speaks to powerfully of the qualitative learning experience.