Deeper Learning By (Re) Design(5/30/2017)

Materials for a workshop for North Central College Faculty and Staff
Tom Angelo
Clinical Professor of Educational Innovation & Research, University of North Carolina

Values Aairmation
On the lines below, jot down 2-3 of your core educational/professional values that motivate you to invest your time and energy in activities like this curriculum redesign effort.
___________________________
___________________________
Goal Ranking and Matching
What specifically do you hope to learn/gain through this brief workshop?
List questions you hope to answer through your participation

Your learning Goals/Burning Questions for this workshop
___________________________
___________________________

Some Key Terms and Concepts that Might Be of Use...

• Backward design
• Constructive alignment
• Formative and assumptive assessment
• Intended learning outcomes and teaching objectives
•  Bus Test, Parrot Test, and Parking Lot Test
• Cognitive load
• The Dance Floor and the balcony
• Novice-expert differences
• Deliberate practice

Seven levers for deeper learning (research based guidelines for effective teaching and learning)
Research suggests that virtually all students can learn more- and more deeply-when we help them to

• Be aware of their own relevant prior knowlege, beliefs, preconceptions, and values
• Set and maintain realistically high and personally meaningful learning goals and expectations for academic success
• Learn how to learn effectively-given their own individual histories, talents, preferences, and goals-so they become increasingly self-directed and independent learners
• Understand the criteria, standards, and methods used in assessing and evaluating their learning and get useful, timely feedback on their performance against those standards
• Seek and find connections to and real-world applications of concepts and skills they are larning in course
• Collaborate regularly and effectively with other learners and with teachers to achieve meaningful, shared learning goals

Six Dimensions of higher learning outcomes

• Factual learning-learning What (Level 1):Learning facts and principles
• Conceptual learning- learning What (Level 2): learning concepts and theories
• Procedural Learning-learning How: learning skills and procedures
• Conditional Learning-learning When and Where: learning applications
• Cognitive Learning-learning How to Lear: learning to direct and manage one's own lifelong learning
• Reflective Learning-learning Why (and Why not): Developing self-knowledge, cultural awareness, ethics, etc.

Designing Courses for deeper learning: 10 clarifying questions

1. what is the overall purpose-or what are the overall aims- of your course?
2. Where does this course fit into the program degree curriculum?
3. For whom is this course designed? (What  are the students' relevant characteristics? needs?)
4. What specifically, should students demonstrated they know and can do by course's end?
5. What standards will be used to evaluate and grade students' learning?
6. How will their learning be assessed against those standards?
7. What specific content will be taught and assessed
8. What will motivate students to learn deeply and well?
9. What kinds of work and how much work must students do to learn and succeed?
10. What kinds of work and how much work must teachers-and others-do to facilitate learning?

"Backward" Course (Re) Design- Sequential Steps in an ideal process: develop or revise

1. Degree/Program-level learning outcomes
2. Course-level intended learning outcomes 
3. Standards for assessing and grading performance
4.  Summative (Graded) assessments
5. Diagnostic and formative assignments
6. Learning activities and assignments
7.  The content-topics, chapters, readings, websites, etc
8. Teaching Strategies, Techniques & tools
9. Course & teaching evaluation/program review

Bloom's Cognitive Domain Taxonomy

1. Remember
2. Understand
3. Apply
4. Analyze
5. Evaluate
6. Create

• Give an example of ...
• Why do...change? 
• What causes the earth's seasons to change (explain how it works)
• when it is winter in..what season is it in..?
• Where on Earth would you predict the greatest variation occurs? why?
• What contribution, if any, will global warming likely make to seasonal change? Explain your reasoning
• What would likely happen to seasonal change in ...if the earth change to..
• If the earth orbit moves ..what difference, if any, would you predit that increased distance would make to seasonal change
• If you were teaching how and why the seasons change to a 5-year-old, how would you expain it?

Bondy's clinical performance (skills) rating scale:
thinking of skills you already possess and your current level of competence, confidence and independence in those skills: identify at least one of your skills in which you are currently "dependent", another skills in which you are marginal, and so on all the way up.

• Expert instructor: extremely safe& proficient- capable of assessing, demonstrating, instructing & supporting learners in level 1-5
• Independent: safe: very proficient-requires no supportive cues
• Supervised: requires only occasional supportive cues
• Assisted: requires frequent verbal and occasional physical cues
• Marginal: safe only when supervised
• Dependent: unsafe&unaware-unable to demonstrate skill-quires continuous monitoring, support & direction-verb & physical cues

-->

Clarifying Intended Learning Outcomes:

On completion of this course, you should be able to demonstrate:

·      Enhanced knowledge of the basic tenets of phrenology and its history

·      Understanding of what was best practice in

·      Appreciation of the relationship of phrenology to modern

When you have completed this course, you should be able to

·      List the six …

·      Identify, locate, and explain functions

·      Explain the significance of ..

·      Identify and summarize the key

 To successfully compete this course, you must demonstrate you can

·      Correlate locate and label..

·      Summarize your analyses of both in writing in the following hour, and achieve at least 85% agreement with expert analyses

·      Prepare a character analysis and related career and marriage advice for a third subject

·      Develop 20-minute talk on your case study

Developing intended learning outcome Statements:

Draft learning outcome (write this only after you’ve answered the questions below):

·      Who?

·      Does/will do what?

·      To/for whom

·      By when

·      Where

·      How

·      How well

·      Why

Developing Intended Learning Outcomes:

First-draft intended learning outcome (ILO)

Teachers in this course will design effective lessons

Second-draft ILO

Who	Each teacher in this course
Will do what	Will design a lesson to pre-assess, give feedback on, teach and post-assess student’ understanding of an important and potentially problematic concept
For whom	The elementary or secondary students in their placement classrooms
When	Between semester weeks four and six
Where	In her or his placement classroom
How	Through an annotated lesson design, related assessments and assignments
How well	At the “meets expectations” level or above on the assignment grading rubric as assessed by the course instructor- and by an expert school teacher
Why?	In order to demonstrate an appropriate level of skill in effective, research –based lesson design

Third-draft ILO

Week 4-6, each teacher in this course will present an annotated lesson designed to pre-assess, give feedback, teach, and post-assess their placement students’ understanding of an important and problematic concept, in order to demonstrate an appropriate level of skill in effective, research-based lesson design

Standard:…

Fourth-draft ILO

All teachers who successfully complete this course will have met or exceeded expectations in:

Designing an effective, annotated, research-based lesson

 A detailed assessment/grading rubric

Macroeconomics Essay Grading Grid

Assignment: write a well-structured, enlightened critical essay about current economic conditions that demonstrates command of existing economic knowledge appropriate interpretation and application of the knowledge, and demonstrates appropriate use of data and argumentation to support well-reasoned policy recommendations

Basic questions:

·      What is the current macroeconomic situation in the U.S.?

·      What is the likely prognosis for the next 12 to 24 months?

·       What are your economic policy recommendations?

-->

Criterion	Distinguish	Exceeds expectations	Meets all expectations	Meets some expectations	Unsatisfactory	Missing or fails minimum req.
Executive summary is effective & concise							Structure
Introduction clearly lays out a roadmap for the paper and places the information in context
Body Addresses all the basic questions, includes the argumentation and data
Conclusion provides summary and closure
Command of existing economic knowledge							Knowledge
Use of terms, theories, and data
Informed judgment demonstrated by selection of terms, theories and data (shown by the exclusion irrelevant and inclusion of)
Argument flows logically so that early statements lay the foundation for later statements and the reader is guided through the arguments							Argumentation
Appropriate application of theory is used to make argument; clearly links theory and data to conclusions
Arguments are persuasive focuses on key points, does not wonder, uses no unnecessary verbiage
Data used is reliable, valid, and pertinent; it provides effective support; no superficial information or tangential data muddies the argument
Presentation is neat and professional, all visuals used are well labeled, clear, and effective conveying information better than words; text contains no error and is easy to read& understand							Striving for excellence and creativity
Creativity: the paper clearly holds the imprint of the author. Original through is demonstrated by innovative organization, the integration of concepts and ideas, the use of new approaches, the novel use of visuals, or

 

Sample Self-and Peer Assessment and grading rubric

High pass/A	Fully meets all the criteria for Pass/B listed below and at least 5 of the 5 criteria below ·      Integrating relevant current research and practice literature ·      Integrating effective strategies and techniques ·      Demonstrates notable creativity and/or innovative thinking in content, execution and/or presentation ·      Contributes new information/ideas/concepts that, when shared, will contributes significantly to the learning, health, well-being and/or success of the intended “target audience” ·      Has already been accepted for inclusion and use in a course, patient ed., continuing ed. Program, etc.
Pass/B	Presents a well-polished, highly readable document that could be shared, as is, with and benefit that target audience Presents a well-integrated final document that contains both a text (prose) explanation and elaboration of the SLED and a details agenda, running sheet or storyboard-complete with timing-to serve as a guide for successful implementation of the SLED Provides a compete concise overview and introduction to the SLED, explaining: who the target audiences are; what the overall purpose is why and how it is likely to be significant and beneficial to that audience(s); where, when, and how it could be implemented; what the key design assumptions are; and, what the author's motivations were in proposing and developing it Provides an appropriate number of well framed, leveled and assessable intended learning outcomes Explains what will motivate the target audiences to participate and how the audience will be engaged and activated-including appropriate strategies and techniques Provides a teaching an learning plan for presenting the SLE, including appropriate strategies and techniques-all clearly aligned with ILOs. Integrates a minimum of five research-based, demonstrably effective, appropriately referenced teaching, learning, assessment and feedback srategies overall-aligned with the ILO Explains succinctly how each strategy and technique aligns with the relevant ILO and why it is appropriate and likely to be effective in promoting that ILO Explains succinctly  what the appropriate criteria would be fore SLED success.effectiveness and how the degree of success.effectiveness in anactual implementation might be deterined or measured Includes a brief refelction on the lessons learned from the SLED exercise and from feedback on it that are likely to be relevant and useful in futuer coursework, experienctial learning, and pracice provides relevant references cites and references correctly, AMA citation sytler Is written clearly, cohearernly and appropriately with no significant errors in grammar, spelling or suage.
Low Pass	displays some non=critical shortcomings in quality and execution. Essentially, Essentially, a Low Pass or C-level SLED would require at least one additional revision cycle to meet the Pass B
Fail	Fail to respond full to all the requirements because it is incomplete or, if complete, because some elements are of poor.unacceptable quality

Authentic Formative Feedback: some discussion points

Why give learners feedback

·      To improve performance& academic success

·      To increases interest& motivation to learn

·      To illuminate and undermine misconceptions and biases

·      To promote self-assessment& self-regulation

·      To develop independence as lifelong learners

To use Feedback Well, learners need MOM

·      Motivation: reasons to use the feedback

·      Opportunities: for safe, guided, productive practice

·      Means-knowledge& skills required for self-improvement, and metacognitive skills are critical in this regard

The order in which we give feedback matters

1.     Good news: what was done well

2.     Bad news: what still needs improvement

3.     Options: what can be done to imrove it

4.     Plans: what the learner intends to do

5.     Commitments: what both parties agree to do, how, to what standards, and by when

Effective, authentic feedback for deeper learning…

·      Feeds forward: focuses on improving future performance

·      Is iterative: part of a robust, regular process

·      Is consequential to an intrinsically valued by the learner

·      Comes from multiple, credible and trusted sources

·      Focuses on outcomes/ behaviors; not on the person or their qualities

·      Is criteria and standards referenced, not norm-referenced

·      Is specific and limited to what maters most

·      Provides sufficient evidence to support judgments and decisions made

·      Can be implemented by the learner, given skills and time available

·      Offers some choices regarding follow up

GIFT: Gathering informal feedback on teaching

1 please give two or three examples of specific things your instructor does that help you learn effectively in this course

At the end of each example, please indicte whether that specific thing is:

(1)  very important; (2) somewhat..(3)not very,

2. Please suggest two or three specific, practical and constructive changes your instructor could make to help you learn more effectively in this course

3. Please suggest two or three specific, practical and constructive change you and/or your classmates could make to help you learn more effectively

4. Any further comments?

APPLICATION CARD

INTERSTING OR PROMISING     SOME POSSIBLE APPLICATION OF THESE TO MY WORK           
IDEAS/TECHNIQUES

WORKSHOP FEEDBACK FORM

1. Over all, the value of what I learned in this workshop is 5 (very high)..1(very low)
2. Overall, the quality of this workshop is 5 (very high)..1(very low)
3. Overall, I rate this workshop leader's effectiveness as 5 (very high)..1(very low)

• Comments on this workshop

1. Which two or three specific aspects of this workshop were most useful/helpful/interesting
2. Which specific aspects could have been improved
3. What possible follow-up, if any, from NCC might be helpful