Student Learning Outcomes
We have all heard of terms like course objectives, goals, and outcomes in association with syllabi, course planning, and accreditation. In recent years, the great "eye of Gondor" (think Tolkien) has come to rest on learning outcomes.
If we google the term, 5.6 million sites magically assemble. So do hundreds of viewpoints and definitions.
So, let's explore a bit.
"Learning outcomes are statements that specify what learners will know or be able to do as a result of a learning activity. Outcomes are usually expressed as knowledge, skills, or attitudes." Source
1. There are learning outcomes that are very general, like those from Kansas State University which lists five goal areas (Knowledge, Critical Thinking, Communication, Diversity, and Academic and Professional Integrity) at the institutional level. For each, there are learning outcomes. Read this one:
"Students will demonstrate the ability to access and interpret information, respond and adapt to changing situations, make complex decisions, solve problems, and evaluate actions."
Can you tell to which goal area this belongs? Check your guess
2. The Association of American Colleges and Universities commissioned a Liberal Education and America's Promise (LEAP) campaign to investigate 21st century skills and solidify general outcomes and ways to assess them.
LEAP was organized around Essential Learning Outcomes, which were developed with input from hundreds of colleges and universitites.
The LEAP web site offers information about High Impact Educational Practices and Authentic Assessments.
So, learning outcomes in a very general way can tell the intended result of an experience, program, or a course of study.
Look through the Institutional Outcomes for State Fair Community College in Missouri:
Here are the areas of WNMU focus:
The goals of the General Education Program at the University (the goals for student learning and development are consistent across the curriculum) are as follows:
1. Provide each student with opportunities for critical thinking and reasoning, communication of ideas and information to others, numerical analysis and decision-making, and insure personal and emotional well-being;
2. Ensure that graduates of the University possess the skills necessary to interact positively and productively in society;
3. Develop personal, social, and cultural awareness that values cultural diversity and recognizes the commonalities that bind peoples of the world;
4. Instill an appreciation for the variety of perspectives that are used to interpret the world in which we live and that provide the foundation for solving the problems that confront society.
Based on these goals, the following objectives are incorporated into the General Education Program, with emphasis placed on integrating an appreciation of cultural diversity throughout the curriculum:
a. Critical thinking;
c. Communication (written, oral, and visual);
d. Multicultural perspectives;
e Social responsibility and cooperation;
f. Literacy of all types (reading, numbers, consumerism, technology);
g. Intellectual curiosity and wonder (continued learning);
h. Environmental responsibility.
For many years, the focus on learning objectives or outcomes was on what instructors did. In traditional lesson plans, the teacher was the "doer" of lesson objectives. Today, the focus is on the learner. What will the learner be able to know or do? How will institutions know that students have successfully learned the concept?
Western Washington University talks about a paradigm shift from what teachers do to what students do.
As the assessment and accountability movements in higher education have converged on student learning as the center of the educational universe, ideas about what constitutes a high-quality education have shifted from the traditional view of what teachers provide to a practical concern for what learners actually learn, achieve, and become.
In the traditional "teacher-centered" model, the focus has been on inputs: the credentials of faculty, the topics to be presented, the sequencing of presentations, and so forth.
In the "student-centered," or "learner-centered" model, the focus is on outputs: what knowledge have students actually acquired, and what abilities have they actually developed? Implicit in the student-centered model is the idea that instructors are facilitators of learning. It is not enough to construct a syllabus and present information; the job of instructors now involves creating and sustaining an effective learning environment based on a wide range of "best practices" in teaching and learning. The fundamental role of assessment is to provide a complementary methodology for monitoring, confirming, and improving student learning.
We start here, at institutional objectives, in the journey. Those are "mountaintop" views.
These general outcomes stand on peaks and point west or south.
We can tell the direction we need to go to end up at the destination, but we don't really know the roadmap to get there or how we will know we have arrived.
Look through this presentation on learning outcomes.
Consider the institutional Student Learning Outcomes (SLOs) from Cascadia Community College in Washington. They are grouped into four areas: Think, Learn, Interact, Communicate.
These outcomes fall under the Interact umbrella.
Successful negotiation through our increasingly complex, interdependent, and global society requires both knowledge and awareness of others and enhanced interaction skills.
Let's look, for example, at the middle bullet point.
That is a noble learning outcome, since we live in a world of "complicated, dynamic and ambiguous situations."
So, how would we determine whether two seniors have met that outcome as part of their college experiences, courses, or programs of study?
We could have seen Gilda giving directions to a fellow student as they cooperated on an assignment, heard her explain a concept to a study group in front of the coffee shop, noticed that she worked on a service project with community members and other students. Can we check off that learning outcome as completed on Gilda's records?
Another student, Harvey, was argumentative with the class while working on a project, disruptive in a service learning activity with community members, and earned low peer evaluations in a group assignment. When we consider marking him incomplete on the learning outcome, he points out that the learning outcome wording does not say students had to be good at collaborating, but only that they did it. He demands a checkmark on that goal.
We can probably agree that using broad general goals and outcomes serves a purpose in planning and vision of a college or university. They can also serve a purpose when determining the goals of a department or program.
However, we need to travel down from the mountain vista to get somewhere more definite than east or south.
From institutional learning outcomes, we create and align program outcomes. Then, we specify what each department's goals and outcomes are. Finally, we focus on a course, what it will teach and how students will demonstrate learning.
In each of these areas, we have to then navigate the jungle of goals, objectives, and outcomes.
Western Washington University has these guidelines for all program learning outcomes:
First, student-centered programs are output- oriented. The primary measure of program success is what graduates actually know and are able to do.
Second, student-centered programs are competency-based. Learning objectives and learning outcomes are tied to the most important skills and knowledge in a program.
Third, learner-centered education is dedicated to continual improvement through ongoing assessment of student learning. By monitoring the effects of program changes on learning outcomes, program faculty are enabled to identify problem areas and to design improvements.
The College of Alameda in California describes the difference between course objectives and courseSLOs like this:
How Do Outcomes Differ From Objectives?
Objectives address the details in a course and are related to the specific course content that will be covered by the instructor. There will be many individual objectives for any class. This objectives list must be exhaustive enough to meet the requirements of equivalent classes [at other univerisities] so that the course will articulate. Objectives tell students what supporting skills, knowledge, and attitudes they will learn during a course that lead to mastery of the course SLOs.
Whereas, SLOs (student learning outcomes) indicate what students will be ABLE TO DO after completing the course. Typically, there will be between three and eight student learning outcomes for any particular course or program. An SLO also provides a context for learning and moves toward a means to evaluate the student's performance. The primary rule to be applied when formulating SLOs is that they must be assessable; there must be some way to measure student success in achieving those goals. Include within the SLO the criteria that must be met in order for students to demonstrate to you that they have achieved the desired outcomes.
One argument might arise: Isn't it obvious what students will learn in a College Algebra course? Can't students intuitively guess what they are required to do in an Introduction to Psychology course?
Well, yes and no.
If you are a junior student looking for a course, how would you know the distinctions between Introduction to Sociology and Social Problems? Looking at a syllabus and learning outcomes should tell you.
We know one purpose of general learning outcomes at institutional, course of study, and program levels: to point the direction from a high vantage point, from a panoramic view.
As we move down into the trip itself, we must have more specific directions to guide us. From institutional and program intentions, we must formulate very specific goals at the department and at the course level.
What purpose do these outcomes have?
Goucher College outlines main purposes and benefits of outcomes:
These are important aspects!
They communicate what the course will teach to students, parents, advisors, faculty, departments, other institutions, and accreditation organizations.
In a course, learning outcomes are the specific indicators of where the traveler will go, what road and mile markers he will pass along the way, and the exact distance the course will take him to the end destination. Because of their purpose in the course, outcomes should be specific, explanatory, and directed at how learning will be demonstrated.
While "Students will understand the principles of chemistry" might be a course objective, it is not a learning outcome. How would we measure or test it? A student could memorize well and ace all the tests and not really understand the principles. What demonstrates understanding?
Passing the test?
Checking off that he understood?
Doing all the homework?
Completing the labs?
If someone needed to see evidence that the student met that objective, could you provide undeniable proof?
The MIT Teaching and Learning Laboratory page describes course learning outcome (it calls them objectives) this way:
"It may be best to start with what learning objectives aren't: They aren't simply a list of the topics to be covered in the course.
Certainly, there will be a body of knowledge that students should know and understand by the time the course is complete. But if the goals for what students should achieve stops there, there may be many missed opportunities for providing them with a more productive learning experience.
A learning objective should describe what students should know or be able to do at the end of the course that they couldn't do before. Learning objectives should be about student performance.
Good learning objectives shouldn't be too abstract ("the students will understand what good literature is"); too narrow ("the students will know what a ground is"); or be restricted to lower-level cognitive skills ("the students will be able to name the countries in Africa.").
The following are examples of learning objectives drawn from several courses at MIT:
From a physics course on electromagnetism
The overall goal is to be able to explain the enormous variety of electromagnetic phenomena in terms of a few relatively simple laws.
From the introductory course in the department of Aeronautics and Astronautics
Students will be able to demonstrate an understanding of the fundamental properties of linear systems, by explaining the properties to others.
From a course in managerial communication
By the time you complete the course, you should be able to formulate an effective communication strategy by selecting appropriate content, organizational structure, and media.
Ideally, learning objectives should be accompanied by measurable outcomes, which describe ways in which students will be asked to demonstrate that they have achieved the learning objectives.
Methods of assessment of student learning can take many forms—exams (written or oral), papers, oral presentations, team projects.
Criteria for success (often called rubrics) should be developed so that students understand what is expected of them, and so that they can use feedback to see where they need to strengthen their performance." (emphasis mine)
Carnegie Mellon lists these course outcomes:
Course: Interpretation & Argument--Communication Globally: The World Is Your Audience
The following objectives emphasize the connection between perceptive analysis and effective writing:
Notice the emphasized words. All are things students can do and things we, as evaluators, can prove they did.
Can you look at each of these outcomes and imagine an artifact or specific project or process that could be documented for each?
So, a student learning objective needs to:
The University of Rhode Island has these guidelines:
Look through the guidelines.
When you create Student Learning Outcomes (SLOs), include the:
Watch the video!
Look at this handout from Eastfield College from Mesquite, Texas. Try the exercises toward the bottom!
Handout from Eastfield College.
One tried-and-true way to formulate active learning outcomes is to use Bloom's Taxonomy as a resource. Bloom organized cognitive processes into "levels." At the lowest, or simplest, level was Knowledge...facts memorized and recalled without much more involved (think of the ABC's). At the other end of the spectrum is Evaluation, in which the learner categorizes the worth, or value, or an array of possibilities using criteria.
Knowing what we want students to end up with is often the beginning place of determining what they will do. If we want them to acquire deeper understanding, then we need to have them accomplish more complicated tasks.
Identifying Earth's continents calls for simpler processes than those involved in determining which continent Earth could do without, with the least physical and economic impact.
|Knowledge||recalling or remembering something without necessarily understanding, using, or changing it||define, describe, identify, label, list, match, memorize, point to, recall, select, state|
|Comprehension||understanding something that has been communicated without necessarily relating it to anything else||alter, account for, annotate, calculate, change, convert, group, explain, generalize, give examples, infer, interpret, paraphrase, predict, review, summarize, translate|
|Application||using a general concept to solve problems in a particular situation; using learned material in new and concrete situations||apply, adopt, collect, construct, demonstrate, discover, illustrate, interview, make use of, manipulate, relate, show, solve, use|
|Analysis||breaking something down into its parts; may focus on identification of parts or analysis of relationships between parts, or recognition of organizational principles||analyze, compare, contrast, diagram, differentiate, dissect, distinguish, identify, illustrate, infer, outline, point out, select, separate, sort, subdivide|
|Synthesis||reating something new by putting parts of different ideas together to make a whole.||blend, build, change, combine, compile, compose, conceive, create, design, formulate, generate, hypothesize, plan, predict, produce, reorder, revise, tell, write|
|Evaluation||judging the value of material or methods as they might be applied in a particular situation; judging with the use of definite criteria||accept, appraise, assess, arbitrate, award, choose, conclude, criticize, defend, evaluate, grade, judge, prioritize, recommend, referee, reject, select, support|
Lorin Anderson, a former student of Bloom, revisited the cognitive domain in the learning taxonomy in the mid-nineties and made some changes, with perhaps the two most prominent ones being, 1) changing the names in the six categories from noun to verb forms, and 2) slightly rearranging them (Pohl, 2000).
This new taxonomy reflects a more active form of thinking and is perhaps more accurate:
Example and Key Words (verbs)
Remembering: Recall previous learned information.
Examples: Recite a policy. Quote prices from memory to a customer. Knows the safety rules.
Key Words: defines, describes, identifies, knows, labels, lists, matches, names, outlines, recalls, recognizes, reproduces, selects, states.
Understanding: Comprehending the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words.
Examples: Rewrites the principles of test writing. Explain in one's own words the steps for performing a complex task. Translates an equation into a computer spreadsheet.
Key Words: comprehends, converts, defends, distinguishes, estimates, explains, extends, generalizes, gives an example, infers, interprets, paraphrases, predicts, rewrites, summarizes, translates.
Applying: Use a concept in a new situation or unprompted use of an abstraction. Applies what was learned in the classroom into novel situations in the work place.
Examples: Use a manual to calculate an employee's vacation time. Apply laws of statistics to evaluate the reliability of a written test.
Key Words: applies, changes, computes, constructs, demonstrates, discovers, manipulates, modifies, operates, predicts, prepares, produces, relates, shows, solves, uses.
Analyzing: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.
Examples: Troubleshoot a piece of equipment by using logical deduction. Recognize logical fallacies in reasoning. Gathers information from a department and selects the required tasks for training.
Key Words: analyzes, breaks down, compares, contrasts, diagrams, deconstructs, differentiates, discriminates, distinguishes, identifies, illustrates, infers, outlines, relates, selects, separates.
Evaluating: Make judgments about the value of ideas or materials.
Examples: Select the most effective solution. Hire the most qualified candidate. Explain and justify a new budget.
Key Words: appraises, compares, concludes, contrasts, criticizes, critiques, defends, describes, discriminates, evaluates, explains, interprets, justifies, relates, summarizes, supports.
Creating: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.
Examples: Write a company operations or process manual. Design a machine to perform a specific task. Integrates training from several sources to solve a problem. Revises and process to improve the outcome.
Key Words: categorizes, combines, compiles, composes, creates, devises, designs, explains, generates, modifies, organizes, plans, rearranges, reconstructs, relates, reorganizes, revises, rewrites, summarizes, tells, writes.
Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.
Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1973). Taxonomy of Educational Objectives, the Classification of Educational Goals. Handbook II: Affective Domain. New York: David McKay Co., Inc
Noddings, N. (1996). Stories and affect in teacher education. Cambridge Journal of Education, Vol. 26 Issue 3, Pp. 435-647.
Pohl, M. (2000). Learning to Think, Thinking to Learn: Models and Strategies to Develop a Classroom Culture of Thinking. Cheltenham, Vic.: Hawker Brownlow.
Crafton Hills College in California has a very informative learning outcomes cycle (which will move us down the right road at a steady clip).
There are not only differing levels of cognitive engagement, there are other domains to consider.
Cindy Vinson lists the learning domains and how they can be planned, delivered, and assessed.
Drill and practice
Short answer essay
Project or problem-based activities
Web-enhanced materials supplementing classroom lectures
Hybrid course with cognitive content on the web
Multimedia simulations of challenging and key concepts
Project based for higher cognitive skills
Multiple choice or short essay questions
Self-reflective writing in a journal
Practice tutorials designed for student success
Streaming audio explanations and encouragement
Interactive video, web casts, conference calls
Self-assessment using check-list
Pre/post attitude survey related to course content
Retention/success in course
Practice of desired skill with feedback
Arranging sequences of an activity in correct order
Pictures with audio and text explanations
Interactive video demonstrations
Performance of skill matches set standard as observed by an instructor or designee
Structured team projects with debriefing
Analyzing video models and identifying correct from incorrect performance
Face-to-face small group coaching and feedback sessions
Check lists, examples, videos and other cognitive support material presented online
Team, instructor and self assessment measures
Analysis of video taped student performance of desired interpersonal skill
The Affective Domain is gaining some educational notice today. Proponents say that, without the "buy in" of affective components, the cognitive process is hindered. For instance, students studying Algebra often ask, "Why should I learn this?" They see no value in learning it, which hinders their motivation to do so. Until they are willing to receive, hear, and respond to new information, they must involve themselves in establishing why they need to learn it.
"Affect has been neglected in education and this neglect reduces the engagement of both students and teachers in their studies… For the past 200 years, philosophers have emphasized reason over affect… Emotion has, for the most part, been dismissed as unreliable." (Noddings, 1996; P. 435-6)
The affective domain (Krathwohl, Bloom, Masia, 1973) includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes. The five major categories are listed from the simplest behavior to the most complex:
Example and Key Words (verbs)
Receiving Phenomena: Awareness, willingness to hear, selected attention.
Examples: Listen to others with respect. Listen for and remember the name of newly introduced people.
Key Words: asks, chooses, describes, follows, gives, holds, identifies, locates, names, points to, selects, sits, erects, replies, uses.
Responding to Phenomena: Active participation on the part of the learners. Attends and reacts to a particular phenomenon. Learning outcomes may emphasize compliance in responding, willingness to respond, or satisfaction in responding (motivation).
Examples: Participates in class discussions. Gives a presentation. Questions new ideals, concepts, models, etc. in order to fully understand them. Know the safety rules and practices them.
Key Words: answers, assists, aids, complies, conforms, discusses, greets, helps, labels, performs, practices, presents, reads, recites, reports, selects, tells, writes.
Valuing: The worth or value a person attaches to a particular object, phenomenon, or behavior. This ranges from simple acceptance to the more complex state of commitment. Valuing is based on the internalization of a set of specified values, while clues to these values are expressed in the learner's overt behavior and are often identifiable.
Examples: Demonstrates belief in the democratic process. Is sensitive towards individual and cultural differences (value diversity). Shows the ability to solve problems. Proposes a plan to social improvement and follows through with commitment. Informs management on matters that one feels strongly about.
Key Words: completes, demonstrates, differentiates, explains, follows, forms, initiates, invites, joins, justifies, proposes, reads, reports, selects, shares, studies, works.
Organization: Organizes values into priorities by contrasting different values, resolving conflicts between them, and creating an unique value system. The emphasis is on comparing, relating, and synthesizing values.
Examples: Recognizes the need for balance between freedom and responsible behavior. Accepts responsibility for one's behavior. Explains the role of systematic planning in solving problems. Accepts professional ethical standards. Creates a life plan in harmony with abilities, interests, and beliefs. Prioritizes time effectively to meet the needs of the organization, family, and self.
Key Words: adheres, alters, arranges, combines, compares, completes, defends, explains, formulates, generalizes, identifies, integrates, modifies, orders, organizes, prepares, relates, synthesizes.
Internalizing values (characterization): Has a value system that controls their behavior. The behavior is pervasive, consistent, predictable, and most importantly, characteristic of the learner. Instructional objectives are concerned with the student's general patterns of adjustment (personal, social, emotional).
Examples: Shows self-reliance when working independently. Cooperates in group activities (displays teamwork). Uses an objective approach in problem solving. Displays a professional commitment to ethical practice on a daily basis. Revises judgments and changes behavior in light of new evidence. Values people for what they are, not how they look.
Key Words: acts, discriminates, displays, influences, listens, modifies, performs, practices, proposes, qualifies, questions, revises, serves, solves, verifies.
So, what do you do with those learning outcomes once they are compiled, organized, and distributed?
At every level, SLOs help with quality control and systems improvement.
If we have a list of program SLOs that are not relevant, not informative for students who are considering entrance, don't really describe the results of the program, are not being met, and don't align with institutional outcomes, then we know that we are wandering without direction. Outcomes are the roadsigns, mileage markers, and highway numbers to get us to the destination.
1. The assessment of student learning begins with educational values.
2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
5. Assessment works best when it is ongoing, not episodic.
6. Assessment fosters wider improvement when representatives from across the educational community are involved.
7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
9. Through assessment, educators meet responsibilities to students and to the public.
(EXCERPT- Emphasis mine)
Alexander W. Astin, University of California at Los Angeles; Trudy W. Banta, Indiana University-Purdue University at Indianapolis; K. Patricia Cross, University of California, Berkeley; Elaine El-Khawas, American Council on Education; Peter T. Ewell, National Center for Higher Education Management Systems; Pat Hutchings, American Association for Higher Education; Theodore J. Marchese, American Association for Higher Education; Kay M. McClenney, Education Commission of the States; Marcia Mentkowski, Alverno College; Margaret A. Miller, State Council of Higher Education for Virginia; E. Thomas Moran, State University of New York, Plattsburgh; Barbara D. Wright, University of Connecticut.
Visit this site for a more complete list:http://www.learningoutcomeassessment.org/PrinciplesofAssessment.html
If we aren't using measures of student success in meeting learning outcomes in our courses, programs, departments, and institutions, we are missing the point. The student is the pivot upon which they all turn.Without evidence of student learning, we are failing to educate.
"Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance."
Dr. Tom Angelo, Reassessing (and Defining) Assessment. The AAHE Bulletin, 48(2), November 1995, pp.7-9.
Watch the video for tips on choosing assessment measures.
We have seen that SLOs should be:
That's a large order for an SLO. But, written well, course learning outcomes guide course development, enrollment, and success.
We have looked at the what, why, and how of student learning outcomes.
Hopefully, your brain has digested some fodder and you are better prepared to write and use SLOs.
Take the quiz and print your certificate!