Overview

Every department or program defines key learning outcomes for the major, maps major requirements to the outcomes, examines student work to see how well the outcomes are being accomplished, and uses findings to improve the academic program.  Departments and programs also review their contributions to general education requirements and assess work in selected courses to see how well students are achieving the general education outcomes.

Triennial Program Report & Annual Update

Senior administration reviews curricular and co-curricular programs on a three-year cycle.  Every department or program submits a Triennial Program Report according to the cycle below.  In the intervening years, the department or program submits an Annual Update.  Triennial reports are entered into an online system (webform).  Annual updates can be emailed to the Associate Dean of the Faculty: Bob Williams (robert.f.williams@lawrence.edu).  Reports or updates are due June 30.

2017, 2020, 2023... 2018, 2021, 2024... 2019, 2022, 2025...

Humanities
Chinese & Japanese
Classics
English
French
German
History
Philosophy
Religious Studies
Russian
Spanish

Interdisciplinary Programs
East Asian Studies
Film Studies
Global Studies
Latin American Studies
Linguistics

Conservatory of Music
Composition major
Music Theory major
Musicology core

University Programs
GER-Competency
Freshman Studies

Fine Arts
Art & Art History
Theatre Arts

Natural Sciences & Math
Biology
Chemistry
Geology
Mathematics & Computer Science
Physics

Interdisciplinary Programs
Biochemistry
Cognitive Science
Environmental Studies
Natural Sciences
Neuroscience

Conservatory of Music
Music Performance major
Major Ensembles (incl. Opera)
Jazz & Improvisational Music emphasis

University Programs
GER-Diversity
December Term
University Courses

Social Sciences
Anthropology
Economics
Education
Government
Psychology

Interdisciplinary Programs
Biomedical Ethics
Ethnic Studies
Gender Studies
Innovation & Entrepreneurship
Museum Studies

Conservatory of Music
Music Education major
B.A. Music major
Dance

University Programs
GER-Distribution
Off-Campus Programs
LU Research Fellows

Department & Program Assessment Guide

A useful way to think about your curriculum is to pose three questions:

  1. How will students be different when they are finished with this program?  What will they know and be able to do?  How will their view of the subject and their own capacities be changed?  Where and how will these changes happen?
  2. How will we know they are different?  What evidence will we have gathered?  Where and how will we gather it?
  3. How will they know they are different?  What will we do along the way to help them reflect on and integrate their learning and sense their increasing knowledge and ability?

These questions lead us to think about intentions, performances, and assessments. To put this into practice, use the guides below to define learning outcomes, create a curriculum map, develop assessment methods, use findings, and prepare updates and reports.

Getting Started

To get started, using the following document to structure a department or program conversation.  Answering the questions posed there will lay the groundwork for your assessment plan.

Click here for guiding questions on Major Learning Outcomes & Assessment (Word document).

+ How to Define Learning Outcomes

Learning Outcomes

Learning outcomes say what students should be able to do upon completion of the major. While students are expected to grow in many ways and to develop their own interests and talents, every department or program should define a few key outcomes that all students in the major are expected to accomplish and that the department will assess to determine the effectiveness of its academic program.

Learning outcomes are generally written in the form:

Students should be able to <action verb> <object>.

Use just one action verb and one object to make the outcome assessable. State the outcome in language a student is likely to understand.

Example:

A student who completes the major in X should be able to: <list items such as the following>

Produce sound philosophical discourse.

Analyze a literary text.

Translate prose from Latin into English.

Critique a mathematical argument.

Design an experiment to test a hypothesis.

Describe monuments of art in historical context.

Design a performance environment for the production of a play.

Use tools of economic theory to analyze an economic problem.

(etc.)

Components

A useful strategy is to define one or more high-level outcomes that provide overarching goals for the major and then to define specific components that can be directly assessed using student work in courses and capstones. Each component can have its own rubric to rate the quality of the work with respect to one or more outcomes.

Example:

A student who completes the major in Economics should be able to develop an economic analysis. Specifically, the student should be able to:

Formulate a problem for economic analysis;

Find relevant literature that bears on the problem;

Interpret quantitative evidence and regression analyses;

(etc.)

Quality Standards

Quality standards for work should spelled out for students and faculty.  A useful way to do this is to incorporate the standards into rubrics keyed to the desired outcomes (see "How to Assess" below).


Tips:

  • Define learning outcomes for every major offered by the department or program. There is likely to be overlap, but there should also be differences that distinguish the majors.
  • Focus on a few key outcomes that are expected of all students and that will be assessed to determine the effectiveness of the program. Students should go beyond these outcomes in individual ways.
  • State outcomes or components with just one action verb and one object to make them assessable.
  • State outcomes plainly so that students, faculty, and others can understand what they mean and agree when they have been achieved.
  • Develop quality standards for evaluating student work with respect to the outcomes.  Incorporate these into rubrics  to rate work samples from courses or capstones.

+ How to Prepare a Curriculum Map

Map Major Requirements to Outcomes

Once learning outcomes have been defined, prepare a curriculum map that shows the relationship between major requirements and learning outcomes.  One way to do this is to make a table or spreadsheet with major requirements as rows and outcomes as columns (or vice versa) and to indicate in the cells how requirements address outcomes and at what level.  You may want to do a similar mapping for individual courses.  The curriculum map will help you identify possible gaps in the curriculum and places to gather evidence for assessment.

Click here for a Curriculum Mapping Tool (Excel file).

+ How to Assess

Once you have your learning outcomes and curriculum map in place, approach assessment in the following way:

  1. Assess work in gateway courses to see if students are developing the desired knowledge and skills.
  2. Assess work in capstones (Senior Experiences) to see if the learning outcomes have been achieved.

For both #1 and #2, it's important to discuss work samples or ratings as a department to identify strengths and weaknesses and plan changes to the curriculum. This discussion could be part of a department meeting or annual retreat.  You should keep minutes to document the discussion for program or accreditation reviews.


Make a rubric

It can be quite useful to develop rubrics to rate work samples with respect to outcomes.  Rubrics can describe performance at different levels of development so faculty can rate work consistently and students can check their progress.  Decide on a number of levels (from 3 to 5) and a label for each level (such as: 1 = initial, 2 = basic, 3 = proficient, 4 = advanced).  Make it clear which level is the target (3 = proficient).

How to write a rubric

  1. Write a description for the "proficient" level first. This should clearly describe achievement of the outcome.
  2. Write a description for the "advanced" level next, indicating features that mark expert performance. This gives students a higher level to strive for and assessors a way to identify performance that is a model for others.
  3. Write a description for the "basic" level next, indicating features that are emerging but not yet mastered. This helps students identify how they are progressing.
  4. Write a description for the "initial" level last, identifying the foundation (prerequisite knowledge or skill) on which the outcome is built. This gives students and faculty a starting point for learning.
  5. Tweak the descriptions to clearly distinguish each level from the one just above and below, forming a progressive scale from initial to advanced.
  6. Edit descriptions to make them concise, and highlight the words or phrases that distinguish each level. This will make it easier to use the rubric to rate student work.
  7. Indicate which level is the target for students in the major.

Click here for a Rubric Template (Word file).

Sample rubrics:

Lawrence's GER rubrics

AAC&U VALUE rubrics

AALHE sample rubrics


Tips:

  • Give precise characteristics of performance at each level; avoid sweeping statements like "excellent," "average," or "poor."
  • State what is evident as well as what is lacking; avoid defining a level solely in terms of deficits ("lacks X, Y, Z... ").
  • Limit the criteria at each level; focus on the essential.
  • Contrast adjacent levels to make sure they distinguish performance well.
  • Avoid recapitulating the GER writing/speaking rubric. You can use that rubric alongside a rubric focused on major outcomes.
  • Test the rubric with samples of student work and check for consistency among raters. (Also do this for training before rating student work.)

Rate student work

To assess how well students are achieving key outcomes for the major, use your rubrics to rate student work from gateway courses and capstones. Mark the level of achievement on the rating scale and jot comments on strengths and areas for improvement. You can incorporate the rubric into your course grading or use it separately for department assessment.


Tips:

  • Focus the assessment: Select particular outcomes or components to assess. Use findings from past assessments to guide you.
  • Sample widely: Rate a selected assignment from many students to get a cross-section.
  • Balance dividing the labor vs. checking reliability: Have different assessors rate different outcomes, or have two or more assessors rate the same outcomes and compare to stimulate discussion.

Pool your findings

Gather faculty assessments of student work, both quantitative (ratings) and qualitative (comments). For each outcome, prepare a table, graph, or narrative summarizing the findings.  Describe common strengths and areas for improvement. Discuss these findings as a department and decide what to do next (see below).

Click here for a sample of how to report assessment data (PDF).

Click here for a sample Studio Art Excel tool for computing results (XLS).


Tips:

  1. Compare findings from your direct measures (assessment of student work) with those from indirect measures (surveys, focus groups, or structured interviews) to try to identify causes for the results you found.
  2. Use a chart or table to present the full range of data (not just the average). This will give a better sense of the variability in student performance.
  3. Look for patterns and compare to previous years.
  4. Relate your findings to your curriculum map as you consider what adjustments might improve student learning.

 

+ How to Use Findings

Assessment should stimulate useful conversations among department faculty.  Most important is that you use assessment findings to make improvements, large or small; otherwise, effort will have been expended for little benefit.  Accreditors are mainly interested in how we "close the loop" by using assessment findings for program improvement.

Discuss findings

Gather your ratings and comments and discuss them as a department.  Make this part of an annual retreat.

  1. Look for evidence of strengths and weaknesses in the academic program.
  2. Examine sample work as a group and discuss.
  3. Update your curriculum map and check where knowledge and skills are developed.
  4. Compare findings to previous years to identify trends.
  5. Document this in meeting minutes for future reports and reviews.

Plan actions

  1. Consider possible changes to curriculum, instruction, or assessment.
  2. Plan follow-up surveys, focus groups, or student interviews to gather more information on areas of concern.
  3. Decide how you will evaluate the impact of any program changes.
  4. Document findings and planned actions for your annual assessment update and triennial program report.

Tips:

  • Tweak requirements, assignments, guides, etc., to close gaps in the academic program or strengthen areas of concern.
  • Refine your outcome statements, rubrics, and assessment methods to better align them or make them more useful and reliable (consistent across raters).
  • Identify other assessment points to gather developmental data or assess outcomes not addressed by the Senior Experience.
  • Shift focus as needed: identify particular areas of concern to probe or follow up in the next assessment cycle.

+ How to Report

Every department or program submits an Annual Program Update with an updated curriculum map and summary of assessment findings.  The update should be concise and forthright: what you did, what you found, and what you're doing as a result. Your annual update provides a record of progress and an important source of information for your Triennial Program Report. 

Prepare Annual Update

Use the Annual Program Update to prepare your update.  Fill in each section concisely as directed, and attach an updated curriculum map.  Email your update to the Associate Dean of the Faculty by June 30.

Prepare Triennial Report

Use the Triennial Program Report to prepare your report.  Gather material and write a narrative on the state of the department or program.  Enter your report in the Triennial Reporting System by June 30.

+ What to Do After Reporting

Consult with the Accreditation & Assessment Committee

If you are planning to probe an area of concern or are considering changes to your learning outcomes, curriculum map, or assessment methods, feel free to consult with the Associate Dean of the Faculty for help from the Accreditation & Assessment Committee.

Archive

Be sure to save your assessment materials (learning outcomes, curriculum map, and rubrics), updates and reports, and committee feedback in your department share space so they are available for follow-up studies and future reporting.

Facebook Twitter Instagram YouTube