Final Report of the Provost's Task Force on Academic Assessment (July 2013)


Context

In order to both foster and enhance the culture of academic assessment at Stony Brook University, Provost Dennis Assanis appointed an Academic Assessment Task Force to develop a university-wide strategy and timeline for the implementation of assessment plans for all academic programs at Stony Brook University. This final report details how we conceived the task, how we organized ourselves, and the recommendations we are making. The Task Force met biweekly as a committee of the whole, with subgroups meeting in the off weeks. We had representation from most areas of campus, including some with experience and expertise in academic assessment and others with no prior experience. This mix proved to be useful as it sparked conversations about the appropriate goals for and wide variety of approaches to academic assessment that has helped us understand the challenges we face in developing a campus-wide strategy and plan for communicating this strategy.

Early discussions centered on the basic questions: What is academic assessment? How does it differ from external evaluation of programs? What are its purposes? What will academic assessment mean in the different academic departments at SBU? Who will do it? How can we foster a culture of assessment across our university that will persist beyond accreditation pressures? We concluded that academic assessment is a process of self-improvement in which evidence is gathered and applied by programs to improve the “learning outcomes” of students in the program. By contrast, program evaluation “grades” program performance from the outside. We recognized that the nature of this assessment process and its evidence vary from discipline to discipline. Assessment plans must be developed and implemented by programs themselves with centralized support from the institution. A change in institutional culture on assessment will require an investment of resources. The focus is on assessing academic programs (undergraduate majors and minors, graduate and professional programs at the master’s and doctoral levels); course-level assessment will be necessary as a component of program assessment. The fundamental questions to be addressed by programs are:

  1. Learning Outcomes/Goals: What do we want students to learn?
  2. Metrics/Data/Evidence: How do we know what they are learning?
  3. Closing the Loop: How can we modify our programs so students better learn what we want them to learn?

It was noted that some faculty and programs already had academic assessment programs in place, usually because of external accreditation demands; in other areas the process is informal and idiosyncratic, but the basis for formal assessment is already in place in the implicit or explicit standards faculty apply in their courses and programs. We must also emphasize that the standards of general accreditation (Middle States) now mandate formal assessment within all programs. It is a good idea for our own improvement, we are already doing it on some level, and it is required by outside bodies.

In order to accomplish the goals of the Task Force as articulated in the Provost’s Charge and as elaborated in our own early discussions, we divided into 4 topical subgroups. Building on the work of these subgroups, the committee as a whole developed the set of recommendations described below. A great deal of work remains to be done to facilitate this change of culture on our campus, and so the Task Force intends to continue working beyond the submission of this report.

Goals of the Task Force (From the Provost’s Charge)

  1. Communicates that academic assessment is an expectation for every program and course.
  2. Recognizes those areas in which academic assessment is currently practiced and identifies academic leaders whose experience and expertise could be helpful in rolling these practices out across the board.
  3. Create an environment in which faculty can be helped to develop measureable learning outcomes for all programs and propose specific mechanisms for helping them reach that goal.
  4. Develop a timetable by which all programs will be held accountable for having written learning outcomes and a way to measure them.
  5. Ensure that those programs with external accreditation are in fact practicing continuous academic assessment (and not just in anticipation of an external site visit)
  6. Develop institutional academic assessment guidelines that include the documentation of program learning outcomes, plans for the collection of learning outcome data, and a format for reporting on an ongoing basis how learning outcome data are used in decision making.

Sub-Groups and Charges

  1. Collection of Information about Assessment Practices
    • Develop a plan to collect information to better understand the current state of assessment practices campus-wide (e.g. development of a template, etc.).
    • Investigate examples of a broad array of assessment plans from different disciplines and contexts.
    • Conduct a detailed survey of current program assessment practices (see Appendix A PDF).
  2. Metrics/Rubrics
    • Identify examples of program goals and curriculum maps to determine where and how key knowledge and skills are taught and show the progress students are expected to make throughout the curriculum.
    • Identify excellent (useful and practical) examples of assessment tools (rubrics, etc.) for groups of disciplines and professions.
    • Examine resources needed to support and facilitate the development and application of assessment methods.
    • Develop a literature list of current texts and on-line references on rubrics, assessment methods, measurement of organizational performance, and creating and maintaining assessment programs.
  3. Structure
    • Identify a structure for “on-going” communication among the colleges, schools, and programs.
    • Identify a plan for the development of a sustainable assessment program campuswide.
    • Develop a “Flow Chart” to identify what programs require academic assessment.
  4. Communication
    • Develop a strategy for communicating the university assessment plan to all constituents (e.g. assessment website, town hall meetings, etc.).
    • Develop a simple guide to assessment.
    • Develop a glossary for assessment concepts and terms.

Recommendations

Allied to a commitment to academic assessment at all levels of the university the task force makes the following recommendations.

Recommendation 1: Establish a permanent Office of Academic Assessment at the university

Stony Brook University must establish an infrastructure of personnel and resources to support a culture of assessment campus-wide. Key personnel must be in place as part of the permanent administrative structure of the university. A Director and Assistant Director of Academic Assessment with responsibility for the oversight of all assessment activities across campus should be appointed as soon as possible. Additionally, to support the coordination of data and information we recommend that a data analyst be hired.

The Office of Academic Assessment would coordinate and provide the resources to support the assessment infrastructure. These resources include but are not limited to:

  • Extensive web-site and internet resources
  • Offering workshops, seminars and conferences on state-of-the- art assessment strategies
  • Assisting with the development of assessment plans
  • Assisting with collection of evidence, analysis and reporting
  • Serving as an assessment data repository
  • Assisting with recommendations to inform quality improvement
  • Maintaining data for regulatory agencies and other program reviews
  • Ensuring that academic assessment is visible and ongoing on campus
  • Assisting in the coordination of the Stony Brook Curriculum assessment plan

It is important to note that we can build on the resources that are already available through the Faculty Center: http://facultycenter.stonybrook.edu/assessment.

Recommendation 2: Appoint Assessment Coordinators in each academic unit

The successful introduction of a more formalized assessment initiative requires skilled leadership, the development of open and clear communication pathways and the identification and input of a team of appropriately qualified faculty. Most importantly, the success of a university-wide assessment program requires widespread faculty/staff support and engagement. The goal is to develop a university-wide culture, which recognizes that the mission of providing comprehensive, high quality education is promoted by the incorporation of rigorous assessment practices into academic programs. The intent is to avoid producing a culture of compliance by adopting a flexible system that meets the varied needs of all the academic entities at the university. A key factor in this development is the appointment of personnel to act as assessment coordinators.

Given the size, scale and diversity of the university, no single system for appointing program assessment coordinators would be appropriate. The chain of command for such an assessment system would follow a hierarchy from the Provost through the deans and department chairs to the undergraduate and graduate program directors, who would be largely responsible for identifying faculty members who have experience and expertise with assessment or who can commit to gaining this expertise. While faculty in each academic program will develop assessment plans for their programs and courses, the program directors would be tasked with appointing the assessment coordinators or teams of coordinators for their academic areas and identifying what resources are needed. The assessment coordinators would monitor the assessment practices and liaise with the University-wide assessment offices. It will be advisable to compensate coordinators in areas that have no previous assessment experience or who may hold wider assessment responsibilities across broad groups, for example undergraduate STEM education, general education, undergraduate colleges, or grouped departments with common missions like humanities or life sciences. While the appointed coordinators will lead the assessment process, faculty within academic programs must be in control of the assessment of their own programs.

Recommendation 3: Establish a university-wide committee for the purpose of ongoing communication and policymaking regarding assessment

We recognize that there is a need for ongoing communication regarding assessment activities campus-wide. A major function of the committee will be to provide a forum for the exchange of information, ideas and practices of academic assessment. This committee should serve in an advisory capacity to the provost and should formally incorporate representation from the Standing Committees of the University Senate that deal with undergraduate and graduate education (the Undergraduate and Graduate Councils).

Recommendation 4: Establish a university-wide assessment recognition program

It is vital that the university encourages broad involvement and recognizes those who engage in quality assessment activities. We believe that a process should be established that makes successful assessment achievement prominently visible on our campus, e.g. Presidential Mini-Grants or awards for those programs or individuals that develop and implement exemplary assessment activities. Appropriate compensation should be provided.

Recommendation 5: Conduct a survey of current assessment practices at SBU

We have developed a survey instrument to be administered after a university academic assessment policy has been announced. The purpose of the survey is to better understand the degree to which assessment practices are used to improve student outcomes at Stony Brook University. The survey will be sent to the coordinators to complete, and is attached as Appendix B PDF to this report.

Recommendation 6: When the assessment policy is announced to the university community, it must be simple, flexible, and under the control of each program’s faculty

We believe that to build a successful culture of assessment faculty must see the value of a formalized program and not be overwhelmed by a “regulatory mandate”. For this reason, we have developed a simple, jargon-free introduction and guide for outcomes-based academic assessment that should be distributed when the assessment policy is announced. This primer is attached as Appendix C PDF.

Recommendation 7: Be very clear on a timeline of activities and provide the necessary resources for timely progression

It is very important that the administration signal the importance of academic assessment by committing the necessary resources to support the activities as described in earlier recommendations. We must strive to have these resources in place in sync with the timeline expected for program faculty to follow in creating their plans. We attach our recommended timeline as Appendix D PDF.

Appendices