KPI in the Planning Hierarchy A one-to-one relationship (e.g. one measure has one target) A one-to-many relationship (e.g. one goal may have many strategies) Mission Goal Objective Indicator Measure Target Result Strategy
KPI in the Planning Hierarchy KPI may be thought of as essential elements of the overall planning and monitoring system. They are derived from the College’s Mission and help track performance towards accomplishing the Mission. A KPI is not a single component, but an integrated collection of components . Mission Goal Objective Indicator Measure Target Result Strategy Indicator Measure Target Result
Kaplan & Norton designed a ‘Balanced Scorecard’, with ‘lag’ and ‘lead’ KPIs. Useful for managing large numbers of KPIs in a complex organisation.
‘ Lag’ indicators tell how well a system performed. Mostly used in ADRI ‘Results’ and ‘Deployment’ phases. Examples include:
Student employment rates; research publications; number of prizes
‘ Lead’ indicators help predict system performance. Mostly used in ADRI ‘Approach’ and ‘Improvement’ phases. Examples include:
Staff training; Investment in advertising; Student-computer ratios
Lag, Lead & the Planning Hierarchy Put another way, ‘lag’ KPI focus on the extent to which outcomes-based objectives have been achieved. ‘ Lead’ KPI focus on the extent to which strategies for achieving the objectives are being successfully implemented. Therefore we can structurally separate lag and lead KPI in the planning hierarchy. Mission Goal Objective Indicator Measure Target Result Strategy
Lag, Lead & the Planning Hierarchy There may be separate KPI for objectives and their associated strategies. Provided that objectives are outcomes-based, and strategies support objectives, then: Strategy Indicator Measure Target Result Lag KPI set Lead KPI set Mission Goal Objective Indicator Measure Target Result
Address staff concerns and ideas about performance management.
Address concerns about the uncertainty of the data.
Regardless of the topic, keep performance review processes fair and transparent.
Place performance review within a context of learning.
Enable a two way dialogue about performance.
Remember to highlight good news!
Workshop Example Indicators for Learning and Teaching
For each of these three objectives, strategies are provided.
KPIs are then constructed using the model: indicator -> measure -> target
KPIs are then tested using SMARTER
2. The program will be viable. 1. Students will learn to solve problems. 3. The program will be relevant to industry and employers. Objectives
Workshop Example #1 Indicators for Learning and Teaching SPECIFIC? Yes. The indicator is tightly focused on the single issue of courses design. MEASURABLE? Yes. The number of courses that comply can easily be counted. ALIGNED? Yes. The measure relates directly to the indicator; the target relates directly to the measure. REALISTIC? Yes. The KPI has been made realistic by separating new courses, which can implement the strategy directly, from existing courses, which will need to be changed. TIMELY? Yes. The measure is each semester, allowing for current data and trend data. The target is time-specific. ETHICAL? There are no obvious ethical concerns with collecting and using this information. RECORDED? The target implies that trend data will need to be kept and analysed until at least 2010. TIMELY? Yes. The measure is each semester, allowing for current data and trend data. The target is time-specific. REALISTIC? Yes. The target does not appear unachievable. SPECIFIC? Yes. The indicator is tightly focused on the single issue of assessing student’s problem solving abilities. ALIGNED? Yes. The measure relates directly to the indicator; the target relates directly to the measure. MEASURABLE? Yes. Student pass rates are easily measured. RECORDED? The target implies that trend data will need to be kept and analysed until at least 2008. ETHICAL? There are no obvious ethical concerns with collecting and using this aggregated information. These are SMARTER KPIs Target: 80% by 2007 Measure: student pass rate/semester Design and implement problem based assessment of student learning Indicator: courses include learning outcomes that relate to problem solving Design and deliver problem based curriculum Students learn to solve problems Target: 100% of new courses; 100% of all courses by 2010 Indicator: courses include PBL assessment exercises Measure: number of courses each semester with problem solving learning outcomes included in institutional course guides KPIs Strategy Objective
Workshop Example #2 Indicators for Learning and Teaching TIMELY? No. There is no time definition provided for this KPI REALISTIC? No. Any national or international benchmark will show that the target of 100% pass rates is not realistic. SPECIFIC? No. ‘Student satisfaction’ is very broad. Satisfaction with exactly what? ALIGNED? No. Student satisfaction and student pass rates are not the same thing. MEASURABLE? Yes. Student pass rates are easily measured. RECORDED? There is no indication on how this KPI will be recorded, reported and stored. ETHICAL? Maybe not. The target of 100% may encourage staff to mark too softly. TIMELY? No. There is no time definition provided for this KPI REALISTIC? No. If the measures and targets cannot be measured and do not align, then how can they be realistic? SPECIFIC? No. ‘Student satisfaction’ is very broad. Satisfaction with exactly what? ALIGNED? No. Staff qualifications is not necessarily related to whether the program is excellent. MEASURABLE? No. What is the means for measuring that the program is ‘excellent’? RECORDED? There is no indication on how this KPI will be recorded, reported and stored. ETHICAL? No. This KPI implies that staff can be held accountable for ‘student satisfaction’ and the ‘excellence’ of the program and by virtue of their qualifications. The case for this is not justified here. These are not SMARTER KPIs Let’s improve them… Measure: (a) Student pass rates (b) Excellent program Target: (a) 100% (b) All staff qualified Indicator: Student satisfaction. Attract the best students by direct marketing of programs to school leavers. The program will be viable. KPIs Strategy Objective
Workshop Example #2 Indicators for Learning and Teaching Measure: (a) Percentage of enrolled students who listed the program as their 1 st preference (via Admission Centre) (b) Proportion of 1 st preference students in national top 10% by academic rank Target: (a) More than 50% (b) At least 20% and up to 40% (assuming four competitor programs in Oman) Indicator: Student program preferences by academic performance. Attract the best students by direct marketing of programs to school leavers. The program will be viable. KPIs Strategy Goal
Workshop Example #3 Indicators for Learning and Teaching Indicator: Acceptance of program by industry Measure: (a) Professional accreditation of program by 2009 (b) Positive feedback from employers and industry Indicator: Graduate employability rate Establish Industry Advisory Boards (IABs) for each program. Invite key employers to be members. The program will be relevant to industry and employers. Measure: % graduates in full time employment in the industry Target: above national average Target: (a) Professional accreditation p assed (b) Positive IAB Program Review KPIs Strategy Goal
Select an area (from the list) that you are currently working on.
What are two or three main objectives for that area?
What strategies could you have in place to achieve each objective?
Construct appropriate KPIs, including Indicators -> Measures -> Targets.
Check them. Are your KPI SMARTER?
Are they Lag or Lead KPI?
When you have finished (and ONLY then!), try clicking each area for further examples of good KPI.
When you are ready you may EXIT
Workshop Activities Student Outcomes (a) Indicator: Student progress rates Progress Scheme identifies ‘at risk’ students, by Department, and invites them to participate in student learning support services. Students will achieve high rates of success in their learning Measure: Percentage of students progressing to the next level of study without having to repeat any units; differentiated by those who participated in the Progress Scheme and those who didn’t. Target: Improvement trends over three years for both cohorts, but steeper for Progress Scheme students. KPI Strategy Objective
Workshop Activities Student Outcomes (b) Return to list Target: At least 90% Measure: Percentage of graduates in preferred employment within 12 months of graduating Academic Advisors are available for career planning Students will be highly satisfied with their career choices Indicator: Graduate recruitment by employment preference KPI Strategy Objective
Workshop Activities Educational Support Resources Return to list Target: Average satisfaction is above 4 on a 1-5 likert type scale Measure: Academic staff satisfaction results on customer feedback surveys Indicator: Staff and student satisfaction with the help desk The library has developed a ‘one-stop-shop’ help desk for increased support to academic staff and students The library supports learning outcomes Actively communicate with College departments to identify IT requirements and problems with IT services IT services support learning outcomes Measure: Library User Survey asks staff & students to rate their satisfaction with and relevance of helpdesk features. Gap measure is developed. Indicator: Client (Dept) satisfaction Target: The features with highest relevance also have equally high satisfaction KPI Strategy Objective
Workshop Activities Research and Research Training (a) Indicator: External grant applications Staff can access institutional seed grants money to encourage external applications for research funding. We will improve the level of research income via external grants
Proportion of FTE Teaching+ Research staff submitting an external grant application (Lead)
Proportion of applications successful (Lag)
50% of all FTE staff submitting an external grant by 2010.
20% success rate
KPI Strategy Objective
Workshop Activities Research and Research Training (b) Return to list Indicator: S tudents successfully completing postgraduate research degrees in a timely fashion System of bi-annual progress reporting for HDR students that includes follow up and evaluation processes Measure: Proportion of HDR students graduating each year against total number of HDR students scheduled to complete each year.
at least 30 by 2010
Over 75% scheduled students graduate
We will improve the postgraduate research experience KPI Strategy Objective
Workshop Activities Community Service / Engagement (a) Click here to return to list Return to list Indicator: Participation on community boards / committees Staff are encouraged to nominate for community level boards and committees We will provide leadership to our local community Measure: Proportion of FTE staff (academic + non academic) who hold positions on boards and committees) Target: At least 50% KPI Strategy Objective
Workshop Activities Community Service / Engagement (b) Return to list
At least 30% in 2007; with an increasing trend of 3% every year for next three years.
At least 50%; and more than any other local College.
Proportion of local region new enrolments against College’s total new enrolments
Proportion of local region new enrolments against total local region school leavers
Schools liaison program is conducted with schools in the College’s catchment area. We will increase the number of new enrolments from students in the local region Indicator: Number of commencing students from our region KPI Strategy Objective
Workshop Activities Organizational Leadership Return to list Target: Mean satisfaction is a t least 4 on a 1-5 Likert-type scale. Measure: Mean staff responses to questions about climate. Indicator: Board and Committee Plans Every Board and Committee is required to develop a Plan that is aligned to the College Strategic Plan and includes performance indicators Strategic planning is practiced across all boards and committees The leadership team meets monthly to identify, discuss and respond to any issues raised by staff about the organisational climate, and to review progress on previous responses. We will create and maintain a positive organisational climate Measure: Proportion of Boards and Committees with a current Plan that complies with the College’s template Indicator: Staff satisfaction with their organisational climate. Target: 100% KPI Strategy Objective
Workshop Activities Staffing (a) Return to list Indicator: Staff turnover due to poor recruitment practices Staff are recruited into positions that are related to goals in the College Strategic Plan. Exit interviews are conducted with staff who leave the organisation. People are placed in meaningful jobs
expenditure related to staff turnover (eg. Recruitment expenses into recurrent jobs, retrenchment pay)
Number of staff who resign each year
No more than 0.5% total annual budget
Reduce by 3% per year
KPI Strategy Objective
Workshop Activities Staffing (b) Return to list Target: At least 60% Measure: Annual percentage of all staff who participated in professional development Professional development is a high priority within the organisation. All staff have access to a professional development program. Staff skills and competencies are developed on the job Indicator: Staff development activity KPI Strategy Objective
Workshop Activities Institutional Infrastructure Return to list Target: Usage at least 60% aggregated across all teaching spaces Measure: The total amount of people using the room over the total capacity of the teaching space. Indicator: Expenditure against budget. All budget centre managers are trained in the College’s financial management processes. Sound financial management is practiced Space planning is incorporated into the Colleges Academic Plan Effective space management supports learning outcomes Measure: Monthly variance of expenditure against budget; reported by account code. Indicator: Occupancy Adjusted for Usage Target: Expenditure no more than 5% over or 10% under budget. KPIs Strategy Objective
Workshop Activities Information and Communication Return to list Target: At least once per month Measure: Number of times QA issues are addressed in minutes Indicator: Level of media awareness of the institution Media and Marketing Strategy clearly targets media outlets Effective marketing maintains the high profile of the institution Heads of Depts hold regular staff meetings where QA issues are discussed Messages about quality assurance from management are consistently deployed to staff
Number of ‘hits’ in targeted media outlets (eg newspapers, radio)
Ratio of positive to negative ‘hits’
Indicator: Evidence of discussions about QA policies and strategies in staff meeting minutes
increasing over time
Greater than 60%
KPI Strategy Objective
KPI Designing Key Performance Indicators Congratulations on completing this training module.