A systematic review of instruments for the assessment of professional competence of physiotherapy students

Purpose: Health professional education programs typically assess competence by direct assessment of clinical practice. The assessment of adequate performance is required for certifying fitness to practice. The focus of this systematic review was to identify all instruments currently used to assess the practice competencies of physiotherapy students, and to summarize the evidence of reliability, validity, feasibility educational impact and costs. Relevance: A preliminary search of the literature revealed no systematic review of existing instruments for assessing professional competence of students in physiotherapy programs. The quality of assessment of competence in professional practice programs cannot be ignored or undervalued, because high-stakes decisions about learners are based on the outcomes of these assessments. Participants: Included studies described an instrument designed for assessment of professional competence of student physiotherapists in a practice environment. Studies were included if it described an instrument intended for use with undergraduate, graduate-entry or post-graduate students of any tertiary institute or registration body offering a physiotherapy qualification, re-registration of a qualification or specialisation. Studies were excluded if the instrument was designed for use with standardized patients or used for peer, patient or self assessment. Methods: Fourteen electronic databases were searched without date limits until May 2009.Two independent researchers extracted data using a standardised form. Differences were reconciled by discussion with a third reviewer. Analysis: Evidence of validity, reliability, acceptability, authenticity, impact on education, training and costs of each instrument was compiled using best practice criteria. Results: 19 studies reporting on 12 instruments met the inclusion criteria. For seven of the 12 (58%) instruments evidence was provided in relation to validity based on test content with respect to measurement aim, target population, item and performance indicator selection and format of the rating scale. Four instruments had validity evidence based on internal structure, and relationship to other variables. No study investigated the educational impact of the assessment process or supplied sufficient information on educator training provided or costs associated with instrument use. ICC for inter-rater reliability was reported for four instruments. Conclusions: Overall, the Clinical Performance Instrument (CPI), Evaluation of Clinical Competence (ECC), Clinical Internship Evaluation Tool (CIET), and two unnamed Irish instruments currently provide the most evidence in relation to their development and measurement properties. No instrument is supported by satisfactory evidence of all desirable measurement properties. Given the high stakes associated with assessment of competency to practice physiotherapy, additional research is required for confidence in an assessment instrument with adequate evidence of validity. Implications: The review found a range of research of variable design and methodological quality. Increased attention to the systematic collection of validity evidence for scores from assessment instruments will improve the outcomes for all stakeholders in physiotherapy education.