Can we perceptually rate alaryngeal voice? Developing the Sunderland Tracheoesophageal Voice Perceptual Scale

Objective: To investigate the inter and intra reliability of raters (in relation to both profession and expertise) when judging two alaryngeal voice parameters: ‘Overall Grade’ and ‘Neoglottal Tonicity’. Reliable perceptual assessment is essential for surgical and therapeutic outcome measurement but has been minimally researched to date. Design: Test of inter and intra rater agreement from audio recordings of 55 tracheoesophageal speakers. Setting: Cancer Unit. Participants: Twelve speech and language therapists and ten Ear, Nose and Throat surgeons. Main outcome measures: Perceptual voice parameters of ‘Overall Grade’ rated with a 0–3 equally appearing interval scale and ‘Neoglottal Tonicity’ with an 11-point bipolar semantic scale. Results: All raters achieved ‘good’ agreement for ‘Overall Grade’ with mean weighted kappa coefficients of 0.78 for intra and 0.70 for inter-rater agreement. All raters achieved ‘good’ intra-rater agreement for ‘Neoglottal Tonicity’ (0.64) but inter-rater agreement was only ‘moderate’ (0.40). However, the expert speech and language therapists sub-group attained ‘good’ inter-rater agreement with this parameter (0.63). The effect of ‘Neoglottal Tonicity’ on ‘Overall Grade’ was examined utilising only expert speech and language therapists data. Linear regression analysis resulted in an r-squared coefficient of 0.67. Analysis of the perceptual impression of hypotonicity and hypertonicity in relation to mean ‘Overall Grade’ score demonstrated neither tone was linked to a more favourable grade (P = 0.42). Conclusions: Expert speech and language therapist raters may be the optimal judges for tracheoesophageal voice assessment. Tonicity appears to be a good predictor of ‘Overall Grade’. These scales have clinical applicability to investigate techniques that facilitate optotonic neoglottal voice quality.