Speaker diarisation evaluation

Speaker diarisation evaluation

We present a new evaluation technique based on segment matches using the F-measure. This gives the user a deeper insight into how well matched the hypothesised segments are to the reference segments.

  • Accounts for reference errors using a collar and merging segments of the same speaker occurring within 0.25 seconds of each other.
  • Segments must match if the start and end boundaries of the hypothesised segment lies on the start and end boundaries (+/- collar) of the reference segment
  • Different distributions (uniform, triangular, Gaussian) on the reference boundary can be considered
  • Speaker mapping finds a score or cost involving the number of segments matched between a reference speaker and an hypothesis label and searches every possible combination of pairs, finding the combination of pairs with the lowest cost


Rosanna Milner, Thomas Hain: Segment-oriented evaluation of speaker diarisation performance. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016, Shanghai, China, March 20-25, 2016, pp. 5460–5464, IEEE, 2016.

Back to Top