Preparing a Rating Scale

Once you have developed or identified your behavioural interview questions, you need to create an appropriate rating scale for your questions. A rating scale is the basis on which all candidates are evaluated. The rating scale should be well defined so that all interviewers can easily understand the scale being used and the meaning of each rating on the scale.

Simple Rating Scales

A rating scale could be relatively simple and only include anchors such as “satisfactory” or “unacceptable". A rating scale of this kind can be useful in that it simplifies the rating process by making it a “yes or no” type of decision, reducing variance and the need for much more deliberation by interviewers. The disadvantage to these kinds of scales is that it reduces variance in the results, meaning that you may end up with all of you candidates either “satisfactory” or “unacceptable".

Complex (Likert type) Rating Scales

Rating scales can also be more complex, for example with multi-point Likert-type scales (e.g., a scale of 1-5). The advantages of a more complex scale are that they allow for more variance in the results, providing more nuanced comparisons between candidates and the ability to use other factors other than just interview responses to drive decisions. The disadvantage to more complex scales is that it increases the need to look more closely at the differences between each candidate.

In general, it is better to go with more variance than less when conducting behavioural interviews. However, the number of ratings in the scale is not nearly as important as how those ratings are defined.

A generic example of a rating scale might look like this:

Far Exceeds Requirements: Perfect answer. Demonstrates competency accurately, consistently, and independently. All points relevant. All good examples.

Exceeds Requirements: Demonstrates competency accurately and consistently in most situations with minimal guidance. Many good examples.

Meets Requirements: Demonstrates competency accurately and consistently on familiar procedures and needs supervisor guidance for new skills. Some good examples.

Below Requirements: Demonstrates competency inconsistently, even with repeated instruction or guidance. Few good examples.

Significant Gap: Fails to demonstrate competency regardless of guidance provided. No good examples.

Behavioural Rating Scales (BARS)

For behavioural interviews, rating scales should include criteria for tying suggested answers to each point in the scale based on the key behaviours that each question supports. This form of rating scale is referred to as a Behaviourally Anchored Rating Scale (BARS). BARS differ from more generic rating scales in that they focus on the behaviours that are determined to be key to performing the job properly rather than evaluating more general candidate characteristics, such as personality or experience. For example, a superior response should look like X behaviour, a satisfactory response should look like Y behaviour, and an unsatisfactory response should look like Z behaviour. The best representation of the most desired job-related behaviours should be tied to a top rating, whereas the weakest representation of job-related behaviours should be tied to the lowest rating. Using a BARS approach for standardising the criterion used helps the interviewer rate candidates’ responses more equitably and improves rating consistency among interviewers.

BARS rating scales are used to evaluate the quality of a job candidate’s responses to competency-based behavioural questions; thus, the questions and rating scales will typically be developed in tandem. The critical incidents technique is one effective method for creating a BARS scale. The critical incident technique involves asking current job incumbents to provide examples of effective and ineffective behaviours that lead to positive and negative outcomes related to particular situations (i.e., incidents) one might encounter on the job. Interview developers then review this information to determine whether it should be used as the “anchors” in the scale.

Click here to download the full SHRM Guide on Behavioural Interviews