Anchor-based Goals and Personality Effects on Hazard Identification in Risk Assessment
, Harry England- DOI
- 10.2991/jracr.k.201014.001How to use a DOI?
- Keywords
- Goal; target; anchor; hazard; risk; identification; assessment
- Abstract
Hazard identification is a crucial first step in risk assessment. There are many cases in which hazard identification is carried out by non-experts. One concern is that valid hazards are overlooked and so not considered for mitigation or prevention. This study examined whether a goal-setting anchor could encourage the identification of more hazards and so reduce the likelihood that they are overlooked. Seventy-two participants were recruited to an online study to identify hazards in four vignettes. The participants were randomly allocated to a high or low anchor condition in which they were told that experts typically identify at least two or at least eight hazards. Participants also completed a five-factor personality measure. It was found that, compared to the low anchor, the high anchor increased word count, time on task and number of hazards identified. The effect of the anchor on hazards identified was robust even taking into account personality, time on task and word count. Conscientiousness was also associated with identifying more hazards. Overall, the use of anchors to set goals for hazard identification offers a low-cost intervention to improve risk assessment for non-experts.
- Copyright
- © 2020 The Authors. Published by Atlantis Press B.V.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
1. INTRODUCTION
Risk assessment involves a structured identification of potential hazards and the processes and outcomes associated with them. It is typically carried out in a professional capacity as a legal or operational requirement. Identifying hazards is the first step to mitigating, controlling or eliminating the likelihood and severity of accidents and thereby save lives, protect people and maintain organisational operations. Hazard identification is also a legal requirement for UK employers as part of employee protection [1]. Failure to identify hazards prevents subsequent mitigation strategies from being developed and then the likelihood and/or severity of negative outcomes cannot be limited. For example, the failure to recognise COVID-19 risk at mass gatherings between 8 and 13 March 2020 in the UK led to tens of thousands gathering in crowds at football matches and a horse racing festival with no mitigation strategies; these gatherings were followed by increased viral infection rates [2]. Risk assessment and hazard identification are vital to mitigate potential hazards.
Hazard identification is carried out in all types of organisation, big and small, but the focus is typically on safety-critical industries and processes. Expert risk assessors and sophisticated hazard identification procedures are used when the consequences of failed risk assessment are likely to lead to loss of life and/or serious harm, e.g. commercial aviation or pharmaceuticals. However, even small organisations without obvious safety concerns must still conduct risk assessments with hazard identification to protect people. Employees and the public might still be at risk from a small business which has no expert risk assessment capabilities. For example, falling, colliding and fire hazards are ubiquitous. The great fire of London, which devastated much of London in 1666, was caused by a fire in a bakery. The UK Health and Safety Executive offers guidance for all sizes of UK organisations and recognises identifying hazards as the first of three minimal steps, prior to assessing likelihood and severity, and then managing the identified hazards [3]. In smaller organisations without obvious safety concerns, hazard identification and risk assessment is likely to be conducted by a non-expert without an established procedure. These hazard identifications often occur across multiple different contexts, even within the same organisation, in micro risk-assessments. These non-expert hazard identifications and risk assessments are relied upon to protect people. However, hazard identification is not straightforward and variability in hazard identification is common.
Existing evidence from the empirical literature shows a considerable range in people’s ability to identify hazards and one possible explanation is experience. In documented risk assessments where hazard identification techniques were used, on average, only 66–90% of hazards were identified across different types of construction projects [4]. A study of Australian miners found the majority of those surveyed failed to identify common hazards (such as slips, falls and manual handling) [5]. Even experts are unlikely to have sufficient knowledge and experience to identify all hazards [4]. However, experience, while not sufficient on its own, does appear to increase hazard identification. Parents who spent more time with toddlers identified fewer than half the childcare hazards presented but still identified more hazards then healthcare and day-care workers [6]. Older and more experienced workers both have fewer accidents [7,8]. The least experienced workers identified the fewest hazards in a hazard identification task [5]. Although experience offers a partial explanation, other factors must contribute to individual differences in hazard identification.
There is limited evidence regarding the influence of personality on hazard identification but there is a wider literature regarding personality, risk-taking and accidents; both literatures suggest a role for attention and personality. Risk-taking, personality and accident research finds that extraversion is associated with more traffic accidents; agreeableness and conscientiousness are associated with fewer accidents in general [9]. These associations have been explained as a result of vigilance, care and arousal associated with both personality traits and accidents. Actual hazard identification evidence confirms greater visual fixation, and therefore attention, with conscientiousness and less with extraversion [10]. It may be that personality differences in hazard identification are partly explained by individual differences in attention.
Attention has been directly linked to hazard identification in further empirical evidence. Auditory attention is associated with reduced vehicle accident involvement [11]. Eye movements are associated with visual hazard identification [12]. Distracted workers recognised fewer hazards [13]. We also know that motivation and attention are linked [14]. Supervisory feedback is effective in adjusting attention and safety behaviour [15–17]. Therefore, motivation and conscious control offer an opportunity to intervene and increase hazard identification.
One promising avenue for an experimental intervention on hazard identification is via anchor-based goal-setting. We know that vigilance can be improved by goal-setting (via feedback on expected and attained performance) [18]. Setting challenging goals compared to easy or no goals is a highly robust method to motivate attention, effort and performance [19]. Even very high, arbitrary goals set by anchors are able to increase self-generated goals and maintain commitment and performance [20]. In one study participants were asked to generate ideas either to “come up with as many uses as you can” or “set a challenging and specific goal for the number of uses you will generate during this next period; for example, 10/120/240”. Performance with higher anchor-based goals was much higher than in the low or no anchor condition [20]. It is this conceptual framework which we have adopted in this study. Challenging goals should motivate greater attention, effort and performance in the domain of risk assessment, therefore hazard identification should also improve with high compared to low anchors.
Hazard identification may be open to manipulation using goal-setting anchors; it may also be influenced by personality factors. A vignette task for hazard identification offers the opportunity to examine people’s ability to make risk assessments. This is crucial to understand as a first step in risk assessment. It is also important to consider non-expert risk-assessors because in the case of small businesses in non-risk-critical industries or micro risk assessments (such as student internships) hazard identification will, in all likelihood, be carried out by non-experts. In previous goal-setting tasks and in risk assessments the number of valid hazards identified offers a measure of performance. We can also consider time spent and words written as supplementary measures of task performance and engagement.
If goal-setting is successful in affecting participants hazard identification performance this has a number of potential benefits. Primarily as a training tool – participants could be made aware that simply by looking for more hazards they would be likely to be successful. It is possible to introduce unreasonably high anchors non-directively to motivate greater performance [20]; however, great caution should be used in using anchors in a genuine risk assessment – too low and genuine hazards might be missed, and too high and the risk assessment may appear untrustworthily incomplete. Nevertheless, if this cheap and powerful technique is effective it could be vitally useful in improving risk assessments and training.
The following objectives will be considered: the effect of high or low goal-setting anchors on number of hazards identified in a risk assessment, time spent on a risk assessment and words written in a risk assessment; the effect of personality traits of conscientiousness, agreeableness, extraversion, neuroticism and openness to experience on number of hazards identified.
We would expect a higher anchor to increase time spent on task, words written and hazards identified. We would expect conscientiousness and possibly agreeableness to be associated with more hazards identified and extraversion might be associated with fewer hazards identified.
2. MATERIALS AND METHODS
2.1. Participants
A sample of UK undergraduate students were invited to take part in the study via either a social media advert or through university participant recruitment software offering course credit. The final sample consisted of 72 participants. Seventy-eight participants completed the survey; one participant withdrew, two failed the attention screen question and one was excluded for completing the survey in under one third of the median time to complete (median completion time for the whole survey was 21 min). Two further participants were excluded as outliers (± 3 SD) in time to complete – both took well over 2 h to complete the risk assessments alone – more than double the time of the next nearest participant. The final sample was 77.8% female with an average age of 23.3 years (SD = 9.34).
2.2. Materials and Procedure
Participants were directed to online consent and study information. They then completed four risk assessment vignettes in random order. Each vignette described a different workplace role – mechanic, hairdresser, nightclub security and newsagent. Participants were asked to identify hazards within the role using an unlimited free text description, based upon the vignette. The vignettes were based on examples provided by the UK Health and Safety Executive [3]. At the end of each vignette participants were asked:
“What risks/hazards can you identify to the role holder and public in the vignette above? Experts usually find at least [2/8] hazards.”
Half of the participants were given the low anchor value (two hazards) for all vignettes and half were given the high anchor value (eight hazards) for all vignettes. Participants were subsequently asked to complete a 44-item 5-Factor personality scale – the Big Five Inventory [21] on a 5-point Likert-type scale from disagree strongly to agree strongly. The scales measure Extraversion, e.g. “Is talkative” (α = 0.86); Agreeableness, e.g. “Likes to cooperate with others” (α = 0.82); Conscientiousness, e.g. “Does a thorough job” (α = 0.78); Neuroticism, e.g. “Worries a lot” (α = 0.83) and Openness, e.g. “Is inventive” (α = 0.74). All items were presented in a random order within the scale. The final questions asked about demographics and an attention screen: “How many Risk Assessment Vignettes were there in this survey?”.
Participant risk assessment responses were coded by two independent raters using the same coding scheme to count the number of valid hazards identified by each participant. The coding scheme was based upon an initial list of valid hazards devised during construction of the vignettes and updated iteratively to include any hazards identified by participants which were not on the initial list and deemed valid by the raters. Each vignette included between 9 and 13 valid hazards to be identified (see Table 1). All 72 participants’ responses were coded according to 46 hazard categories across the four vignettes leading to 3312 coded items. The two raters coded the vignettes, blind to the experimental condition codes, with an initial 3044 agreements and 268 disagreements (i.e. 91.9% agreement). The kappa for the two-raters pre-agreement hazards identified inter-rater reliability was κ = 0.81, p < 0.001, which indicates substantial agreement [22].1 A final inspection of all cases of rater-disagreement was conducted and a final coding was agreed between raters for all cases. This coding was aggregated for each participant to give a score for total hazards identified across all vignettes, which was used in the analysis.
| Vignette (total possible hazards) | Condition | N | Mean (SD) | Min | Max | t |
|---|---|---|---|---|---|---|
| Newsagent (12) | Low anchor | 35 | 2.1 (1.40) | 0 | 5 | 2.53* |
| High anchor | 37 | 3.0 (1.55) | 0 | 6 | – | |
| Nightclub (13) | Low anchor | 35 | 2.8 (1.64) | 0 | 6 | 4.01** |
| High anchor | 37 | 4.4 (1.64) | 2 | 8 | – | |
| Hairdresser (9) | Low anchor | 35 | 2.6 (1.22) | 0 | 5 | 4.50** |
| High anchor | 37 | 3.8 (1.17) | 2 | 6 | – | |
| Mechanic (12) | Low anchor | 35 | 3.5 (1.60) | 1 | 7 | 1.91 |
| High anchor | 37 | 4.3 (1.88) | 0 | 9 | – |
p < 0.05,
p < 0.001.
Averages and variability across the anchor conditions for hazards identified for each of the four vignettes
3. RESULTS
In addition to the total number of hazards correctly identified by each participant (as coded by two independent raters), two other measures of participant behaviour on the risk assessment task were calculated: (1) the total word count for the participant-response descriptions of the potential hazards and (2) the total time on task taken to identify hazards on the four risk assessment vignettes. A mean of 13.3 hazards (SD = 5.4) were identified by each participant across the four vignettes. Total word count (Mdn = 196 words, inter quartile range (IQR) = 109–257 words) and total time on task (Mdn = 19.9 min, IQR = 11.0–17.4 min) were positively skewed.
The effect of the anchor condition (low anchor = at least two hazards versus high anchor = at least eight hazards) was tested on hazards identified by vignette (Table 1), total hazards identified, total word count and total time on task (Table 2). We can see that there was a trend toward greater hazard identification in the high anchor condition for each of the four vignettes (Table 1). This difference was significant for three of the four vignettes. While some participants identified no hazards, only one found more than eight and no participants identified all of the possible hazards. There were significant effects of anchor condition in which total hazards identified and word count were higher with the high anchor, but there was no effect on time on task – see Table 2. There was no significant difference between self-reported personality scores across anchor conditions (t < 1.5).
| N | Hazards identified | Word count | Time on task | |
|---|---|---|---|---|
| Mean (SD) | Mdn (IQR) | Mdn (IQR) | ||
| Low anchor | 35 | 11.0 (4.87) | 154 (90–233) | 15.2 (7.7–21.6) |
| High anchor | 37 | 15.5* (5.05) | 228† (157–296) | 17.4‡ (12.7–29.1) |
t(70) = 3.77, p < 0.001;
U = 415, z = −2.62, p = 0.009;
U = 510, z = −1.55, p = 0.121.
Averages and variability across the anchor conditions for outcomes of the hazard identification task
The associations between the personality factors and the hazard identification tasks were examined using Spearman Rho correlations, see Table 3. Conscientiousness was associated with time on task, word count and hazards identified. Word count was highly correlated with time on task, and hazards identified; hazards identified and time on task were moderately correlated. Anchor condition was associated with word count and hazards identified. The pattern across the individual vignettes was similar to the total hazards identified.
| 1 E | 2 A | 3 C | 4 N | 5 O | 6 AC | 7 ToT | 8 WC | ||
|---|---|---|---|---|---|---|---|---|---|
| 1 | Extraversion | – | – | – | – | – | – | – | – |
| 2 | Agreeableness | 0.219 | – | – | – | – | – | – | – |
| 3 | Conscientiousness | 0.157 | 0.160 | – | – | – | – | – | – |
| 4 | Neuroticism | −0.408** | −0.292* | −0.156 | – | – | – | – | – |
| 5 | Openness | 0.133 | 0.117 | −0.150 | −0.090 | – | – | – | – |
| 6 | Anchor condition | −0.157 | −0.027 | −0.094 | 0.131 | −0.045 | – | – | – |
| 7 | Time on task | −0.102 | −0.002 | 0.262* | 0.009 | 0.107 | 0.184 | – | – |
| 8 | Word count | 0.004 | 0.124 | 0.316** | 0.009 | –0.075 | 0.311** | 0.599** | – |
| 9 | Total hazards identified | 0.008 | 0.075 | 0.256* | 0.088 | 0.046 | 0.373* | 0.406** | 0.672** |
| 10 | Newsagent hazards | −0.047 | 0.068 | 0.235* | 0.150 | 0.102 | 0.281* | 0.267* | 0.464** |
| 11 | Nightclub hazards | 0.000 | −0.012 | 0.297* | 0.043 | −0.056 | 0.424** | 0.392** | 0.628** |
| 12 | Hairdresser hazards | 0.034 | 0.161 | 0.143 | 0.094 | 0.066 | 0.459** | 0.412** | 0.608** |
| 13 | Mechanic hazards | 0.050 | 0.118 | 0.090 | 0.002 | 0.086 | 0.216 | 0.286* | 0.584** |
p < 0.05,
p < 0.01.
Non-parametric, Spearman’s Rho correlations between personality factors and hazard identification tasks
In tests of regression assumptions one influential case was found.2 The regression analysis reported below omits the influential case (see Table S1 for the regression analysis including the influential case). The preceding analyses were not substantially affected by inclusion/exclusion of the influential case and are reported with the case included.
The initial model was run with the five personality factors and the anchor manipulation (coded as 0 and 1 for low and high respectively), see Table 4. Conscientiousness was associated with significantly more hazards identified as was the high anchor condition. A second model was run including time on task and the total word count used in addressing the task; these significantly improved the regression model. In the second model word count and high anchor condition were associated with more hazards identified; conscientiousness was no longer predictive of hazard identification.
| Model | Variables | B | 95% CI | SE | Beta | t | p | F | R-squared | R-squared change |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Constant | −9.726 | (−24.246, 4.794) | 7.268 | – | −1.338 | 0.186 | 4.321 | 0.288* | – |
| Extraversion | 0.019 | (−0.182, 0.220) | 0.101 | 0.022 | 0.185 | 0.854 | – | – | – | |
| Agreeableness | 0.065 | (−0.126, 0.257) | 0.096 | 0.077 | 0.683 | 0.497 | – | – | – | |
| Conscientiousness | 0.284 | (0.063, 0.505) | 0.111 | 0.290 | 2.569 | 0.013 | – | – | – | |
| Neuroticism | 0.116 | (−0.109, 0.341) | 0.113 | 0.123 | 1.026 | 0.309 | – | – | – | |
| Openness | 0.176 | (−0.027, 0.380) | 0.102 | 0.191 | 1.728 | 0.089 | – | – | – | |
| Anchor condition(0 = low, 1 = high) | 4.929 | (2.578, 7.280) | 1.177 | 0.454 | 4.188 | <0.001 | – | – | – | |
| 2 | Constant | −5.337 | (−17.086, 6.412) | 5.878 | – | −0.908 | 0.367 | 9.709 | 0.556** | 0.268** |
| Extraversion | 0.025 | (−0.138, 0.189) | 0.082 | 0.030 | 0.309 | 0.759 | – | – | – | |
| Agreeableness | −0.009 | (−0.165, 0.147) | 0.078 | −0.010 | −0.111 | 0.912 | – | – | – | |
| Conscientiousness | 0.153 | (−0.031, 0.337) | 0.092 | 0.156 | 1.663 | 0.101 | – | – | – | |
| Neuroticism | 0.074 | (−0.109, 0.257) | 0.092 | 0.079 | 0.805 | 0.424 | – | – | – | |
| Openness | 0.146 | (−0.022, 0.315) | 0.084 | 0.159 | 1.733 | 0.088 | – | – | – | |
| Anchor condition(0 = low, 1 = high) | 3.236 | (1.241, 5.230) | 0.998 | 0.298 | 3.243 | 0.002 | – | – | – | |
| Time on task | −0.020 | (−0.108, 0.068) | 0.044 | -0.048 | −0.453 | 0.652 | – | – | – | |
| Word count | 0.027 | (0.017, 0.037) | 0.005 | 0.587 | 5.474 | <0.001 | – | – | – |
p < 0.01,
p < 0.001.
Regression analysis predicting total hazards identified from personality factors, anchor condition (low or high), time and words used on task
4. DISCUSSION
Anchor-based goal-setting was effective in increasing the number of hazards identified in a range of vignettes. When participants were asked to make risk assessments they wrote more and identified more hazards when told that experts find at least eight rather than at least two hazards. Unsurprisingly, word counts, time on task and hazard identification were all positively associated with each other. Greater conscientiousness was associated with greater hazard identification and a higher word count in the risk assessment. The effect of the anchor was additional to the effect of word count, time on task and personality.
The use of anchors to set goals appears to be a simple and effective manipulation to improve the hazard identification stage of risk assessment. In this study the higher anchor caused more words to be written in the attempt to identify hazards. However, the effect of the anchor on hazard identification remained even over and above the effect of word count. Not only did the anchor appear to increase participant effort in writing more words to try to identify hazards, but more hazards were identified even when the effect of number of words was accounted for in the regression. It may be that people do not know how many hazards to expect to identify and that the anchoring cue provides a goal to aim for. We know that challenging goals increase performance [19,20]. In this study without a baseline condition we cannot be sure if a target of at least two hazards reduced performance or if at least eight hazards increased performance. Only three participants identified eight hazards (i.e. met the target) and only one participant exceeded the target by identifying nine hazards (all in the high anchor condition) and it therefore seems unlikely that the high anchor provided a ceiling to hazard identification. However, a higher more aspirational high anchor might be appropriate in future research. A pilot of four participants prior to the study indicated that participants identified an average of 2.5 hazards per vignette when asked to complete the task without being exposed to an anchoring figure. By comparison, participants in the low anchor condition identified an average of 2.75 hazards per vignette, whilst those in the high anchor condition identified an average of 3.88 hazards per vignette. Further research should examine the effect of different levels of anchors compared to a baseline with no anchors.
Personality did have some influence on hazard identification. The effect of conscientiousness on hazard identification is consistent with the research suggesting that greater conscientiousness is associated with fewer accidents. This might be attributed to greater effort, attention or motivation on the task. This is also consistent with the finding that conscientiousness was associated with word count and time on task. Although hazard identification is not the only reason conscientiousness might be associated with fewer accidents – hazard awareness is one very important contributor to accident avoidance (in addition to care around known hazards). It is also noteworthy that higher word count, potentially indicative of more effort, did not produce more irrelevant or inaccurate hazard suggestions – as evidenced by the highly positive correlation between word count and hazards identified. This suggests that any technique which might increase effort would be useful in improving hazard identification in risk assessments. Further research might investigate the association of effort with conscientiousness by looking closely at information search and anchored goal-setting and conscientiousness as part of a hazard identification task.
There are a range of possible applications of this finding. It would be useful to habitually inform trainee risk assessors that if they aim to identify a very high number, for example 25 hazards, they are more likely to get a comprehensive list. Further testing could confirm if this is effective for even extremely unlikely anchors, for example 1000 hazards. Prior research would suggest that even extreme goal-setting anchors are effective [20]. If extreme anchors are effective it is possible that this approach could be extended beyond training to actual risk assessments without the concern that too low an anchor would cause hazards to be missed. It is also possible that this approach might influence expert risk assessors and this could also be examined in future research.
There are some limitations to this study, for example, the sample is relatively homogenous with most participants being young with limited work experience. The participants are not experts and are being asked to identify hazards from settings with which they may be quite unfamiliar. The vignettes and hazard classifications were developed for this study and the online nature of the task led to noisy data – particularly for the time on task measure. There was no control on participants who may have been distracted while completing the study. However, the results show a clear pattern despite this noise, and the analysis across vignettes and typical time on task are largely as expected. Furthermore, future studies might consider other relevant factors in hazard identification – the effect here was primarily driven by anchor-based goal-setting and it may be that constructs such as self-efficacy are useful in understanding differences in goal-setting [24].
Overall, this paper demonstrates the usefulness of anchors in hazard identification. Important risk assessments are often carried out by non-experts. The first, crucial, step of risk assessment is hazard identification. This study demonstrates that hazard identification is much improved by high-anchor goals. Appropriate bodies should consider using anchors to prompt increased hazard identification performance as a low-cost intervention.
CONFLICTS OF INTEREST
The authors declare they have no conflicts of interest.
AUTHORS’ CONTRIBUTION
HE designed the study, collected the data and co-wrote the report; PF supervised the project, analysed the data and co-wrote the report.
SUPPLEMENTARY MATERIAL
Supplementary data related to this article can be found at
Footnotes
Re-analysis of the results using the rater 1 or 2 coding does not substantially affect the results obtained.
The case produced a Mahalanobis distance score of 26.7 and DFBETA for neuroticism = 1.3 which provide grounds for further consideration [23]. The regression analysis was run with and without the influential case. The coefficient for neuroticism was significant only when the influential case was included, it became non-significant when the influential case was omitted.
REFERENCES
Cite this article
TY - JOUR AU - Piers Fleming AU - Harry England PY - 2020 DA - 2020/11/02 TI - Anchor-based Goals and Personality Effects on Hazard Identification in Risk Assessment JO - Journal of Risk Analysis and Crisis Response SP - 113 EP - 118 VL - 10 IS - 3 SN - 2210-8505 UR - https://doi.org/10.2991/jracr.k.201014.001 DO - 10.2991/jracr.k.201014.001 ID - Fleming2020 ER -