Introduction

A system dynamics simulation model of occupational safety has been developed. It is built using the idea that a number of systems archetypes are at work in occupational safety (Moizer, 1994). From a simple causal diagram, a generic simulation model has been developed and subsequently validated using data from a medium-large manufacturing site in central Scotland (Moizer, 1996).

Ithink 3.0 was the system dynamics package used to build the quantitative model. The simulation model has been validated with company data stretching back in monthly time steps over a three year period. There was a close match between the model outputs and historical data. This was backed up by a number of structural, behavioural and policy tests suggested by Forrester and Senge (1980) and Sterman (1983). Several different verification tests confirmed that the model was representative of the safety system which emerged, both behaviourally and often on a point-by-point basis. The validated model met with the managers approval. They were satisfied that the model replicated the behaviour of their own occupational safety system. The simulation was set up to run from the present to three years into the future. The purpose of the study was to capture the managers opinions on the potential of the model in assisting their company and the wider business world in dealing with occupational safety.

The managers were neither involved in the collation of data or its analysis, nor in initialising the model. The finished product could be regarded by the company as a 'black box' model.

Three managers were presented by the model builder with an overview of system thinking and were taken in summary form through the model's conception, construction and operation. The interface shown in figure 1 was revealed to them. The managers were then asked to become players and experiment with the safety game simulation. They were able to follow through the instructions, and quickly involved themselves in model experimentation. As you can see from the interface shown in figure 1, none of the construction layer of the model was visible to the players. The slide bars represented policies. The players were given the opportunity to make policy decisions simply by dragging these bars back and forth to either switch them on or off, or heightening or lessening their effect.

The three players experimented or 'flew' the simulation. They were then asked to discuss the suitability of the model as a learning or policy tool.

A facilitated group discussion was conducted with the three managers. They each held very different positions within the company. The first, and most dominant interviewee was the plant safety professional. He was a graduate engineer with twenty years engineering and safety experience, and was very adept at building large databases and spreadsheet models. The second was a long-serving senior line manager, having worked for thirty years at the same plant. The last was a young management trainee who was a graduate engineer and had been with the company for one week. This mix allowed the interviewer to obtain the respective opinions of a group consisting of people from different backgrounds.

The qualitative method applied to gain an understanding of the processes used to evaluate the simulation model's potential role allowed a reasonably unbiased picture of the groups opinion to emerge.

Initial Responses to the Simulation Model

The discussion opened up with the general question;

"Is the occupational safety model, a learning or policy tool ?" [Interviewer]

This was followed up by the more direct question;

"How might the simulation model assist your company ?" [Interviewer]

The initial responses were lengthy and encouraging;

"Well I think it could be used by managers or anyone involved in the field of safety to learn about causes and effects, and what can happen if we change anything in particular, how that might affect the overall picture . . .

It would be used to try and explore what the most effective measures would be . . .

We would follow the path that the model suggested was the best path and monitor the effect to see if the two were in agreement . . .

If successful then we could use it [the model] to look at the future so that we could set policies so we weren't stabbing in the dark, but initially very much a learning tool and probably less of a policy making tool." [Safety Professional]

"I think this can certainly be as in this company . . . a way they can see what it [safety] is going to cost . . .

It would certainly highlight that it [following the model's policies] would save a lot of injuries, but at the same time can save money and that makes good management sense." [Production Manager]

"I've got a degree of scepticism, especially when looking or basing future policies on those sort of results. I wonder how it would differ from department to department, where we consider accidents do occur frequently that are basically unavoidable . . . it looked good, but that would be my one worry." [Management Trainee]

Here three different views of the model unfold. The safety professional appeared to have a good layman's understanding of how the model worked, and what its potential and limitations were. He saw the model as suitable for teaching people about the effects of different policies as a first staging post, and the potential to use it to assist with strategic decisions, although most of his comments suggested using the model as a strategic evaluation tool.

The line manager immediately saw the model in a demonstrational capacity, describing how you could show people the effects of safety upon accidents and their requisite costs. This appeared to be a double-edged response, acknowledging the use of the model to stress accident reduction and cost minimisation.

Lastly, the trainee, being new to the company and occupational safety could obviously make a lesser contribution to the debate. His comments may suggest that he had a lower understanding of the holistic nature of the model, and was querying specific operational detail. He may not have appreciated the reason for aggregating data, i.e. to obtain a manageable overview of the principal variables in this complex safety system.

The purpose of the model was then re-iterated to the group. That being to capture an understanding of the emergent behaviour of the system, rather than examining the numerical outputs on a point-by-point basis.

Discussion of the Principal Model Policy

The group had discovered from experimenting with the model that from the policies at hand, training appeared to exert the most leverage over both accidents and safety costs. This allowed the policy to be given the lions share of verbal evaluation.

The model had stimulated the group to debate about the role of training and its effectiveness, and they were able to debate this issue at length. A discussion about the various training approaches ensued. This allowed the group to query the validity of the model. Questions were raised as to why all the training mediums had been aggregated into one policy;

"You could decide to spend double the amount of time on training . . . the model doesn't know how effective that training is. Inappropriate training you would expect it to have a minor effect, whereas, better targeted training would obviously be more effective. Now it's probably too much for a model to be able to pick up." [Safety Professional]

"If we've got a lot of duff [poor] training courses, then really were only ourselves to blame." [Production Manager]

It was again pointed out to the group that the model was not an operational one, rather it worked at the strategic level, examining wide policy areas;

"You could have broad bands of training, you've got training where you go away on a course off site, or if it's on site you're isolated from your work environment . . .

That would be external training . . .

Off-the-job training is a defined course that covers topics, and you go back to your job and it might not change the way you work. On-the-job training should, because you're doing it on-the-job, change the way you work, and in many cases the effects could be significantly different." [Safety Professional]

A debate had opened up about the validity of the model. This concerned the aggregation of internal and external training. An understanding developed of the impact of different training approaches upon safety behaviour. The group made it clear that to disagregate the types of training could more accurately reflect real policy.

This debate could be regarded as very pertinent to the argument that the model was focused at policy making, because the discussion had gone beyond the emergent system behaviour and onto the actual structure of the model and how it should best have been initialised. The group was trying to find a way to most effectively evaluate their training policy using the model of their work environment. Building a satisfactory structure to fit their firm took precedence here over learning about the general effects of training.

The Effect of the Interface on the Model's Application

The group were again asked for their opinions on the model's effectiveness in enhancing learning and policy-making. The actual interface was brought into the discussion, and its user friendliness was brought to their attention. They were then queried about the interface's importance within learning or policy-making situations;

"The problems you would face there are the scepticism. If they're [users] not familiar with computers or they don't understand modelling . . .

They have to be comfortable with the process and understand what it is setting out to achieve . . . the model has been developed to the stage that you can use sliders and check graphs. Depending on the audience you're trying to reach that will only reach a certain proportion of them . . .

Ideally you want something which will work in an interactive way which that [the model] does . . .

With someone at a lower level of management or supervisory level, you could play with that and it might be instead of a chart it would show a pile of dead bodies. So that they could visually, not just on a graph get an appreciation . . . so taking that [the model] a stage further . . .follow the same as a flight simulator and keep developing that, if the aim is to develop a package that can be used for training." [Safety Professional]

The group were asked about the interface for the model acting as a policy tool to assist with resource allocation;

"It doesn't need to be pretty to do that, just understandable." [Safety Professional]

The group were able to differentiate between the needs of the user interface for learning, where it was made clear that the outputs needed to be more visual than graphs, and the low priority of aesthetics to the planner. Useful suggestions were made as to how to improve the learning experience, for example, the dead bodies piling up instead of a graph unfolding. The slide bars and table functions close at hand were appreciated. This suggests that the group were satisfied with the current user interface for ease of policy experimentation, but as an aid to supervisory learning it needed to be more visual.

The Behaviour of the Model

The interviewees were asked whether they were surprised about any of the behaviour of the simulation output;

"It's interesting to see which ones [policies] affect [the behaviour of the model], and I suppose we could concentrate on basically the ones which we could influence the most, quickly and cheaply . . . get as many people involved." [Production Manager]

"Getting over the credibility gap, let's say you've got a group, say we looked at departments one and two, and you have that group of managers and supervisors, they think the model's credible, we've reached that point it would allow them to understand safety." [Safety Professional]

The safety professional describes how he argues often with the production management group about safety issues, and they complain that they can't fit enough guards on machines;

"Sure if it should be guarded it should be guarded, but to expect that to make a tremendous inroad into accident rates is false. It's what people do that cause accidents. A blend of on-the-job and off-the-job training is required . . . it is generally accepted that training has that benefit, and you have a factory where the guarding is not adequate and hardly any accidents, and then you can have the converse where you can have everything guarded and lots of accidents. The difference there is the people." [Safety Professional]

Comments were made about the interesting nature of the model's output, and the ease with which policy experimentation could be conducted. The safety professional agreed that he was aware from experience that training would have a major impact upon safety, and that the model confirmed his opinion.

How to Convince Senior Management of the Model's Worth ?

Whether strategies identified in the model could ultimately be pursued would depend on the safety budget. This would be set by the senior accountants and board members. Safety would have to be sold as a cost centre to these people. The group was queried as to their perceived judgement of senior management's response towards the usefulness of the model;

"if you've got a group of directors . . .they are not at the technology end. So for a start it's a computer and they're not entirely familiar with that and then you've got the scepticism about modelling which would be a concept with which they would maybe not be familiar. I think there would be a point where they would say 'Well very interesting but I'm too busy, go talk to someone else'. So before you've got over those two hurdles to get the benefits of the model they might have gone and lost interest . . . they've got to the bullet points [model summary], they want to be there, if the bullet points don't confirm their pre-digested thinking, their own prejudices and beliefs, then you're obviously wrong and they move onto something else." [Safety Professional]

"But saying that if the bait's taken with a few bullet points then all of a sudden you find that they're running very fast to the beat . . .

The best place to target is the accounts, accountants." [Production Manager]

"That might be the best place to start . . . general mistrust about computers, no knowledge about modelling. I think that's where we are, and most companies might be like that." [Safety Professional]

The group offered a range of opinions on the perception of senior managers. To use the model in a demonstration capacity seemed to be the opinion of the group. Selling the concept and the power of the model for safety evaluation would have to be done carefully, as there appeared to be some doubt about the open mindedness of the senior managers.

Exploring Alternative Strategies Not Covered by the Model

The group were questioned as to whether they believed that the model policies covered the fundamental influences on safety;

"What about incentive schemes ? You could have that as an additional policy" If you had a slider bar, pounds (£'s) per month per employee invested in the safety scheme. Then you've got a cost . . .

It could have a similar effect to training." [Safety Professional]

This statement from the safety professional strongly suggests that he has a good grasp of the structure of the model despite having minimal contact with it. He was able to identify a key input as units per unit per time. He saw the potential to evaluate policy by the addition of a viable policy and identified its dimensions. This was an observation about how the model could be re-engineered in order to improve policy evaluation;

"Management competence . . . their competence in safety could be measured and some term found which you could vary to influence the model . . .

If you've got ignorance on the part of the senior people and an unwillingness to act that would have massive effects on how your company performs, well in any sphere." [Safety Professional]

It was explained that management competence was indirectly included in the model as the policy decisions actually taken would be those of the management.

The Model as an 'Edutainment' Game

The role of the model as an 'edutainment' game was raised and responses were short but encouraging;

"somebody who is enlightened in safety using it [the model] as a policy tool, whereas, a person from a more general background using it as an educational or edutainment tool ?

I think as an edutainment tool people would be more inclined to play with it [the model], and it's playing with it you actually learn." [Safety Professional]

The observation that the longer people played or experimented with the model, the more they would learn was very astute. This shows that the manager believed in the ability of the model to help people to learn about safety through teaching themselves.

Optimisation of the Model

Optimisation was an area brought up directly by the group. They were probing to discover if the model could be optimised, and if so how ?

"Could it be programmed to work out itself what the most effective variables are in the model ?" [Management Trainee]

"An optimising program, would you set the target of minimising all costs ?" [Safety Professional]

It was explained that the model had been manually optimised to identify a policy mix which would minimise costs. It was pointed out that optimisation programmes were available with certain system dynamics packages.

The question of removal of human interaction by using an optimisation program was raised by the interviewer. It was suggested that people may not get the opportunity to explore policies if they were to let the computer make the decisions for them;

"If there was a function you could just switch on and off then you could just use it for both . . .

Like for somebody who was familiar with the model and believed what it told them . . . he'll want to run the optimisation won't he ? He won't want to spend hours in front of the keyboard . . .

But you want the learning course because it might be that the optimisation will have to take account of the amount of money you have available or the amount of time, so you want to optimise given that these inputs must be maintained below this level. So what can we do with the resources we've got ? . . .

"That's an interesting philosophical debate. Do you want to optimise on accidents, which is zero accidents, or do you optimise on cost and you might not choose zero accidents ?" [Safety Professional]

Optimisation was seen as useful for policy making, in fact an interesting discussion ensued as to whether the model should be optimised on accident rates or safety costs. Certainly the group were thinking here purely in policy-making terms.

Who would use the Simulation and for what Purpose ?

One question posed to the group was;

"Would they use the model to enlighten other people ?" [Interviewer]

"From top to bottom really" [Production Manager]

"I think there's still work to do on enlightening, because we can still slip back into 'Well I can't get enough fitters to fit the guards . . . I can't do anything about safety.' . . .

It would be for anybody to come and use it, it would be for learning, once you've achieved that aim, you could then use it by adding features to mould it into a policy tool, but first of all it must be a good learning tool" [Safety Professional]

Abstract or 'Real World' Model ?

The group were asked their opinion on using an abstract model or one validated with their companies data;

"It [an abstract model] would still help us to learn and we might discover policies that we would not normally discover." [Safety Professional]

"If you took it off a really successful firm, like a Japanese leader, something not necessarily in this industry." [Management Trainee]

"You could switch on the generic model, which is designed to show how safety works for any company . . . introducing policies and switching them on. If people played with that the next thing they would want is something they could use for their own situation." [Safety Professional]

Summary of Discussion Findings

A wide range of views were put forward by the three interviewees, all possessing varying knowledge of occupational safety and simulation models. The common thread that can be teased out of the discussion is that the model in its present form was found to be helpful by the group for understanding the firm's occupational safety system.

Many of the explicit observations made by the group pointed to the model being more suitable as a tool to either demonstrate the effects of safety policy, or to assist people in learning more about the company's safety system. There was acknowledgement that the simulation would still be of value in learning or even policy making when set in an abstract context, although there was a greater appreciation of the model in its present 'real world' form. Much of the underlying discussion pointed towards using the model to assist with policy evaluation. Suggestions were made concerning the introduction of other policy parameters into the model.

The results of the policy experiments that the group conducted were certainly pertinent to the discussion. Training was suggested by the model as the policy most able to exert either a virtuous or vicious effect over the whole system. The model had allowed the interviewees to appreciate this and much debate had followed as to how training might best be used.

The group had considered carefully the initial query put forward, i.e. learning or policy tool ? This allowed the interviewer to elicit quite a rich picture of opinion on the uses of the model in their safety environment and beyond.

References

Forrester, J.W. and Senge, P. 1980. Tests for Building Confidence in System Dynamics Models. ed.'s A.A. Legasto, J.W. Forrester and J.M. Lyneis. In Studies in the Management Sciences: System Dynamics 14: 209-228. Amsterdam: North-Holland.

Moizer, J.D. 1994. Safety Hazard Control in the Workplace: A Dynamic Model, Information Systems. Proceedings of the 1994 International System Dynamics Conference, Stirling, UK.

Moizer, J.D. 1996. Safety Knowledge and its Influence over Accidents in the Workplace, Proceedings of the 1996 International System Dynamics Conference, Boston, MA.

Sterman, J.D. 1983. Appropriate Summary Statistics for Evaluating the Historical Fit of System Dynamics Models, Proceedings of the 1983 International System Dynamics Conference, Pine Manor College, Chestnut Hill, MA.

ISDC '97 CD Sponsor