|
|
9 March 1993
Doc. 6792
REPORT
on safety-critical systems
(Rapporteur: Mr Fulvio CACCIA
Switzerland, Christian Democrat)
Summary
Just when it seemed that nothing could halt the development of computerised systems, which was accompanied by unshakeable confidence in mathematically proven techniques, doubt has crept into the mind of the public. Many recent accidents (for example, plane crashes due to computer failure, malfunctioning robot killing a mechanic, patient dying because of malfunctioning of computer-controled intravenous drip) have called into question the reliability of these systems. At present, there are no specific regulations governing safety control of these systems.
Although all human activities involve a degree of risk, the safety issue should be regarded as an area of public interest which must take precedence over industry's economic and budgetary concerns.
For this reason, the Assembly proposes among other things, that an inventory of selected accidents due to these systems be instituted, that a list of existing national and international standards be drawn up and that a glossary of technical/legal terms be compiled to ensure that future legislation corresponds to the reality of technological systems. The aim, therefore, is to lay down guidelines reflecting the technical complexity of the concepts and systems concerned.
I. Draft recommendation
1. The Assembly is aware that the use of computers in safety-related applications is growing, particularly in areas such as control systems of aeroplanes, high-speed trains and nuclear power stations, medical equipment and medical records, anti-lock braking systems for vehicles and machine engineering in general, and last but not least, modern weapons and their guidance systems.
2. Many recent accidents (for example, plane crashes due to computer failure, malfunctioning robot killing a mechanic, patient dying because of malfunctioning of computer-controlled intravenous drip, rocket launch failure traced to computer error, software piracy etc.) cause public concern and raise the question of the reliability of such systems.
3. While there is undoubtedly real concern among professionals and scientists to guarantee the safety of these sophisticated automatic systems, the fact remains that economic and budgetary priorities may at times take precedence in industry over other factors. Consequently, the safety issue should be regarded as an area of public interest.
4. Safety is a difficult concept to grasp. It is closely bound up with the idea of danger and risk. Absolute safety does not exist. All human activities involve a degree of risk and technological innovations are no exception to the rule. The important thing is to minimise the risks as far as possible and to keep the public informed.
5. In most cases safety-control systems are not covered by any specific regulations and quality control is carried out only by internal procedures specific to the industrial sector concerned. Moreover, the mathematical codes underpinning the software of such systems are often considered as commercial secrets, thus making investigation difficult.
6. It would actually be much more cost-effective if industry were to put more effort and money into the design stage, which is crucial to the development of sophisticated systems. Failing this, laying down standards, regulations and penalties could nevertheless serve as a deterrent against technological "short cuts".
7. The "Guidelines for the security of information systems", recently adopted by the OECD, give evidence of an increased awareness towards problems raised by the diverse use of computers. They focus nevertheless on aspects of data protection in order to ensure that the level of confidentiality, availability and integrity of these systems are not eroded. They should be completed by an up-stream study, namely technological reliability.
8. For the above reasons, the Assembly recommends that the Committee of Ministers call on the governments of member states to undertake an interdisciplinary activity within the OECD, and in particular its Directorate of Science, Technology and Industry, whose aim would include:
i. determining areas of public interest, including future technologies that are currently being developed such as computer/communications between road and vehicle, the pan-European railway system, etc.;
ii. instituting a European inventory and analysis of selected major accidents due to computerised systems;
iii. drawing up a list of existing national and international standards and of any specific legislation in existence;
iv. compiling a glossary of technical/legal terms so that future legislation can correspond to the extremely complex reality of the technological systems concerned;
v. working out general principles of safety-assessment methods in the early design stages of sophisticated products and systems;
vi. developing the outlines of appropriate training, which should be multidisciplinary and include in-depth study of economic parameters, for those with responsibility at national and European levels for assessing and judging safety implications of new technologies;
vii. carrying out a study that compares legislation on the determination of liability for accidents and suggests ways of improving it ;
viii. providing special measures for scientific and legal investigations in the context of rules for the protection of intellectual property;
ix. introducing, if need be and once the necessary data has been acquired through the initial research, quality control and certification systems at both national and European levels, prompted by the experience already acquired in the field of electrotechnical standardisation by the work of International Electrotechnical Commission (IEC) and of the European Committee for Electrotechnical Standardisation (CENELEC).
II. Explanatory memorandum
by Mr Fulvio CACCIA
CONTENTS
Part A : Preamble by the Rapporteur 5
Part B : Technical report 6
1. INTRODUCTION 6
2. DEFINITIONS 9
3. DESIGN, SAFETY AND REGULATION 9
4. MAIN PROBLEM AREAS 13
5. PROPOSALS FOR STUDIES 15
6. CONCLUSIONS 17
*
* *
LIST OF REFERENCES 19
APPENDIX I Industrial Sectors 22
APPENDIX II Nomenclature 24
APPENDIX III Extract from executive summary of
"Software in Safety-Related Systems" 29
APPENDIX IV Progress towards unified standards 31
Part A: Preamble by the Rapporteur
The first big shock was the explosion of the Challenger space shuttle during takeoff in January 1986. For the whole world it was unimaginable that a system developed and controlled by extremely powerful computers could blow up in full view of the engineers.
Since then, a whole series of fatal accidents, from Chernobyl to automatic pilot errors on aircraft, not to mention the failure of missile-guidance systems during the Gulf war, have shattered public confidence in these automatic systems.
As the year 2000 draws closer, and mankind thought at last it had the upper hand over technological casualties and scientists and manufacturers were assuring us that there should be no repeats of the "Titanic", here we are plunged once more into uncertainty.
Ever since the first tool was invented, man has had to battle with his own technological progress. Although the new computerised systems are highly sophisticated and complex, there is no guarantee of infallibility. What is more, it is not always easy to establish the respective share of responsibility to be borne by man and machine.
One thing is sure: the development of these systems is no longer the preserve of a handful of experts, but requires long and expensive multidisciplinary collaboration. The few laws that do exist do not reflect the technical complexity of the concepts and systems concerned.
It would be wrong to assume that we are thinking here only of major aerospace projects, etc., which do not have a a direct effect on the general public. Safety-critical systems are also to be found in our everyday surroundings: in our cars, hospitals and railway networks, to give but a few examples. Tomorrow, the symbiosis of computers and biotechnologies may have a few surprises in store for us.
The legislator is faced with a difficult choice. On the one hand, technical innovation ensures human progress and keeps the wheels of the economy turning, but on the other, the public expects reassuring measures to be taken and demands that it shall be elected representatives and not the technologists who have the final say.
In this case, there appear to be two possible approaches: either the State does not interfere with the manufacturer and penalises those responsible when accidents do occur (even then, liability still has to be established); or the State specifies and regulates everything, down the to the tiniest detail, at the risk of slowing down innovation and development.
Good sense would however suggest a gradual approach, in other words drawing up basic standards at international or even European level, closely monitoring their enforcement and setting up the necessary instruments to analyse failures in detail, and what is more, reserving the possibility of a certification system should the earlier stages end in failure.
Part B : Technical Report1
1. INTRODUCTION
The potential impact of computers on safety issues has been known for at least 30 years [1].
The reasons for the accidents (and hence the possible cures) are all very different. There is a semantic trap here which is of fundamental importance (throughout this whole report) in that the words used to label very complex engineering systems are extreme simplifications of the realities they describe, when systems under the same heading behave and fail in very different ways then it is illusory to assume common causes and worse still to seek common legislative regulation.
Although in most cases safety-critical systems are not covered by any specific legislation, enquiries show that there is great professional concern for safety and well-founded efforts are made to make complex automatic systems adequately safe. Furthermore there are already many national, professional and governmental studies and regulations in place at national and international level. Agreement across countries is better than across sectors. There are indeed formidable difficulties in dealing with the question of software secrecy and protection of intellectual property rights, but these problems are more general and need resolution in a much wider context, not just in relation to safety-critical systems.
How has the problem of safety-critical software arisen? Essentially from an ever-increasing complexity in engineering. One may compare the steam locomotive of 1830 with the APOLLO Moon spacecraft of 1970 as an example. In 1917 WM FARREN designed, supervised the construction of and testflew an aircraft - the CE 1 and with acceptable safety! [2]. Even in 1965 a chief designer would be familiar with all the decisions taken in the design of a complex product such as an aircraft or ship. The management operation was deeply hierarchical [3] , but as systems became more complex and design teams included more and more specialists it became necessary to formalise the interfaces between the specialist groups to gain benefit and yet maintain overall design disciplines. This led to the matrix design management system in the 1970s to cope with design teams 50 times larger than before [4].
There were also significant developments in technology. For example, in aircraft the flight control system was originally performed by mechanical rods, cables and links operated by the pilot: then came the automatic pilot, later the digital computer and "fly-by-wire' control. The CONCORDE's flight control is by computer but the pilot has a manual over-ride in case of computer failure. The European Fighter Aircraft, however, has all its automatic functions embodied in the computer, with no pilot reversionary mode.
As the computer took over more and more of the control of aircraft systems and with the advent of digital databus technology, it was logical to express the engineering functions in computer language throughout. The computer and its software then assumed an over-riding importance, and were inevitably extremely complex. Moreover, although specific design and performance requirements could be stated reasonably simply and with little ambiguity, the implementation of the complex software, which must be logically expressed right down to the last digit, is unbelievably complicated. Furthermore, developing the automatic system and its software required new armies of specialists, usually mathematicians and physicists, so that those with overall reponsibility for the design were no longer directly able to be aware of the details of every system in their design as they had hitherto. Such delegation of the safety function led to an explosion of paperwork with scope for increased misinterpretation.
Another factor in aviation was the steady increase in safety levels experienced in practice. The fatal accident rate is now, typically, one in a million flying hours for civil airliners [1]; somewhat more frequent for helicopters and military aircraft. There is then a fundamental problem that integrity at this level cannot be tested or proved in statistical terms.
In recent decades there has been a vast increase in documentation of the design process: the HAWKER HUNTER fighter of 1950 had a three-page specification; that of a recent USAF aircraft required a HERCULES transport to carry all its documentation. The identification and management of safety, given still improving safety levels in practice, under these circumstances requires the utmost scrutiny of what is inevitably a most complex matter. Similar matters apply to other industrial sectors and these are listed in Appendix I.
Of all the categories of manufactured product we deal here with discrete, specially designed projects such as aircraft, missiles, robots, medical apparatus etc. Failures to function that lead on to hazard to persons or the environment can arise from:-
* errors in design, not identified and rectified during the commissioning stage
* failure of components (structure or computer-driven sub-system) which lead to catastrophe
* incorrect rectification of faults in the field
* human errors of operation.
* the latter can be exacerbated by poor ergonomic design
Again, attempting to place the present situation in a long-term context: in the 1940's, (say) it was typical to find a well-established engineering company that had a stable organisation and leadership going back decades. It was accepted that its products would be reliable and cost effective, and a customer placing a contract would know that the product would be dependable. The increasing rate of change of technology and its products and a corresponding increase in performance and capability have created chaos in the procurement process, particularly in the U S. As projects were delivered late and over budget more and more new administrative restrictions were imposed in a futile attempt to bring the attainment (cost, time and performance at delivery) into alignment with claims (the contract offering). No matter how many administrative overlays were introduced (e.g build-to-cost, reliability, affordability, prototyping, life-cycle cost etc.) the rapid rate of change of technology outpaced the controls. Even the introduction of Quality Management systems is no longer seen as the panacea expected when it was introduced [5].
We have also to recognise the 1 billion fiasco of the Airborne Early Warning Aircraft "NIMROD", which, in spite of more formal documentation than ever before, had to be cancelled because it just did not work properly [6]. If for "in spite of" we should read "because of" then we do really have a Pandora's box of trouble! This was not a safety issue, rather a case of the management control function (organisation, responsibility and paperwork) not being effectively superior to the problem it should have been designed to control.
Another example, which did have safety connotations, was the fire in the APOLLO capsule during ground tests which killed three astronauts. The subsequent examination of systems, possible failure modes and hazard revealed a number of defects which had not, at that time, materialised in service or in test, led to a major redesign of many components, sub-systems and systems.
The engineering community, in striving to reconcile quality with complexity, still continues with new methods: "Concurrent Engineering" now aims to include all participants in the design process including especially Systems Engineering and the implications of risk.
As an example of the "Jungle of Buzz-words" with which the subject is surrounded one official document (which sets out a new framework within which to develop safety standards) has to grapple with terminology like this: [7]
comprehension residual doubt & fallibility
formal mathematical modelling finite state machine diagrams
Fagan inspection hardware architecture
adequate protocol watchdogs
re-try fault recovery hot stand-by
cold stand-by graceful degradation
dynamic reconfiguration task analysis
cues prototyping
multidisciplinary reviews error-correcting codes
z transforms execution of properties
diversity of tools & people sneak circuit analysis
walkthroughs automated configuration management
safety culture administrative access controls
QMS, QA accepted mathematical inference calculus
V & V techniques boundary value analysis
error guessing error seeding
But the scope of the problem is not confined solely to the big aerospace and nuclear projects. It is much wider than realised, and extends to many activities. A simple fault in the program of a microprocessor-based vehicle weighing machine can cause a lorry to be dangerously overloaded. A fault in a chemical assay program can lead to the materials of components in safety-critical production processes having the wrong physical properties. Computer errors in medical records could lead to death or severe injury [8].
In summary; the type of problem we are considering may involve very expensive, complex machines, containing hundreds of thousands of parts, which are required to work at untestably high levels of integrity, and which depend critically on software that has to be accurate to every digit. There is the classic case of a US spacecraft which failed completely in flight because one digit had been typed wrongly in its trajectory programme. In practice there are error-detection strategems in the critical computing elements backed up by planned recovery actions [9]. A very good example of how engineering companies see a way ahead in overcoming this problem is in ref.[1] where HOWARD sets out a programme to improve civil aircraft safety by a factor of 10.
2. DEFINITIONS (See Appendix II)
One of the problems inherent in the subject covered by this report is the specialised - and sometimes controversial - terminology. As in all technical disciplines, scientists and engineers use words, drawn from natural language, in special ways. Unfortunately, different groups use them differently, and often in quite contradictory ways. This is especially the case in a new and rapidly developing field such as the one we are discussing [10]. Some of the more important terms are given in Appendix II [11].
The digital logic intrinsic to software enables mathematically precise definition, more so than in usual engineering practice. In many respects this is advantageous but since many matters in project evolution are unquantifiable the search for mathematical precision can cause unwanted distortions. Many engineering judgements require the balancing of irreconcilables e.g. aerodynamic qualities with structural advantages: in such judgements there is a temptation to give more weight to the quality that can be expressed explicitly in mathematical terms, e.g. performance, than handling qualities, which often cannot.*
These terminological problems become more acute in accident investigations or litigation when the technical culture meets the legal culture. One must expect some major difficulties here in the future.
3. DESIGN, SAFETY AND REGULATION
1. Design
A difficulty embodied in tackling the safety related to software in engineered products arises because of software complexity and the mathematical rigour of some parts of it distorts and clouds the fundamental processes of creative engineering design. I have alluded to some aspects of this in previous sections and, in order to set out the problem that the Council of Europe wishes to address, I find it best to embody the software issues within the total engineering process. Thereby a new concept evolves to become a manufactured entity, satisfying all the requirements for cost, serviceability, performance, delivery on time, security and safety. Most people are purchasers and users; a minority have actually been responsible for creating machines in industry. From the latter experience a distinctly different viewpoint results [12]. A user becomes familiar with the finished product and is not aware of the evolutionary steps by which it was fashioned. This stage of the engineering process is termed the preliminary design phase. At the end of this stage the paper design will be complete, a specification available from which the contract is drawn up. After this point it is very difficult and certainly extremely expensive to make major alterations. But by this stage only a very small proportion of the total contract money will have been spent: typically 1%. The consequence of this is that decisions embodied in the contract will have been taken on the basis of very little hard knowledge [13]. Estimates, together with models, simulations, partial tests and full-scale demonstrations are employed to minimise the risks of non-compliance with the ultimate objectives. These can only be established properly during the commissioning stage by which time probably 95% of the expenditure will have occured [14]. Another feature of the preliminary design phase is that none of the ultimate, irrevocable decisions is a foregone conclusion. For any parameter, configuration, sub-system or program language there will be many alternatives - up to several hundred in some instances [15]. The designer's task is then to make judgements between these alternatives, not entirely on the basis of rigorous, well-completed quantitative analysis but using incalculable judgements as well. Unless this process is fully understood it is easy to fall into the trap of proceeding with solutions which have not been rigorously assessed against a large number of alternatives [14]. Because of these features of the preliminary design process hidden faults may be embodied in the design only to be revealed later either by more detailed design work or testing [APOLLO]. A corollary is that it is in the preliminary design phase that faults can be eliminated for the least time and money - but the skills and experience to do so are not embedded in calculable software and digits! It is much more cost-effective to put more effort into the preliminary stage than to pile on more standards, rules, paperwork and penalties much later on in the product devlopment process.
The present situation has been likened to a" Mathematical tail wagging the engineering dog!"
2. Design for safety
Before discussing safety definitions and integrity a brief mention of design techniques to enhance safety. One way of increasing safety is to develop more reliable components and systems. At the outset, once the general preliminary design is defined there will be a "safety budget" allocating tolerable levels of integrity for every subsystem. Then Reliability Analysis evaluates the probability of failure and Failure Mode Effect and Criticality Analysis deals with the likely results of failure. Once the "life" of a part has been measured then the inspection and maintenance function will act to replace the part with a new one in good time. Another technique is to design an item to "fail-safe" i.e. even if it does fail it does not create a safety risk before the fault can be rectified. This has been extensively used on structures and coping with the development of fatigue cracks. "Fail- operate", "fault tolerant design" and "graceful degradation of systems" are other methods.
In complex spacecraft, where no possibility of inspection and repair can be expected once the craft is launched, the premium for avoidance of design error is so high that two independent design teams are sometimes employed. The RED/BLUE team concept may sound expensive but it is a proven means of achieving the overall health and safety of a complex automatic system.
In aircraft, for example, the replacement of heavy mechanical links by electric signalling was introduced to save weight, overcome shortcomings of the mechanical system and allow inclusion of automatic control functions [16]. But the electric lines were far less reliable than the strong rods and levers. To regain an acceptable level of safety it was necessary to duplicate, triplicate and even quadruplicate the electric signalling cables. With four parallel lines it was then necessary to include "vote-takers" that would decide which of differing signals to accept. Another technique is dissimilar redundancy where duplication is not provided by two copies of the same sub-system but two different kinds of sub-system are used.
Even with these precautions failures in electronic control systems may arise from
Design/specification fault
Computer bugs
Mismatch between specification & behaviour
Component failure, and
Electro-magnetic Interference (EMI)
A particular hazard peculiar to digital electronic systems is their susceptibility to errors caused by external electrical signals. "Spikes" of voltage variation can enter a software specified system and introduce digital errors which could prejudice safety. This has been known for some time and many techniques to overcome this by design are available. Test stations exist where systems can be subjected to agreed levels of EMI. European Regulatory Authorities are already imposing a Europe-wide requirement on EMI.
One of the difficulties with software has been that software engineering has followed a traditional route over the last decade which takes a requirements specification and progressively decomposes this in order to cope with the software design. Then comes the coding stage which is followed by a synthesis process which includes the test to demonstrate compliance with the requirements. The problem with this process is that it leads to fresh software on each design occasion and therefore the reliability of software cannot be established in the same way as a mechanism or structure which may be re-usable in many projects. A new approach to software described as "object oriented programming" may allow re-use of software which would permit its reliability to be established or improved in service. Hence the good software design features could then be inherited by future generations of systems. This then places software into a regime in which a statistical analysis of its performance can be conducted after a period of service. [9]
3. Safety
Safety is a concept which is not fixed in either space or time. Attitudes to safety, and tolerance to danger, are subjective and variable over time and circumstance [17].
Industries have attempted to address this problem by identification of more objective aspects of safety, with particular definitions of words such as 'hazard' and 'risk'. The intention is that, in any given situation, specifications of safety, whether required or claimed as achieved, should be based on these more objectively defined terms [18].
Hazard severity is categorised in many ways, one being:- [19]
NEGLIGIBLE - Events have no effect on safety
MARGINAL - Event or combination of events with some potential for adverse effects on safety and resulting injury to persons
CRITICAL - Event or combination leading to the inability to continue safe operation with a high probability of a resulting catastrophe. Events leading to serious injury
CATASTROPHIC - Event or combination leading to total loss of safety. Destruction of system and serious injuries and deaths.
Integrity level - is either a measure of the catastrophic accident rate achieved or that to be designed into a new product. This is typically in the range 10-2 for negligible hazards up to 10-6 for catastrophic hazards in e.g. transport aircraft or nuclear power stations. The units vary and may be expressed in events per hour (for ground-based systems) or for transport, the fatal accident rate per mile or per journey or per passenger mile.
Safety audits [20] are now made at different stages of design and commissioning. They include methods categorized as; hazard analysis, static & dynamic analysis, tool validation, checklists and formal methods. Formal methods, which are strict mathematical disciplines applied to the software logic, are for the most part notations rather than methods. This is not to deny the value of the notations, or the value of the disciplines which their use necessitates, nor the value of the analyses which they facilitate [21]. But we must beware a false sense of security of formal notations through their apparent, but not actual, precision. Using the terminology of Appendix I they usually provide a definite framework within which there can still be considerable lack of definition in the objects and their relationships. They have been likened to the assembly languages of specification - well-defined operators, leaving open the meaning of operands and whether the operation of the operators on the operands achieves the desired effects. The definite framework is a distinct advantage, but no panacea.
Safety is, in fact, an elusive subject to manage. It is the absence of hazard, which can usually be defined..But perceived hazards may be forbidden; it is more difficult to eliminate those that are unperceived. Just because safety levels are so high creates special problems, e.g. an airline pilot has only a 2% chance of encountering a 1 in 106hour event in his whole flying career [1].
4. Regulation
Most industrial sectors already have traditions of design procedure and technical practice to ensure and assure the integrity of their systems. Those traditions have, for the most part, proved satisfactory over the years. But those existing traditional design methods do not readily accommodate the computer software of complex computer-based control systems. Moreover, in recent times the search for common or 'generic' international standards, and for ways in which software could be accommodated have begun to pose questions about the rationale which underlies even existing practice. It is not that there is no rationale, but it is often neither explicit nor linked clearly to the procedures and techniques that are used [8].
Regulation involves legislation, standards and safety guidelines. These have evolved from different origins and are embedded in different organisations with overlapping terms of reference. There are also product liability and customer protection acts. There seems to be discordance between the extreme precision of definition in safety-critical software and the inevitably less precise procedures in the regulatory mechanisms.
Standards follow the creation of new industries, and suffer delays as international agreements are negotiated. In practice, especially in inter-company (especially international) contracts an ad hoc agreement would be reached to define which available requirements would be deemed to apply for any particular project.
There are examples where mandatory standards published before being properly considered by all parties are proving to be unworkable. Large companies have found it inefficient to attempt to make standards universally applicable.
In this complex field of design, safety and regulation it does not always follow that incremental steps towards unification will be optimal. For the Council of Europe a large question must be "what will be the most favourable means of intervention?"
Another fundamental issue which is likely to confuse progress towards unification concerns the type of law to be adopted. In the area of safety-critical software this could raise particularly severe conflict of opinion.
I must urge an over-riding need to simplify wherever possible as this should return valuable rewards.
In conclusion there are some basic flaws inherent in certain misguided attempts to maximize safety. "Absolute safety" has been demanded by politicians; it is said that "safety is paramount". What does this mean? In reality everything in life is attended by danger and hazard so why should we expect man-made devices to be the exception. Any man-created product has to satisfy a multiplicity of criteria and, like it or not, safety costs money. In balancing the criteria, safety must take its appropriate place. In spite of all, and very professional attempts to make airliners safe (at the 10-6 integrity level) accidents still happen. The DC 10 which suffered a major uncontained engine failure in 1990 lost all its powered flying controls and yet the crew made a miraculous landing saving 250 out of a complement of 350 [22]. "Pilot Error" has not been eliminated in spite of steadily improving accident rates: last century ships suffered "Captain Error" [23].It is to be hoped that in the next century we (public, engineers, regulators and politicians) will not have to live with "Computer Error". This possibility, however remote, must be mastered by well co-ordinated efforts to circumvent catastrophe. But in the end there are ultimate limits to certainty in this field. To quote reference [19] - "By definition, conformance and certification against objective tests cannot address the uncertainties of either the real world or specifications. In other words 'complete' conformance to the real world safety requirements can never be demonstrated. So the prospect of releasing into service only systems that have been 'completely' certified as safe, against objective tests, is simply, if sadly, not viable. There are fundamental limitations to the assurance that can be gained about the operation of practical, entire, safety-related systems".
4. MAIN PROBLEM AREAS
1. Benefits and problems
Computers can be more reliable than special-to-type mechanisms. Computer programs can provide more flexible, sophisticated and responsive functionality. Those who can manage the application of computers in safety-related roles stand to gain considerable competitive advantage. Those who cannot will find it increasingly difficult to compete - apart from the risks they run [30]. Alignment of regulations should assist European companies in collaborative ventures. Standards can facilitate the opening up of markets thereby encouraging innovation. The danger is that they may become bureaucratic and over-prescriptive and so inhibit invention. Standards largely produced by staff from the larger organisations could assist the many small companies that should expand into this field.
2. Vital for Pan-European systems
There is a clear need for unification in Air Traffic Control [1], particularly as steps proceed to eliminate the humans in the loop with ground-based computers communicating directly with computers in the aircraft. Moves to improve civil air safety see a need to encompass the totality of an integrated AirTransport System, as distinct from the current practice of assessing Aircraft Airworthiness, Air Traffic Management, and Air Navigation systems in isolation [1]. This seems of great importance for Europe in order to reduce chronic congestion. Progress in these technologies will further export business.
Rather more speculative, because of inherent systemic limitations, is the progress to computer/communications between road and vehicle under the ARISE and EUROADS programmes [31]. Unification of standards and equipment seems of crucial importance [32]. There seems adequate time to get this sector in place.
New European Railways will assume increasing importance over the next 40 years. The already growing collaborations between the hitherto National railways should be encouraged now and steps urged to unify many safety-related issues. Funds should be allocated to do this with deadline dates for implementing new standards. However, recognising the longevity of the long-lasting rolling stock these deadlines should be well into the future and, of course, agreed with the railway authorities [33]. Sector interests rather than generic standards seem much more relevant in this case.
3. Difficulties
There is proliferation of multi-layered regulation, with typical participants:- [34]
Research Associations
Regulatory Authorities
Standardisation Bodies
Government Research Establishments
The UK is obsessed with formal methods of *software management*, which may make agreements with Europe difficult.
There are inadequate legal definition of failures of safety, allocation of responsibilities and penalties.
There are gaps and inconsistences in this rapidly expanding field of science and technology. Duplication of work done or proceeding elsewhere such as Esprit and Drive should be avoided [8]
Because of the identification of Information Technology and Artificial Intelligence [35] as separate "cultures" (for very good reasons) there is much overlapping of responsibilities, research and regulation that is peculiar to safety-critical software.
It is necessary to assess the past record of the European Parliament in tackling regulation and control of highly technological issues of this type.
A series of incremental changes does not necessarily lead to an ultimate optimal solution. At crucial times a thorough overhaul and revision may be the best solution.
4. Projects of the future
Regulation follows as a new technology and industry become established. The time period for getting regulations into equilibrium with an industry is very long. Any initiatives taken by the Council of Europe will probably not become effective for some years. As it is suggested that great good could come from introducing simplifications over the complete process from concept to penalties for accidents it is also relevant to consider likely new technologies and industries of the future.
This then also raises the question of how to stimulate innovation. The New World Order and reduction of military imperatives will result in weakening many of the engines for innovation in society that have been in place for hundreds of years. There is (a yet unperceived) need to replace the military drive for new technology with other stimuli. This will not be easy: I believe it must and will come and hence it will be sensible to align this development with the safety issues considered in this report.
The following are examples of possible innovations:-
* The Commonwealth of Independent States' (Ex USSR) natural gas-nuclear energy industry [36]. In the next 50 years the CIS must gain hard currency to build its infrastructure towards EEC levels. Its major resources are oil and gas of which gas will eventually predominate. Unless treated, this natural gas will be the main atmospheric polluter (World 50 X 109 tons CO2 per annum by 2040). Reference [37] describes the prospects for the steam reforming natgas by nuclear heat to hydrogen in the CIS region. The plant-derived CO2 is pumped into underground cavities or the oceans. Clearly safety is involved: a hybrid-culture between gas and nuclear industries essential and improving the ex- Soviet Union safety standards to the rest-of-the-world level. Early discussions with the Russians on this could provide extremely useful work for their displaced nuclear scientists.
* World Water Management [38]: with only 1% water on land and 99% in the sea and the growing desertified zones water will become a commodity needing desalination plants, pipelines, shipping and storage. Many new technologies are involved.
* Biotechnology [39]
* Genetic engineering [40]
5. PROPOSALS FOR STUDIES
An indication of the scope of studies in this field is usefully summarised in an extract from reference [19]. This is quoted in Appendix III.
The following is essentially a "menu", a list from which the most relevant can be drawn after an exchange of views.
* Overview of the alignment of national standards;
* Consider the revision of liability law in the light of new issues raised by safety-critical systems;
* The means of assessing the impact of proposed technicological legislation for safety, prior to implementation, bearing in mind also the strong economic implications;
* Assessment methods of safety in the early design stages;
* Post-project audits and European register;
* Cost-benefit analysis of "best" mix between established design practice in Industry, on the one hand, and regulatory processes on the other. This should look into specific case histories e.g car ferries, undergrounds and medical equipment;
* Improvement in qualification of certificators;
* Overall simplification of standards and systems engineering codes of practice;
* Dictionary of technical/legal words/acronyms;
* Assessment of past records of technological legislation in Europe and elsewhere;
* European digest of accidents/incidents in this field;
* IPR, software secrecy: rights of "inventors" [aa] and also those in overall safety responsibility. Implications of employing third parties and use of object oriented approach;
* Review of educational requirements;
* Public information campaign based on properly considered engineering programme;
If the Assembly wishes to make a decisive and additional contribution to this subject we recommend that it makes a very fundamental review of the whole field. This is because there is already a great amount of national and international work in progress on safety-critical systems in setting down standards, best practices and resolving the correct relationships between generic and sector specific issues. But the matter is technically and legalistically complex and above all desperately needs simplification in the interests of trade and in fact for the sake of safety itself.
Whatever is done it is important to assess any special advantage of a European solution rather than a national or international one. It is essentially not what should be done but whether anything should be done.
A working group could consider the following points :
* Clarify the studies programme e.g.
Pan-European systems such as Air Traffic Control
Pan-European car-road systems such as ARISE
Pan-European railway systems;
* Unification of international nomenclature in the safety/technology/legislation field;
* Steps to ensure that appropriate education in the field is available at all levels;
* Taking decisive steps to simplify all the procedures;
* Pursue the important question of IPR in relation to software systems and introduce rules protecting originality;
* Couple this work with technologies of the future and provide incidentally stimuli to innovation as some replacement for the weakening of military drive in this field;
* To consider the formation of a European Interdisciplinary Institute which would embody both law and technology and which would recognise also the economic dimension and give full weight to its implications for safety. It would produce double graduates, foster and manage research programmes and be charged with assessing experimental legislation.
6. CONCLUSIONS
* I must emphasize that any conclusions are inevitably tentative at this stage. However we believe that the essentials of the field have been addressed and most of the issues defined.
* Safety-critical systems are a very important safety domain but the field is complex both technically and legally.
* Because there is already a large effort underway in this area nationally and internationally, in sector industries, governments and professional institutions, considerable deliberation is needed to decide the optimal intervention from the Council of Europe.
* However, as the existing efforts embrace much historic, piecemeal and overlapping rules and legislation, the Council of Europe could be in a unique position to encourage simplifications, in the interest of harmonisation, increasing the competitiveness of products and enhancing trade.
* The proper approach by the Council of Europe is via the overall engineering design discipline, as this can enhance safety in the most cost-effective manner.
* The objective of regulation and legislation is to encourage identification and elimination of hazards in future systems. Clearer legislation and definition of infringement and penalties are needed. Present non-technologically worded legislation needs urgent overhaul. Existing civil and criminal laws are not self-consistent for handling infringements of safety.
* Putting together many of these factors leads on to a suggestion for consideration - the establishment of a unique European Institute covering both Technology and Law. This could produce double-degree graduates, embody and co-ordinate a Pan-European Research Programme, and attend to records, harmonisation of terminology and public awareness campaigns. Within its terms of reference it could perform experimental legislation typically embodied in on-going projects in order to assess the likely success prior to implementation. The long term effect on safety, professional standards and competitiveness could be impressive.
LIST OF REFERENCES
No Author TITLE
1 Howard R W Breaking Through the 106 Barrier, International Federation of Airworthiness Conference "Working together to stop aircraft accidents, Auckland, New Zealand, 20-23 October 1991.
2 Allen J E New Forms of Air Power, Trenchard Memorial Lecture, R Ae S at R A F College Cranwell, September 1985.
3 Fozard J W Sydney Camm and the Hurricane, Airlife Publications, Shrewsbury, U.K., 1991, ISBN 1-85310-270-9
4 Stamper J T Management of Design, pp. 368-400 in The Future of Aeronautics, John E Allen, Hutchinson, London, 1970, ISBN 0 09 100920 0.
5 Malcolm B. A Large Embedded System Project Case Study December 1989, p.5.
6 Ibid, p.9.
7 Bloomfield R E & SafeIT 2, Standards Framework, A Government Consultation
Brazendale J. Document on the Safety of Computer-controlled Systems. June 1990.
8 Malcolm B. Safety Critical Systems Research Programme, DTI, London, February 1991.
9 Williams J S Safety-Critical Systems, Private Communication, 29.1.1992.
10 Malcolm B. Software in Safety Related Systems:Basic Concepts and Concerns, from "Safety of Computer Control", Butterworth,1991.
11 - Safety-related Systems, Professional Brief, Institution of Electrical Engineers (UK) October 1991
12 Allen J E. Blue Steel; U.K's first stand-off bomb, Lecture to Royal Aerospace Establishment, Farnborough, 4.10.89.
13 Allen J E. Managing Major High-Technology Innovation. Lecture to ARRL, Salford, 7th July 1989.
14 Allen J E. Preliminary design of the Hawk, Lecture to the College of Aeronautics, Cranfield, 17th February 1978.
15 Allen J E. The Black Art of Preliminary Design, Bristol University, 10 March 1988.
16 Stix G. Along for the Ride, Scientific American, July 1991. p.76.
17 Reference 10, p.3.
18 Terry G J. Engineering System Safety, Mechanical Engineering Publications, Bury St Edmunds, Suffolk, November 1991 ISBN 0 85298 781 1.
19 - Software in Safety-Related Systems, A report prepared by a joint project team of The Institution of Electrical Engineers and The British Computer Society. October 1989. ISBN 0 852963 91 2.
20 Ibid. B-10
21 Reference 10, p.22.
22 Haynes A. Airmanship extraordinary, United Airlines DC 10 Crash, Aerospace, R.Ae.S. December 1991. p.8.
23 Hurst R. Pilot Error. A Professional Study of the contributory factors. Crosby Lockwood Staples, 1976. ISBN 0 258 97072 3.
24 Functional safety of programmable electronic systems: Generic aspects, "IERC/TC65A (Secretariat) 96, IEC Geneva, 1989.
25 Software for computers in the application of Industrial safety-related systems, IERC/TC65A (Secretariat) 94, IEC Geneva,1989
26 HSE Guidelines in PES in Safety-Related Applications,
1) An Introductory Guide;
2) General Technical Guidelines, U K
Health & Safety Executive, 1987.
27 Bloomfield R E. SafeIT 1, Overall Approach, A Government Consultation Document on the Safety of Computer-controlled Systems. May 24th 1990.
28 - Safety Critical Research Programme, Technical workplan for the second phase. (Ibid.) ref8
29 Reference 8, p.4.
30 Reference 8, p.2.
31 Cullyer W J. High-Integrity software for automotive applications, Proceedings of the Institution of Mechanical Engineers. Part D, Journal of Automobile Engineering, ISSN 0954-4070. Vol.205 No.D3.1991.p185.
32 Kayton M. Standardization of intelligent vehicles and highways, Ibid., p.193.
33 Wickens A H. Steering and Stability of the Bogie: vehicle dynamics and suspension design. Proceedings of the Institution of Mechanical Engineers. Part F, Journal of Rail and Rapid Transit. ISSN 09544097. Vol.205. No.F2. 1991. p.109.
34 Reference 8, p.4, top.
35 Wallich P. Silicon Babies:Trends in Artificial Intelligence. Scientific American, December 1991. p.82.
36 Marchetti C. How to Solve the CO2 Problem Without Tears,7 th World Hydrogen Conference "Hydrogen Today",Moscow, September 1988. [M-34 (BB 187)].
37 Allen J E. Some Geo-engineering Initiatives, Financial Times Conference on Pollution Management, Birmingham, October 1990.
38 Allen J E. World Water Management, Private communication, September 1991.
39 Erickson D. New Targets for Young Companies, Scientific American, January 1992. p.115.
40 Roberts L & Innovation and Environmental Risk, Conference at the University
Weale A. of East Anglia, organised by the Interdisciplinary Committee, 27-28 March 1990. Belhaven Press, ISBN 1 85293 156 6, 1991.
A P P E N D I X I
INDUSTRIAL SECTORS
Taken from DTI Safety-Critical Sytems Research Programme [28]
MEDICAL
Equipment ME
Diagnosis MD
Records MR
(The very effective fuse)
MANUFACTURING MACHINERY CONTROL MC
ROAD
Automotive * RA
Highway signalling RH
Design & structures RD
* good self-regulation through Trade Associations
RAIL
Line-side signalling TL
On-board control TT
Construction TC
Mostly signalling in which over 100 years experience of interactive systems. Signal box typically had 30,000 relays. Small but growing amount of safety-critical systems on traction-side. Panic button in signal box. BR kills av no of 7 pax/yr. Political drive for absolute safety. New auto-train protection for 700 million.
SHIPPING
Machinery control SM
Navigation SN
Construction SC
AEROSPACE
Avionics AA
ATC AT
Structures AS
Certifying authorities - CAA, FAA, EuroJAC, MOD; crashworthiness
POWER
Conventional generation PC
Nuclear generation PN
Distribution PD
Structures PS
PROCESS INDUSTRIES
Petrochemical PP
Pharmaceutical PH
Food processing PF
Other chemical e.g.paint PL
Other continuous processes e.g. steel PO
Process equipment suppliers PE
DEFENCE DE
(Ordnance Board. Accident damage limitation|
ADVANCED ROBOTS - mobile, in hostile environments: legislation for product liability. Grave disadvantage arising from public accident.
A P P E N D I X II
NOMENCLATURE
Compliance:
A synonym for conformance (QV). Alternatively compliance may be used distinctively from conformance. Where a distinction is made, conformance implies reference to a standard with objectively assessable requirements, and conformance implies reference to documents such as codes of practice in which some or all of the requirements stated in terms such that they are not readily amenable to objective assessment of being met.
Conformance:
A product or process is said to conform to standard when it meets the relevant requirements within it. (See also Compliance.)
Dependability:
The property of a PES that allows reliance to be justifiably placed on the service it delivers.
Dependability is a term covering all the principle characteristics used in connection with safety-related software.
Design Fault:
A design (specification,coding) fault results from a (human) failure occurring during the design of a system.
A design fault causes an error, which may reside undetected within a (sub)system until the input values to that (sub)system are such that the produced result is not appropriate to the circumstance. This constitutes a failure of that (sub)system. If the same input values appear again, the same erroneous results will be produced.
Environment:
1 The situation and conditions within which a system exists and operates.
The specification of a safety-related system may refer to the requirements of a particular environment (e.g.submarine)
2 The natural and man-made human world, which might be at risk from the failure of a safety-related system.
Error:
An error is a detected deviation from the agreed specification of requirements which is liable to lead to failure.
Fagan inspection:
Of the design phase of a project using checklist and walkthrough methods.
Fail-Safe:
The built-in capability of a system such that predictable (or specified) equipment (or service) failure modes result only in the system reaching and remaining in a safe fall-back state.
Failure:
A system failure occurs when the delivered service deviates from the specified service, where the service specification is an agreed description of the expected service.
A failure is a manifestation of an error in the system of software.
Fault:
A fault is a defect that gives rise to error.
Fault Avoidance:
The process carried out prior to the bringing of a system into operation in order to maximise its reliability: these include thorough testing of individual modules, and validation of the design
Fault Recovery: (Re-try Mechanism)
Fault recovery refers to the inclusion in software of elements intended to ensure that, in the event of failure of a piece of software (a module or part thereof), there will be an automatic attempt to re-try that piece of software.
Fault Tolerance:
The built-in capability of a system to provide continued correct execution, i.e. provision of service as specified, in the presence of a limited number of hardware or software faults.
Formal Methods;
Formal methods are mathematically based techniques whose application enables verification of some properties of software to be verified, including some aspects of conformance to its specification. For this process to take place, there must also be a formal specification. The creation of this specification is itself part of 'formal methods'.
Graceful Degradation:
When a failure is detected in a system, the system may respond by completely ceasing to operate. Alternatively it may continue to operate partially, therby providing those of its functions which are not affected by the failure. Further failures may lead to further diminution of functionality. This alternative, and the provision of appropriate warnings to operators, is referred to as graceful degradation.
Hazard:
A situation in which there is actual or potential hazard to human life or limb, or the enviroment.
HAZOP:
Hazard and Operability study technique.
Maintainability:
The ability of an item under given conditions to be retained in or restored to a state, in which it can more effectively perform the required function.
Programmable Electronic System (PES):
A system, based on a computer, connected to sensors and/or actuators on a plant for the purpose of control, protection or monitoring.
Redundancy:
Provision of additional elements or systems so that anyone can perform the required function regardless of the state of operation or failure of any other. Redundancy can be implemented by identical elements (identical redundancy) or by diverse elements (diverse redundancy).
Reliability:
The probability that during a certain amount of time a system performs the functions described in the specification of requirements under the stated conditions (environment,costs, inputs and hardware, time constraints).
Requirement:
A state of a criterion which must be met if a particular product or process, etc, is to be deemed to be acceptable, in relation to a technical or procurement specification.
Risk:
The combination of the frequency, or probability, and the consequence of a specified hazardous event.
Safety:
Freedom from unacceptable risk of personal harm. The likelihood that a system does not lead to a state in which human life or environment are endangered. Note:safety relates to all aspects of a system, all its sub-systems, to the environment in which it operates, to human factors such as operator error or wrongdoing, and to incorrect data. Some factors related to safety can readily be quantified;others may be considered only qualitatively.
Safety Integrity:
The likelihood of a safety-related system achieving the required safety functions under all the stated conditions within a stated period of time.
Safety-Related System:
A system by which the overall safety of a process, machinery or equipment is achieved. Safety-related systems fall broadly into two types:those in which separation of control and safety functions cannot be made (e.g. some types of flight control system); and those which are designed to respond to hazardous conditions, and protect against them, independently of other control systems.
Software:
Programs, procedures, rules and associated documentation pertaining to the operation of a computer system.
Software Safety Integrity Level:
An indicator of the extent to which there is assurance that software is safe, or that software achieves the necessary safety integrity.
Software Lifecycle:
The period of time that starts when software is conceived and ends when the software is no longer in use. The software lifecycle typically includes a requirements phase,design phases, an implementation phase,test phases, an installation phase and a maintenance phase.
Standard:
A document which establishes criteria by which the qualities of products or processes may be objectively assessed; each criterion involves a requirement. Standards of this kind are specifically referred to as normative standards.
More generally, "standards" also refers to guidelines, codes of practice,etc. These may be concerned with advice, with methods by which the requirements of a normative standard can be met, or with standard-like information which is, however, open to interpretation rather than being capable of objective assessment. Some standards contain both requirements and guidelines as to the achievement of those.
System:
A system is a set of components which interact according to design. A component of a system can be another system (called a subsystem). Such components (subsystems) may be:
* controlling or controlled systems;
* hardware,software,human interaction
* input and output subsystems
Tolerable Risk:
Safety can never be guaranteed. It is necessary to accept that there is a certain level of risk associated with any product. There is also a point at which increased safety measures cannot be justified because their cost is out of proportion to added safety which they provide. This is the 'tolerable risk'. Note:in this respect,cost refers not only to economic costs, but to optional and functional requirements,convenience etc.
Validation:
The test and evaluation of the integrated software system to ensure compliance with users'requirements. Validation is generally used to refer to a larger process than verification. In particular, whereas verification tests software against the specification for that software,validation of a system is concerned with whether the operation of the system provides the results needed by the user. Validation therefore involves consideration of whether the specification of a system sufficiently and accurately represents the needs of the intending user.
Verification:
The process of determining whether or not the product of each phase of the software life cycle development process is consistent with the requirements specified in the previous phase. Note that this does not necessarily imply fulfilment of those requirements.
A P P E N D I X III
SOFTWARE IN SAFETY-RELATED SYSTEMS
Executive Summary Recommendations of IEE/BCS Report [c]
The establishment of means for the continuous review of the overall position to ensure both the dissemination of appropriate knowledge as it becomes available and its use as the basis of a programme of certification. This could be achieved by continuing the Project Team, with payment of fees to the specialists from private industry
The establishment of an awareness programme to ensure that all involved with safety-related systems including non-engineering professionals in other fields are aware of all the activities in the field, of their responsibilities, and of the factors involved in achieving safety. Consideration should be given to the production of a newsletter. An extensive publicity campaign will be needed.
The dissemination of information about the use of methods of assessment of safety-related software. Realisable detailed recommendations overcoming problems with divulgence of commercially sensitive information may result from further discussions.
The establishment in the UK of a scheme to monitor the introduction of programmable elements in safety-related systems. Active participation by the HSE is likely and legislation may be required.
The development of a technical framework and the adoption of a Safety Lifecycle as a procedural framework for all safety-related systems. This could be undertaken as an independent study or by the HSE.
The development, through a research project, of definitions for quantitative parameters to be used as measures of safety integrity.
Numbers used to categorise systems should be presented only as 'claim limits' based on engineering judgement. Those making such claims must be able to justify them. Research should be undertaken to enable more rigorous quantification.
The technical and procedural frameworks should be adopted via IEC standardisation to align Europe with the wider international community; they should be given as much force and authority as possible. Standards developers should take cognisance of existing and developing IEC standards.
The focus of attention should not immediately be on the development of certification systems; rather research and development should be undertaken to achieve needed levels of knowledge and to assess the usefulness of certification and how it may best contribute to improved safety assurance. Established certification practices should be continued.
Organisations should, when developing standards, distinguish between generic requirements, industry-and application-specific requirements, and procurement procedures. Within the UK, industry should be encouraged to develop a pan-sector code of practice starting with the adoption of the Safety Lifecycle.
All those involved in the Safety Lifecycle should be competent with respect to their safety-related work. Organisations which supply or operate systems should be responsible for nominating competent persons to identify hazards and ensure that appropriate action is taken. For the most critical systems there should be at least one named competent person with overall responsibility.
Financial liability should lie with the organisation rather than the individual engineers.
The required education, training and experience should be determined and courses made available at post-graduate, post-experience level.
The principles of System Safety Assessment Reports and of independent safety assessment are recommended.
A programme of research funded through a SERC IRC to:
a) analyse previous experience to avoid repetition of earlier errors.
b) determine the contribution to safety integrity of different factors individually and in amalgamation and in particular circumstances; develop means for quantifying these.
c) refine the technical framework.
d) study architectural forms and their appropriateness for particular applications.
e) establish and if possible quantify the benefits of diversity of systems, software and hardware.
f) investigate testing; statistical inferences drawn from experience; complexity, its measurement, and its relationship to other measures.
A P P E N D I X IV
PROGRESS TOWARDS UNIFIED STANDARDS
(Case study : United Kingdom)
This is a brief review of the UK scene; nevertheless I have found a great awareness of the need for and advantages of overseas collaborations and these are also referred to [24,25]. In 1987 the U K Health & Safety Executive (HSE), a major regulatory safety body published guidelines in Programmable Electronic Systems in Safety related Applications [26]. The major international activity is the International Electrotechnical Committee (IEC). Another major player in Europe is Comite European de Normalisation Electrotechnique (CENELEC) (*), which, together with other national standards, e.g. DIN and BSI have agreed to collaborate with the IEC.
A starting point in the U K was the 1986 ACARD report "Software: A vital key to UK competitiveness" which led to the study "Software in Safety-Related Systems" by the Institution of Electrical Engineers and the British Computer Society published in 1989 [19]. This comprehensively treated technology, legal liability, certification, the UK and overseas situation, and recommended a programme of future research. It was clear that although there was a great deal of relevant work in co-ordinating requirements and standards the situation was very fluid and some time would be needed before a generally acceptable and workable overall plan could be implemented. The Chairman of this Study Group has given valuable assistance in the preparation of this report.
Another major British Government initiative is the SafeIT programme: the safety of Computer-controlled Systems, inaugurated in 1990 [27]. It embraces a wide range of national authorities viz.
Interdepartmental Committee on Software Engineering
This has a wide-ranging representation from many official bodies:
Department of Trade & Industry + Health & Safety Executive [27]
Ministry of Defence + Science and Engineering Research Council
Central Computer and Telecommunications Agency + Home Office
(*) It should be pointed out that these organisations deal mainly with the harmonisation of electrotechnical standards. For example, CENELEC (the European Committee for Electrotechnical Standardisation) is specialised in electromedical equipment, electromagnetic compatibility, equipment for use in potentially explosive atmospheres, measuring instruments including electronics, safety of machines, and for public procurement purposes. A recent initiative aims at setting up a sector for the certification of Information Technology Equipment.
ICSE Working Group on Safety Related Software
DTI, HSE, MoD, SERC, Depts of Transport, Environment & Health,
Civil Aviation Authority and National Power.
SafeIT's remit is:-
(i) To facilitate the development of technically sound, feasible, generic international standards together with consistent sector or application specific standards which are likely to achieve widespread acceptance
(ii) To encourage the use of Programmable Electronic System (PES) technology in safety-related applications and promote the adoption of advanced techniques and best IT practices in relation to PES.
(iii) To ensure the application of PES in safety-related applications enhances safety.
(iv) To find better solutions to assure PES integrity.
(v) To promote within the U K and internationally open markets in safety-related PES.
SafeIT 2 [7] sets out a framework for standards designed to encompass the complete safety engineering of PES
There is now an extensive, ongoing government-led research programme on the whole field outlined above. It covers 29 industrial sectors as given in Appendix IV and has three main themes, Technologies, Human Factors & Unification [28], with a total of 22 possible items for each sector. This programme is accordingly extremely complex, and while under way it would be premature to draw conclusions or expect very quickly to make external accomodations. The time is ripe however for mature discussions with the Council of Europe to ensure that UK and C of E ideas are working towards a harmonious outcome and not in conflict of timing or intent.
The UK safety critical system research programme has four main targets:-
* How should we produce and manage the production of safety-related software in particular situations?
* Can we find a common approach?
* What is the rationale for present tradition and practice in a variety of sectors?
* How can we harmonise them in such a way that a common approach to software can be incorporated.
The Safety Critical System Club [29] is expected to have a major role in achievement of consensus. It is the primary means of dissemination of intermediate and final results of the programme, and a sounding board for them; it is the forum in which workers in very different areas can compare notes; it is the focus for newcomers to the safety-critical scene, where they can quickly find out who is knowledgeable about any particular subject. It now has 1000 members and its extension to European organisations is now under active consideration.
The scope of the SafeIT programme indicates that it will have to probe and codify all engineering practice in the UK and bring it into a logical framework compatible with the software digit. This looks surprisingly like an insoluble problem: have the number of professionals to perform this task in a finite time been calculated? What would be its cost? Could the effort be better spent in making goods rather than procedures? What would happen to the detail standards etc if another technology for performing machine control appears?
Two extreme situations may be postulated. In the first, "minimum intervention", the manufacturer has to meet legally defined safety levels which are agreed at the project start, and then proceeds without further control. Failure to achieve the set safety levels would result in very severe fines and imprisonment. In the second, "maximum intervention", every stage of the product evolution is subject to regulation and audit, penalties would be of little purpose as the authorities would have assumed responsibility throughout.
Of course, neither of these alternatives is practical, but between the two extremes there lie very many hybrids and it is considered that the choice of which intervention should be used, how successful it would be should be the subject of study, prior to introduction.
Reporting committee: Committee on Science and Technology.
Budgetary implications for the Assembly: None.
Reference: Doc. 6465 and Reference No. 1745 of 28 June 1991.
Draft recommendation unanimously adopted by the committee on 5 March 1993.
Members of the committee: MM. Bassinet (Chairman), Roseta, Mészáros (Vice-Chairmen), Aarts, Mrs Arnold, MM. Bauer, Berger, Borderas, Bosco, Fulvio Caccia, Paolo Caccia, Cunliffe, De Decker, Dees, Dimmer, Dincerler, Draus (Alternate: Bartodziej), Dumont, Fourré, Hagård, Mrs Hjelm-Wallen, MM. Kalos, Karakas, Konecny, Korakas, Lenzer, Lopez Valdivielso, Lotz, Luczak, Moran, Mrs Nybakk, MM. Panov, Pecriaux, Poças Santos, Albert Probst, Mrs Ragnarsdottir, MM. Schädler, Semerdjiev, Mrs Terborg, Sir Donald Thompson, MM. Jack Thompson, Tiuri, Trabacchini (Alternate: Ferrarini).
N.B. The names of those who took part in the vote are printed in italics.
Secretaries to the committee: Mr Perin and Mrs Meunier.
1 * Part B is based on a study prepared by a consultant expert to the Parliamentary Assembly, Professor John E. Allen, President of the Interdisciplinary Committee of Chartered Engineering Institutes and Combined Scientific and Technical Institutes, United Kingdom.