Basics in Research Methodology- the Clinical Audit

Department of of Oral and Maxillofacial Surgery, Kothiwal Dental College Research Centre and Hospital, Kanth Road. Moradabad

Abstract

Health care organisations regularly undergo quality assurance for safe and effective patient care. Such quality improving programs are considered as audit. The purpose of audits is to generate findings that will benefit patients and their programmes of care. Audits should be regularly carried out in a systematic manner as patient welfare is at the heart of any audit process. Auditprocess should maintain professional and ethical perspectives also. However, clinical audits are often poorly carried out and consequently have minimal effect on improving patient care. Health care organizations should encourage clinicians to participate in regular clinical audit. This review explains the basics of audit and describes in detail how a clinical audit should be performed and monitored.

Research, Health Technology, Quality, Data.

Healthcare delivery organisations globally are utilising various quality indicators to measure the efficacy of specific interventions as well as to identify the healthcare improvement opportunities. These quality indicators are further being used for performance and outcome measurements as a means to measure, monitor and improve the quality of the care and services of these organisations (1).

‘‘Audit’’ means to evaluate (1). The standard definition of clinical audit is ‘‘a quality improvement process that seeks to improve the patient care and outcomes through a systematic review of care against explicit criteria and the implementation of change (2). Clinical audit explicitly entails the review of the clinical performance against the agreed standards and the subsequent improvement of the practice, followed by further audit to continually drive up the standards. Its aim is to improve the quality of the care which is delivered. It is different from other types of audit that may be conducted in the clinical workplace such as a financial or organisational audit.(3)

WHY CLINICAL AUDIT
The purpose of audits is to generate findings that will benefit patients and the programmes of care for them. Patient welfare is at the heart of audits and so, from both the professional and ethical perspectives, healthcare organisations should not stint on the audit activities (4).

Clinical audit is one of the key elements of clinical governance .Clinical governance is a system through which healthcare organisations are accountable for continuously improving the quality of their services. It is described as ‘‘a framework through which organizations are accountable to continue to improve the quality of their services and to safeguard their high standards of care by creating an environment in which the excellence in clinical care would flourish and would be central to the modernisation plans of the National Health Service (NHS)". The provision of safe, high-quality care is a statutory obligation for the healthcare organisations which are under the Health Act, and clinical governance (3),(5). Clinical audit is central to these quality improvement principles. It provides the means to review(2).The quality of the care which is given to the patients with common conditions.The health screening activities.

• Significant events. Audits are time consuming and if the staff perceives them as chores which are required by the management that attract little or no feedback, they can become demoralised. In such cases, the staff will not perceive any benefits in audits and will engage in them only reluctantly. This leads to frustration all round (6).

There are also examples within the dental practice such as the impact of clinical audit on antibiotic prescribing in the general dental practice across the east of England which led to significant improvements in the appropriate prescription of antibiotics, an overall reduction in the total number of prescriptions which were made over comparable timeframes, and a significant increase in error-free prescriptions. In addition to these tangible outcomes that can directly affect patient care, other commentators have described less overt benefits (7). One review of clinical audit found evidence from across the healthcare professions of professional benefits to those personnel who were participating in the audit, including a better team communication, job satisfaction, learning from the behaviour of colleagues and an increase in the staff enthusiasm (2). It is important to understand that while clinical audit and research share some common features, they are distinct disciplines and therefore are not the same. Research is concerned with the creation of new knowledge. Clinical audit ensures that this knowledge is being applied appropriately (1),(3).

THE DEVELOPMENT OF AUDIT
As early as 1750 BC, King Hammurabi, the 6th king of Babylon, instigated audits for the clinicians. In modern medicine, one of the first clinical audits was undertaken by Florence Nightingale during the Crimean War of 1853–1855; she applied strict sanitary routines and hygiene standards that decreased the mortality rates from 40% to 2%. Another famous figure who advocated clinical audit was Ernest Codman (1869–1940), an orthopaedic surgeon at Harvard Medical School. He became known as the first true medical auditor following his work in 1912 on monitoring surgical outcomes. Despite the early work of these pioneers, clinical audit is relatively new to the modern medical practices.

In 1989, the Department of Health published a white paper which was entitled ‘Working for Patients’. The paper proposed several measures, one of which was that the arrangements ‘for whatdoctors call medical audit’ be extended throughout the health service.

A working paper on medical audit was published the following year to provide the details as to how the proposals had to be taken forward (1),(2),(3),(7).

By 1994, the term ‘clinical audit’ appeared to have largely replaced the earlier term ‘medical audit’. Benchmarking schemes, which share similarities with clinical audits have been in existence in the USA for many years. Benchmarking is the process of measuring products, services and practices against the leaders in a field. Thus allowing the identification of the best practices that lead to a sustained and improved performance (1),(8).

The audit team: A structured approach, with an effective leadership and a working team, will underpin a successful audit. The team may be unidisciplinary or multidisciplinary. It is imperative to engage at the planning stage, all those who will be involved centrally or peripherally and all those who may be affected by the audit, whether they are colleagues or service users.

It will be difficult to achieve any change in the practice following the audit if the colleagues have not been committed to the project from the outset. Also, patient involvement is an essential part of the clinical governance framework. Note that a clinical audit requires funding and it may increase costs and require protected time (3),(9).

THE AUDIT CYCLE (5).
STAGE 1: SELECT THE AUDIT TOPIC: Any topic which is selected for clinical audit - perhaps an investigation, treatment or procedure - should be chosen on the basis of its relevance to improve the patient outcomes and not to satisfy the personal curiosity. The topics may be selected from the reports of adverse incidents, activities which are identified as high risk, expensive treatments or perhaps, evidence-based interventions. Senior topics like the awareness about the brushing habits, dental caries, priodontities and antibiotics for dental infection can be useful for a better health care management. The examples of some topics are given in the Table below (10). (Table/Fig 1)

STAGE 2:IDENTIHCATION:
The next step is to identify as to what aspects of the best practice should be included in the audit. Local and national guidelines, national service frameworks and research papers can help to determine as to what is considered as the best practice. A useful structure for planning audits is to ask what, why, who, where, when and how, in which: * ‘What’ concerns the subject area or the specific topic of an audit.

* ‘Why’ concerns the objectives for conducting the audit with respect to what people wantto find out. * ‘Who’ concerns the population or the sample group. * ‘Where’ concerns the location or setting in which the audit is going to be conducted. * ‘When’ specifies the time in which the audit will take place, and how it will be a part of the ongoing data collection with a review of the findings in an established audit cycle. * ‘How’ relates to the data collection techniques and the strategies.

STAGE 3: AGREE WITH THE CRITERIA AND THE STANDARDS
The use of the terms ‘criteria’ and ‘standards’ in a clinical audit is often misunderstood. The audit criteria will provide a statement on what should be happening and the standards will set the minimum acceptable performance for those criteria. The criteria and standards must be specific and measurable. In selecting the criteria, one should carefully consider exactly what he/she wants the audit team to achieve. It may be helpful to phrase the aim as a question which is to be answered or a statement about how the topic should be. A common understanding among the team will support the quality of the audit. Simplicity is important -bear in mind the acronym KISS (‘Keep It Simple, Stupid’). Remember that the standards may need to be revised, to reflect a new evidence for an intervention or an activity. Some criteria and standards are so important that 100% achievement is required but this is likely to be unusual.

STAGE 4: COLLECT THE DATA
Collect only the data that are specifically related to the audit criteria. Decide on prospective or retrospective audit and on how to collect the data -- for example, on a pro forma, by direct entry into a computer or by searching on the Read codes. Undertake a small pilot audit to ensure that the tool is robust and is collecting appropriate data. If someone is searching the patient records and the practice team has not been using the templates for data entry or has been making free-text entries, it will be difficult or impossible to retrieve the required information.

The data which is collected must be relevant, accurate and representative. Most of the audit data are collected by using either manual data collection forms or they are recorded by using electronic computer software such as the Microsoft applications, Excel and Access. A careful review is necessary to ensure that the data which were collected are representative and that the correct data were collected.

For the assessment of the effectiveness, diffusion and equity in health technology (HT), we have classified the routine data into three broad groups: Group I datasets: which identify both the HTs and the health states. Group II datasets: which identify the HTs but not the health states Group III datasets: which identify the health states but not the HTs Clearly, the datasets in group I are the most promising, although there are occasional potential uses of the groups II and III at the population level enquiries and in adjunctive roles. Group I datasets can be further classified into (a) clinical registries, (b) clinical–administrative datasets and (c) population-oriented datasets. Group III datasets can be divided into (a) adverse event reporting and confidentialenquiries, (b) disease-only registers and (c) health surveys. Group I datasets can be used not only to assess the effectiveness but also to assess the diffusion and equity. By contrast, the databases in group II (HT only) can only help in assessing diffusion. Those in group III (patient health- related characteristics only) have restricted the scope for assessing the HTs except for the analysis of adverse events.

Patient Data: Caldicott Principles:3 In March 1996, The Caldicott Committee was established for the protection and the use of patient information,

The Caldicott principles for using patient information are: * Justify the purpose. * Don’t use patient-identifiable information unless it is absolutely necessary. * Use the minimum necessary patient-identifiable information. * The access to the patient-identifiable information should be on a strict need-to-know basis. * Everyone should be aware of their responsibilities. * Understand and comply with the law.

STAGE 5: ANALY SE THE DATA
Analysis involves interpreting the collected data to discover how the current practice compares to the agreed criteria and the standards. It identifies the areas both of underperformance, which should be reviewed in detail to identify why the care falls below the desired levels and how it can be improved, and of over performance.

STAGE 6: IMPLEMENT THE NECESSARY CHANGES:
Implementing changes that will improve the poor results is often the hardest part of any audit project. All the team members should be involved in discussions about what changes should take place, so that all the possible solutions are explored. These changes invariably depend on the specific circumstances of the audit, but often include staff training and the introduction of* better systems of practice or new protocols and guidelines.

STAGE 7: CONDUCT A RE-AUDIT
Re-audit is another key part of the audit cycle which should be carried out within a year of implementing the changes. Re-audit involves collecting a second set of data to and review the progress after the changes have been implemented to identify whether further improvement is needed. The numbers which have been audited should be comparable to those from the first data collection phase.

STAGE 8: WRITE AND DISSEMINATE AN AUDIT REPORT
This being the final stage of the audit cycle is intended to create a record for the auditor, the team and the organization which is involved. This report should also be shared with the colleagues who have taken part in the work, so they can see what effects the audit has had on their practice. Sharing audit reports widely also helps those who want to conduct clinical audits by using the same methodology. Audit feasibility scoring grid (11),(12). * Does the audit address a problem that is relevant to patient care? * Is the topic a priority for the team or the organisation?

* Can the data be collected quickly, ideally in less than a month? * Is there confidence that the data will be reliable and accurate? * Could the changes which were recommended as a result of the audit be implemented?

Scoring In answering the questions, award two points for a ‘Yes’, one point for a ‘Not Sure’ and no points for a ‘No’. Audits that score five or less are unlikely to succeed, those that score six or seven are worth considering and those that score eight or more will usually succeed.

In the Indian scenario for example, whether the prescription of antibiotics for the patients by the doctors is correct or not, can be adited. First, select the topic i.e. antibiotic prescription by the doctor. The next step is to have the best practice. For example, the drugs which are prescribed, their dosage, strength duration, whether there is a set criteria as to whether the drugs were prescribed in their correct dosages and their strength and whether or not their correct duration is being followed or not should be evaluated. Collect the data which were followed for the drug prescription at regular intervals. The collected data should be analyzed for their efficacy in the process of the clinical audit. If there are any shortfalls in the process that should be corrected, then necessary changes should be implemented. Again the data should be analyzed and the report of the findings should be formulated, as to whether the audit has met the set criteria or not. These findings should be implemented for a better outcome in the clinical practice.

In a study which was conducted by Andrew and Alan, the audits on clinical record-keeping standards were performed by using the 7 domains of the case history which are Personal details, History recording, Dental Examination, Periodontal examination, Radiographs, Record of the review of the radiographs and Notewriting. The audits revealed a wide variation between the dentists in clinical record-keeping. The recording of the soft tissues (36%), periodontal status (30%), radiographicalreview (27%), and notetaking (25%), all fell below the standards that had been set.(10)


TEN TIPS FOR SUCCESSFUL AUDITS.(3)

1. Start small clinical audit projects. 2. Involve the team members. Audits are most effective when they are carried out by teams. All the staff should be asked to suggest suitable topics and they should be told about the results. 3. Distinguish between research and clinical audits. Remember that research is undertaken to find out what the best practice should be,audits are undertaken to find out whether the best practice is taking place. 4. Learn from the completed projects of others. . 5. Select audit topics that relate to the current work. 6. Gather support. The local support for the clinical audits varies but some trusts have audit teams. 7. Plan the audits properly. Simple audit calendars which are used to map out the audit activities over the course of a year for example are useful. 8. Pilot the audits. A small number of data collection forms should be tested to make sure that they are providing all the information that is required. 9. Re-audit is vital. Without undertaking re-audit, there is no way of knowing whether the changesthat have been made have improved the patient care or the service delivery.

10. Get the most out of clinical audits. Although audits deal with the identification of the weaknesses and the improvement of the patient care, they can also be used as an example to improve the teamwork or communication.

Conclusion

Audits are a part of the continuous quality improvement in the health care systems. Audit is a cyclical process: it compares the practice of standards, measures performance, makes improvements and, most importantly, involves a re-audit after a time period to ensure that the improvement is sustained.

DOI and Others

JCDR/2011/1350