Content Validation of Questionnaire for Survey on Practice Preferences of Indian Anaesthesiologists in Difficult Intubation and “Cannot Intubate, Cannot Ventilate” Situations (CICV)
Correspondence Address :
Dr. Govardhane Balasaheb Tukaram,
Associate Professor, Department of Anaesthesia, Dr. DY Patil Medical College, Navi Mumbai, Maharashtra, India.
E-mail: drbgovardhane@gmail.com
?Introduction: American Society of Anaesthesiologists (ASA) and Difficult Airway Society (DAS) have provided guidelines on management of difficult intubation and Cannot Intubate, Cannot Ventilate (CICV) situations. There have been many advances in equipment available to anaesthesiologist in difficult airway like Supraglottic Airway Devices (SGA), Video-Laryngoscope (VL) and optical stylet. The knowledge and practice of these guidelines are important for patient safety in difficult airway scenario.
Aim: To validate the questionnaire to assess practice preferences in difficult intubation and CICV in Indian anaesthesiologist.
Materials and Methods: This questionnaire-based observational survey was carried out between April 2021 to June 2021 at Department of Anaesthesia, DY Patil School of Medicine, Navi Mumbai, Maharashtra, India. Nineteen questions were framed based on the literature related to difficult intubation and CICV practices. The questionnaire was sent to 20 experts to grade each question on relevance, clarity, simplicity and ambiguity on a likert scale of 4 as provided in information sheet. Sixteen out of 20 experts responded. Nineteen questions were framed based on the literature related to difficult intubation and CICV practices. The questionnaire was expanded to 22 questions after pilot testing with 10 senior expert anaesthesiologists suggestion of including the question on apneic oxygenation. The collected responses were used to calculate Item-wise Content Validity Index (I-CVI), Scale-wise Content Validity Index (S-CVI) and Modified Kappa Statistics (MKS) in Microsoft excel sheet.
Results: The S CVI for relevance, simplicity, clarity and ambiguity was 0.77, 0.77, 0.77 and 0.73, respectively. S CVI/average or Average Congruency Percentage (ACP) was 0.95, 0.95, 0.95 and 0.94 for relevance, simplicity, clarity and ambiguity, respectively. Question 4, 14, 20, 22 received I-CVI of 0.75 in terms of relevance, clarity, simplicity and ambiguity and modified as per experts instructions.
Conclusion: The survey questionnaire developed to assess practice preferences in difficult intubation and CICV fulfilled the content validity criteria both by qualitative and quantitative analyses.
Difficult intubation, Questionnaire, Survey, Validity
Anaesthesiologist must be well-trained in airway management which is one of their prime responsibility. The difficult airway is defined as the clinical situation in which a trained anaesthesiologist experiences anticipated or unanticipated difficulty or failure including but not limited to one or more of the following: face mask ventilation, laryngoscopy, ventilation using SGA, tracheal intubation, extubation, or invasive airway (1). CICV or Complete Ventilation Failure (CVF) situations are quite rare but present as life threatening emergencies. CICV defined as a situation where intubation, ventilation using SGA and facemask failed after giving best attempt, even if oxygenation maintained (2). The American Society of Anaesthesiologists (ASA) (1) and Difficult Airway Society (DAS) (3) releases practice guidelines for managing difficult airways from time to time which assist the practitioner. The latest in this line is the ASAs practice guidelines for management of the difficult airway released in January 2022 (1). In last decade, several airway devices introduced to cope-up with these life threatening situations; however, the most optimal airway device has not yet been determined.
The Canadian Anaesthesiologists’ Society (CAS) conducted an online questionnaire based survey in 2003 and again in 2013 to determine the anaesthesiologists’ proficiency and preferences in difficult intubation and CICV situations. The researchers concluded in 2013 survey that VL is the most frequently selected device followed by flexible bronchoscope, in difficult intubations, while in CICV situations, wire-guided Cricothyroidotomy (CT) was the most commonly chosen followed by intravenous (i.v.) catheter CT (4). Similar survey was conducted in Korean Anaesthesiologists and intensivists finding similar to Canadian survey in difficult intubation but in CICV CT by i.v. catheter was first choice (5).
Rajesh MC et al., conducted a survey of current practice of difficult airway management among Indian Anaesthesiologists’ and concluded that when difficult airway is anticipated, the preferred choice for junior doctors was attempting conventional intubation. The experienced anaesthesiologists chose awake fibreoptic bronchoscopy. However, in case of unanticipated difficult airway, most of the residents and consultants preferred SGA (6).
However, no such survey was conducted among Indian anaesthesiologists for both difficult intubation and CICV situation. So, the present questionnaire based survey was framed to determine practice preferences of Indian Anaesthesiologists in difficult intubation and CICV situations.
Making a questionnaire and validating it is a long process and involve different steps from developement, validation to pilot testing of the same. Development of a questionnaire involves identifying the dimensionality of the construct, determining the format in which questionnaire will be administered, determine the item format and lastly the item development. The next step will be reviewing the initial pool of items and revising it. After a raw form of questionnaire is formed it is subjected to content validation using standard method face validation with the help of subject experts. The validation stage is important to make it psychometrically sound. After validation, the questionnaire can be used for pilot testing and it can be reviewed later on for further changes. Validity support that an instrument measures what is designed to measure (7).
Extensive literature review and validation of the questionnaire based on expert review and opinion are two pivotal steps in the process of development of a validated tool. Content validation which include expert reviews is an essential step in instrument development as it signifies the extent to which the content is conveying similar meaning to all those who use it and is not misleading (7).
The present study aimed to validate the questionnaire to assess practice preferences in difficult intubation and CICV in Indian anaesthesiologist using CVI and MKS (8),(9).
The present questionnaire based observational survey was conducted from April 2021 to June 2021 at Department of Anaesthesia, Dr. DY Patil School of Medicine, Navi Mumbai, Maharashtra, India. The content of questions related to anaesthetic practices followed in case of difficult intubation and CICV by Indian anaesthesiologists’ were framed after an extensive literature search by four independent anaesthesiologists. The authors searched data bases that included PubMed, Google Scholar, Cochrane Library and guidelines framed by ASA (1), All India Difficult Airway Association (AIDAA) (2) and DAS (3). The collected information was organised. The search keywords were ‘difficult airway’, ‘difficult intubation’ and CICV. Six articles included recent review articles and latest guidelines by various societies (1),(2),(3),(4),(5),(6).
Questionnaire
Initially, there were 19 questions which underwent intensive scrutiny for simplicity, relevance, clarity and ambiguity by the authors. Grammatical and spelling errors were taken care of. The questionnaire was expanded to 22 questions after pilot testing with 10 senior anaesthesiologists who have more than 10 years experience. The section A, first part of the questionnaire collected general demographic information about the anaesthesiologist like demographic data including age, gender, years of practice, type of hospital and the area of practice and the position in the institution. Section B of the questionnaire focused on encounter with difficult intubation and devices and techniques used to anticipate and manage it like VL, Intubating LMA or similar and Retrograde Intubation (RGI). Three questions on apnoeic oxygenation were also included in section B after discussion with experts. Section C enquired about CICV; the number of times CICV encountered, type of surgery in which it was encountered, the device and technique used at that time, knowledge of CICV steps etc.
Also, an information sheet was made for an expert group to grade each question based on relevance, simplicity, clarity and ambiguity by using 4 point scale [Appendix-1] (10). The acceptable cut-off value of CVI is 0.78 for that atleast nine experts opinion needed. And considering 50% non response for face-to-face or non-face-to-face approach authors selected 20 experts for the present study (11). This questionnaire and information sheet for grading each question in questionnaire was sent by email and manual form to 20 experts. The experts have atleast 10 years experience in anaesthesiology at Medical College and dealing with difficult airway. A time frame of 45 days was provided with two reminders after three and six weeks to the experts to assess and grade the questionnaire. The target was receipt of response from atleast 50% of the experts at the end of the 45 days, however the authors got 80% response as 16 out of 20 experts responded to the questionnaire.
Statistical Analysis
The results were compiled and tabulated in Microsoft excel™ sheet. The I-CVI was calculated for each question from the questionnaire 50and the S-CVI and S-CVI/average or ACP were also calculated based on the methods described by Polit DF and associates [8,9]. Probability of chance agreement on relevance (Pc) was calculated using the formula:
Pc={N!/{A!(N-A)!}}×0.5N, Where, N=number of experts and A=number of experts in agreement on relevance. MKS was calculated for each question using the following formula:
K=I-CVI-Pc/1-Pc, I-CVI was calculated using a proportion of content expert giving item a relvnce rating 3 or 4 (I-CVI=agreed item/number of expert). S-CVI was calculated based on the average of I-CVI scores.
If the I-CVI was less than 0.78, that question need to change as per expert suggestions (10),(11). The values obtained for I-CVI and MKS for relevance, simplicity, clarity and ambiguity were calculated for the evaluation of each item done. In the present study, the author used the parameters given by Fleiss J et al., as they had provided four grades. They grade the kappa (k) statistic as <0.40=poor, 0.40-0.59=fair, 0.60- 0.75=good, 0.75-1.00=excellent (12).
In the present study, 16 out of 20 experts completed the assessment of each question as per grading scale and forwarded the completed proforma. S-CVI calculated for relevance, simplicity, clarity and ambiguity were 0.77 (Table/Fig 1), 0.77 (Table/Fig 2), 0.77 (Table/Fig 3) and 0.73 (Table/Fig 4), respectively. ACP for relevance, simplicity, clarity and ambiguity was 0.95 (Table/Fig 1), 0.95 (Table/Fig 2), 0.95 (Table/Fig 3) and 0.94 (Table/Fig 4), respectively.
In terms of relevance, most of the questions were relevant as per the experts (Table/Fig 1). Eighteen of the 22 questions were graded as relevant but need minor revision or very relevant by all the experts (I-CVI-1, K-1). Four questions were thought to be ‘relevant but needed revision. Only for question no 4, 14, 20 and 22, the experts opinion was divided as few experts gave less than 3 score to these questions but overall pooled relevance was good for these questions.
In terms of clarity (Table/Fig 2) similar findings as relevance were received. In terms of clarity, the same questions came under scrutiny. However, again as relevance, the pooled clarity of these questions was also good and thus the questions were retained after revision. The simplicity of questions (Table/Fig 3) too followed the same pattern and all the questions were retained in terms of simplicity too after modification. In case of ambiguity sixteen questions received I-CVI=1 and K=1, meaning is clear. Again question 4, 14, 20 and 22 came under scrutiny but the views of experts were also divided on question 2 and 3. But Question 2 and 3 received I-CVI=0.88 and K=0.87 which indicate meaning is clear.
Question 4 answers were modified as per expert instructions in terms of relevance, simplicity, clarity and ambiguity. The experts had advised to mentioned junior consultant or Associate Professor and removed postgraduate students from the options. Six out of 16 experts advised inclusion of questions on Apnoeic Oxygenation which was framed in question number 10,11 and 12. Question 14 answer needed minor correction as per experts instructions to specify tracheostomy by surgeon and after re-arrangement, this question became 17 number in final questionnaire. The experts had suggested to add the All India Difficult Airway Association (AIDAA) Guidelines in the answer for question 20. Question 22 needed minor revision as per experts opinion [Annexure-1].
Content validity is the degree to which elements of an assessment instruments are relevant and representative of all aspect of the targeted construct for a particular assessment purpose. The content validation is important to support the validity of an assessment tool like questionnaires (13).
Developing a questionnaire could be a tiring and lengthy process with steps involving developing a questionnaire by identifying the number of dimensions in which one wants the questions, determining their format, development of items, determining the length of questions and developing the initial pool of items. The next most crucial step is content validation (7). Content validation refers to extent to which the items on a test are fairly representative of the entire domain the test seeks to measure (7),(11). In simple terms, content validation refers to the fact that the question or content of a questionnaire is able to express everything which the author wants to ask in simple and understandable language. In the present study, the authors used CVI and MKS to estimate content validity of developed questionnaire [8,9]. This study reports the content validity process of the newly developed questionnaire which might help to collect data of practices in difficult intubation and CICV in anaesthesiologist.
The I-CVI was used because of simple to calculate, easy to understand, provide each item information which can be used for modification, revision or deletion of question. Agreement proportion measured by averaging method (S-CVI/Avg of all I-CVI) and expressed as S-CVI showed 0.94 agreement. S-CVI when measured by universal agreement method (S-CVI/UA=Total number of items agreed relevant by all 16 experts/Total number of items), it showed 0.76 agreement for relevance, clarity, ambiguity and simplicity (14),(15). According to Lynn MR, an I-CVI of 0.78,S-CVI/average of 0.90 and S-CVI/UA0.70 are acceptable when the gradation of the tool is done with more than six experts. So, results in the present study were in congruence with the criteria given by Lynn MR (14),(15),p16]. These results showed excellent content validity of developed questionnaire. Considering these criteria, 18 questions are quantitatively valid.
While the questions 4, 14, 20 and 22 need minor revision or restructuring in questions or answers. The experts suggested to remove postgraduate students from answers in question 4 to remove bias as survey for anaesthesia practitioners and add junior consultant and associate professor. It could be used to collect data about difficult intubation and CICV practices and helps to choose an appropriate method and device by anaesthesiologists to make an informed decision in cases of difficult airways.
In original questionnaire total 19 questions framed. After few experts suggestions, the authors added three questions (10, 11 and 12) on apnoeic oxygenation. The experts had suggested to add AIDAA- 2016 as option in question 20. The CVI though commonly used by researchers, it overlooks chance agreement between experts. To adjust for chance agreement it is suggested to calculate both CVI and kappa statistics (17). In the present study, all 22 questions were in the category of excellent (k >0.75). None of the questions were in fair or poor category.
Limitation(s)
The experts feedback is subjective that lead to bias. If the content domain is not complete, that leads to lack of assessment of data which might have missed from the questionnaire.
Thus questionnaire met the content validity criteria by qualitative and quantitative analyses. The present questionnaire is reliable and valid instrument for data collection on difficult intubation and CICV practices in Indian anaesthesiologists. It can be used in surveys so that a common consensus can be made on the techniques and gadgets to be used in case of difficult airway in India. The authors conclude that the content validation of questionnaire using systematic and evidence based method important before conducting survey so that it covers all relevant parts of the construct it aims to measure.
DOI: 10.7860/JCDR/2023/63545.18101
Date of Submission: Feb 25, 2023
Date of Peer Review: Mar 29, 2023
Date of Acceptance: Apr 28, 2023
Date of Publishing: Jun 01, 2023
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? NA
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Mar 03, 2023
• Manual Googling: Apr 01, 2023
• iThenticate Software: Apr 21, 2023 (13%)
ETYMOLOGY: Author Origin
EMENDATIONS: 7
- Emerging Sources Citation Index (Web of Science, thomsonreuters)
- Index Copernicus ICV 2017: 134.54
- Academic Search Complete Database
- Directory of Open Access Journals (DOAJ)
- Embase
- EBSCOhost
- Google Scholar
- HINARI Access to Research in Health Programme
- Indian Science Abstracts (ISA)
- Journal seek Database
- Popline (reproductive health literature)
- www.omnimedicalsearch.com