Virtual clinics are used to meet the communication and consultative needs of patients and healthcare professionals. They are described as non-face-to-face consultations with health care professionals that can be synchronous (video/telephone) or asynchronous (monitoring/tracking technology).1,2 This paper focuses on synchronous virtual clinics, which are aligned with the traditional outpatient consultation between doctors and/or nurses and patients. These clinics comprise planned contacts involving consultation, monitoring, management, or treatment planning for a healthcare condition.3 Virtual clinics can be a collaborative effort between health care professionals4 or led specifically by nurses5 or doctors.6 Due to their accessibility, feasibility, and safety, virtual clinics are acceptable to patients, with research demonstrating high levels of patient satisfaction.4,6–9
In Ireland, before the coronavirus disease 2019 (COVID-19) pandemic, with a few exceptions, there was a lack of published literature, guidelines, or policy relating to the delivery of virtual clinics.9–11 However, the contact restrictions imposed due to the pandemic required healthcare organizations and staff to adapt and reconfigure healthcare services with virtual clinics to the forefront of this strategy in Ireland, with the Health Service Executive (HSE) publishing guidelines for the management of virtual outpatient clinics in 2020.3 Studies undertaken since the emergence of COVID-19 concluded that virtual clinics are a viable option to continue patient care when used appropriately, allowing patient access, decreasing the need for hospital visits, and reducing the risk of cross-infection.12–15
Nurse-led or co-ordinated virtual outpatient clinics are now a familiar component of healthcare delivery across many disciplines, including cancer care, orthopedics, rheumatology, and gastroenterology.7,16–18 However, establishing a nurse-led virtual clinic is not without its challenges for nursing management, particularly in the areas of funding, administrative support, and provision of appropriate technology. Managing the structure of nurse-led clinics is also important, including having in place appropriate pathways, standard operational protocols, safety checklists, and a pro forma guiding the virtual clinic consultation.7,19 Given the limited focus of research on nurse-led virtual clinics and the relatively new ways that nursing care is delivered, there is a need to examine nurses’ practices and processes in the delivery of these clinics and patients’ satisfaction in participating in such clinics.
This study aimed to investigate nursing practices and processes and patient experiences in relation to virtual outpatient clinics in an acute care hospital. Identify barriers and enabling factors for nurse managers to support nurses in developing and delivering virtual clinics.
This was a cross-sectional, descriptive study using mixed data collection methods. The setting for this study was an acute care hospital in Ireland. All patient virtual clinics co-ordinated and/or facilitated by nurses were included in the study. In total, 4 virtual clinics met these criteria (fracture, gastroenterology, diabetes, and rheumatology clinics).
The sample size for the Virtual Clinics Patient Satisfaction Questionnaire (VCSQ) survey was informed by estimates of those scheduled and who attended virtual clinics from 17th March 2020 to 18th October 2020 (n = 2012), as all 4 clinics were in operation during that period. A total sample of 324 was needed to ensure adequate study power. The inclusion criteria included those >18 years of age, who attended virtual clinics from March 2020 and were able to give informed consent.
Purposive sampling was used for the focus group interview with nurses who led or facilitated a virtual outpatient clinic. Given the low number of nurses involved in virtual clinics at the research site, one focus group interview was conducted (n = 5 nurses). Participating nurses were recruited by a gatekeeper who was independent from the research site.
Audit: The audit form, designed by the research team, was based on current practices in established virtual clinics. Its content validity was independently assessed by expert nurses and included content validity index calculation. The final instrument consisted of 4 sections examining clinic data, personnel, patient support, and resources. The audit was conducted from 17th March 2020 to 18th October 2020.
Documentary Analysis: The framework for the documentary analysis was informed by Kerr et al.20 and required information in relation to the category, purpose, procedure, and personnel using any relevant policies or guidelines for the virtual clinics.
VCSQ: This is an 11-item, 5-point Likert scale instrument with scores ranging from 1 (very dissatisfied) to 5 (very satisfied). The questionnaire was adapted from an instrument used at the research site and from the literature. Additional questions were added relating to satisfaction with communication, clinical competence, duration, technical aspects, convenience, and overall satisfaction. A pilot study with a convenience sample (n = 5) was conducted, and content validity was assessed by an expert panel, which included content validity index calculation. A reliability analysis for internal consistency reported a Cronbach’s alpha of.92.
Focus group interview: Explored nurses’ experiences of developing and delivering virtual clinics. The interview guide for the focus group interview was informed by findings from the audit and documentary analysis.
All 4 virtual clinics provided audit data covering the period from 17th March to 18th October 2020. Nurses from the 4 participating virtual clinics completed the electronic audit and documentary analyses forms.
In relation to the VCSQ survey, eligible patients were contacted by telephone, informed of the study, and asked to consider participation. Study information packs were posted to those patients who agreed to receive the pack and a follow-up telephone call was made 1 week later. During the second call, verbal consent was obtained and the electronic survey completed.
The focus group interview was conducted via a Zoom call and recorded using a digital recorder and transcribed verbatim. A member of the research team facilitated the interview using an interview guide with a second team member acting as moderator; both were independent of the research site. Participants engaged in a group discussion on their experiences’ of developing and/or delivering virtual clinics.
The quantitative data collected from the audits and patient satisfaction survey data were entered into Excel 2016 (Microsoft Corporation, Redmond, WA, USA) and IBM SPSS Statistics Version 25 (IBM Inc., Armonk, NY, USA) for descriptive and inferential analyses where possible. Categorical data were analysed by counting the frequencies (number and percentages), and continuous variables reported using mean, median, and standard deviations (SDs). The qualitative data were transcribed and analysed using thematic analysis.
Ethical approval was granted from both the hospital and university relevant research ethics committees in September and November 2020.
Over the 7 months of data collection, 200 clinics were held for a total of 2019 scheduled patients (Table 1). Data from the 4 specialist virtual outpatient clinics showed that 23% of patients did not attend their scheduled virtual clinic. All the clinics were delivered using the telephone. None of the nurses involved in the clinics received formal preparation or training to facilitate a virtual clinic. Although all the clinics had some level of administrative support, it was limited, and none of the nurses had scheduled/protected time to undertake administrative duties linked to running the virtual clinic. Technical support used to deliver the virtual clinics was limited to telephone, email, and digital health records when available. Access to a reliable virtual platform for face-to-face consultations with equipment such as a webcam and headset was recorded as desirable by nurses.
Audit of virtual clinics.
| Item | Result | |
|---|---|---|
| Clinic Data | ||
| Number of clinics | Total 200 | |
| Mode of communication | 100% telephone | |
| Duration of consultation | Not recorded | |
| Duration of preparation, running, and follow-up of virtual clinic | 0.5—12 h | |
| Average number of patients per week | 9–30 patients | |
| Total number of patients scheduled | 2019 | |
| Total number of patients attended | 1555 | |
| Clinic personnel | ||
| Lead nurse—years qualified | Range 7–36 years | |
| Lead nurse—highest qualification | 50% had masters | |
| Virtual clinics training | 0 | |
| MDT involvement | 75% had MDT involvement | |
| Patient support | ||
| Informal psychological support | 75% provided | |
| Referrals | 17 different services identified | |
| Patient satisfaction assessed | 25% informally assessed | |
| Resources | ||
| Nurses location | Variety of locations; outpatients department, office, nurse’s home | |
| Admin support | Not recorded. Estimates range 1–10 h | |
| Protected/scheduled time for virtual clinic | 66% | |
| Technical support used | Telephone/email/digital health records, | |
| Technical support desired | Access to reliable virtual platform for consultations Webcam/headset | |
Note: MDT, multi-disciplinary team.
Two of the four virtual clinics did not have any documentation. For the other 2 clinical areas, there were a total of 9 documents listed, with 2 documents completed and 7 at draft or pilot stage (Figure 1).
Documents supporting virtual clinics.
The overall purpose of the documents was to ensure clear guidance, direction, and consistency in delivering patient care. More specifically, they detailed the development, delivery, and follow-up for patient care in virtual clinics.
In total, 341 participants took part in the telephone survey using the VCSQ. The mean score for each of the 11 items ranged from 4.16 to 4.5, showing that all the items scored on average in the “satisfied” or “very satisfied” points on the scale (Table 2).
VCSQ Scores.
| N | Question how satisfied are you….? | Mean | SD |
|---|---|---|---|
| 1. | Purpose of virtual clinic appointment was explained | 4.5 | 0.9 |
| 2. | Contacted in a timely manner | 4.49 | 0.93 |
| 3. | Diagnosis/treatment fully explained | 4.21 | 1.1 |
| 4. | Nursing/medical care/advice received | 4.41 | 0.98 |
| 5. | Duration of virtual clinic appointment | 4.42 | 0.96 |
| 6. | Technology used and ability to take part | 4.41 | 0.89 |
| 7. | Convenience of virtual clinic | 4.49 | 0.91 |
| 8. | Concerns/problems were addressed | 4.2 | 1.17 |
| 9. | To be treated through virtual clinic in the future | 4.16 | 1.2 |
| 10. | Access further nursing/medical advice | 4.27 | 1.08 |
| 11. | Overall satisfaction | 4.4 | 0.95 |
Note: SD, standard deviation; VCSQ, Virtual Clinics Patient Satisfaction Questionnaire.
The items are presented in descending order with the “purpose of virtual clinic appointment was explained to you” having the highest score for satisfaction. The final item asked about overall satisfaction, which is recognized as a standalone measure of satisfaction. The breakdown of scores demonstrates the high percentage (86.3%) of participants who were satisfied and very satisfied overall with the virtual clinics.
The virtual clinics which participants attended comprised of nurse-led virtual clinics (22%) and multidisciplinary team-led clinics (78%). Examining the distribution of scores on the Likert scale and the mean scores for each item by the type of clinic indicated differences between the nurse-led and multi-disciplinary team (MDT) clinics. With the exception of item 2 (satisfaction with being contacted in a timely manner), the nurse-led clinics had a higher patient satisfaction score. Mann-Whitney tests revealed 4 items had significant differences, with higher satisfaction reported for the nurseled virtual clinics on the following items (Table 3):
duration
technology used and ability to take part
concerns/problems addressed
knowing how to access further care and advice if needed.
Difference between nurse-led and MDT clinics.
| Clinic type | n | Mean score | SD | Mann–Whitney U | P |
|---|---|---|---|---|---|
| 1. Purpose was explained | 9195 | 0.201 | |||
| Nurse-led | 75 | 4.53 | 0.96 | ||
| MDT | 266 | 4.48 | 0.88 | ||
| 2. Contacted in a timely manner | 9357 | 0.318 | |||
| Nurse-led | 75 | 4.48 | 1.07 | ||
| MDT | 266 | 4.49 | 0.89 | ||
| 3. Diagnosis/treatment explained | 9018 | 0.160 | |||
| Nurse-led | 75 | 4.35 | 1.02 | ||
| MDT | 266 | 4.17 | 1.13 | ||
| 4. Nursing/medical care/advice | 9512 | 0.471 | |||
| Nurse-led | 75 | 4.43 | 1.03 | ||
| MDT | 266 | 4.41 | 0.96 | ||
| 5. Duration | 8057 | 0.003** | |||
| Nurse-led | 75 | 4.68 | 0.70 | ||
| MDT | 266 | 4.35 | 1.02 | ||
| 6. Technology used and ability to take part | 7651 | 0.000*** | |||
| Nurse-led | 75 | 4.65 | 0.81 | ||
| MDT | 266 | 4.35 | 0.90 | ||
| 7. Convenience | 9635 | 0.579 | |||
| Nurse-led | 75 | 4.51 | 0.74 | ||
| MDT | 266 | 4.49 | 0.90 | ||
| 8. Concerns/problems were addressed | 8377 | 0.017* | |||
| Nurse-led | 75 | 4.44 | 1 | ||
| MDT | 266 | 4.14 | 1.20 | ||
| 9. Future | 8757 | 0.069 | |||
| Nurse-led | 75 | 4.37 | 1.04 | ||
| MDT | 266 | 4.1 | 1.20 | ||
| 10. Access further nursing/medical advice | 7787 | 0.001*** | |||
| Nurse-led | 75 | 4.57 | 0.93 | ||
| MDT | 266 | 4.18 | 1.11 | ||
| 11. Overall satisfaction | 9021 | 0.141 | |||
| Nurse-led | 75 | 4.52 | 0.88 | ||
| MDT | 266 | 4.37 | 0.97 |
Note: MDT, multi-disciplinary team; SD, standard deviation;
P < 0.05,
P < 0.01,
P < 0.001.
Five nurses took part in the focus group interview: 1 advanced nurse practitioner (ANP), 3 clinical nurse specialists and one staff nurse. Thematic analysis revealed 3 themes.
Nurses’ perceptions of nurse-led virtual clinics
Benefits of virtual clinics
Barriers to nurse led virtual clinics.
Theme 1—Nurses perceptions of nurse-led virtual clinics
The rheumatology nurse-led virtual clinic, operated across 2 sites, and the fracture clinic appeared to be organized and efficient– this is probably due to the electronic health record (EHR) system in place.
“we have an electronic patient record that’s shared between hospitals. So it means then even if you’re off site you can access the patient records so if you get an unscheduled issue, you can address that and document the interaction as you go” (P1)
Nurses were generally positive about the development of nurse-led virtual clinics
“In one way it’s a positive thing from COVID because, I suppose, it’s something that we probably would have talked about for years but never got around to doing” (P4)
Some nurses felt that many patients seemed more comfortable using the phone, which was considered sufficient for the purpose of the nurse consultation, especially if the patient was already known to the nurse.
“patients were usually familiar with their condition and they’re able to describe to you what they’re feeling or what they’re seeing, you know, and we’d have a good relationship already” (P1) Theme 2—Benefits of virtual clinic
Telephone consultations were reported to be more focused than face-to-face interactions because proformas were used in some clinics to ensure standardization and consistency.
“when you’re on the phone you stick to the checklist of the pro forma. And I suppose you don’t feel bad for having to look at the pro forma” (P1) “it’s time saving because you’re not inclined to have the chat” (P3)
Benefits for nursing and patient services included improved interprofessional collaboration, which was evident in some clinics (orthopedic). This resulted in a greater number of patients being seen in a shorter time-period.
“I found that I learned a lot from sitting in with them, with the doctor and the physio and that, I learned a lot by looking at the injuries and looking at the x-rays and the plan of action” (P2)
Decision-making skills and autonomy were also noted as being enhanced by both nurse-led and MDT virtual clinics.
“it makes you more autonomous because you have to be, it probably encourages us more to be more decisive and ask more questions. And when it’s on the phone, you have to train yourself to kind of tune in to the nuances” (P1) Theme 3—Barriers to nurse led virtual clinics
Some patients did not fully understand their role in terms of ensuring that blood and other samples were provided before the virtual appointment so that results would be available for the consultation.
“they’re [patients] not really being properly reviewed because they’re not doing their part” (P4)
Language was also a barrier to conducting virtual clinics. Nurses preferred face-to-face interactions to overcome this problem.
“I try to do a cognitive assessment using a translator over the phone and it’s a disaster, you know …, anyone who needs a translator, you need to bring them in” (P5)
Other barriers identified included a lack of adequate IT support and a reliable internet supply.
“IT systems would need to be upgraded as well and we’d need better quality machines to be able to… we’d need more access to laptops and everything else like that” (P2)
Lack of clerical support was also identified as a potential barrier to delivering virtual clinics.
“So there’s quite a lot that we could use the added clerical support for. It would save a lot of time if they had that done and that frees up to do our nursing work then” (P3)
The Virtual Clinics Patient Satisfaction Questionnaire (VCSQ) survey showed a very high satisfaction rate with virtual clinics, particularly with nurse-led clinics.21 The item “purpose of the virtual clinic appointment was explained to you” scored the highest level of satisfaction. This is a very encouraging finding as virtual clinics at this study site are relatively new, with some virtual clinics emerging during the COVID-19 pandemic. The study also found high satisfaction in “diagnosis and treatment fully explained” and “concerns/problems were addressed.” This reflects findings by Beaver et al.22 while exploring patients’ experiences of a telephone clinic after colorectal cancer, found that patients were more likely to ask specific questions, which led to a better understanding of their condition and treatment.
The main communication medium used for the virtual clinics was the telephone, using a structured and consistent approach, guided by using proformas, which was viewed as beneficial as it contributed to a standardized form of consultation. Having this structured and organized form of consultation promotes a standardization of care.23 However, it was noted that these more focused conversations reduce the opportunities for informal “chat” and social interaction and may have influenced participants (13%) reporting dissatisfaction in “being treated through a virtual clinic in the future.” Similar findings are reported by Byravan and Sunmboye24 who noted 8% of patients were not satisfied to participate in a virtual nurse led clinics in the future. This is reported also by Lun et al.25 who gave patients the option to use a virtual platform such as zoom or a telephone service, but only 7% (8/118) of patients reported not wishing to participate in virtual care or felt this type of clinic was unsafe. The research suggests that a virtual outpatient clinic may not be for everyone, with a small cohort of patients opting to attend face-to-face in the future, and as such, these patient preferences should be accommodated. An alternative strategy for patients may be a combination of both virtual and face-to-face appointments.
The quantitative and qualitative findings from this study highlight the infrastructure and resources needed to deliver effective, efficient, and sustainable synchronous virtual clinics. This includes dedicated and sufficient administrators, IT support, education/training, and relevant policies/procedures. This study demonstrated that although administrative support was available to all 4 clinics, the number of hours provided varied, and only one of the clinics had protected time for nurses to engage in clinic-related administrative duties. This increased the workload and time commitment for nurses who have had to subsume administrative tasks into their normal working day. Administrative support is vital to the efficient running of a virtual clinic and as such, clerical and administrative staff should form part of the nurse-led virtual clinics team.6,18
There was a wide variation and inconsistencies amongst the different virtual clinics in relation to the available IT support used to deliver and administer the virtual clinics. Across current and future virtual clinics, there is a need for the standardization of communication and technological equipment and supports that are deemed necessary for the successful operationalization of virtual clinics. There is also a need for collective access to relevant and up-to-date software, for example, booking appointments, diagnostics, capturing all virtual interactions, EHR across all sites, stable internet connectivity, and readily available technical support. The audit and focus group highlighted the challenges of planning and implementing virtual clinics without the necessary IT systems.
The documents analyzed and reviewed for this study were linked to only 2 of the virtual clinics and were in draft format. Patient advice and information instructions were also included as a key aspect of the documents. The policies dealt with both singular individual clinic staff and multi specialities. More focused policies dealt with prescribing both medication and radiation therapy in the virtual environment without the patient being physically present. Supporting documentation needs to be available in advance of the establishment of new virtual clinics or in the development/expansion of existing virtual clinics. These documents ensure a transparent and standardized approach to establishing and maintaining effective and efficient virtual consultations.26
Developing and standardizing documentation in conjunction with improved auditing of virtual clinics emerged from this study as key to quality control and successful virtual clinic implementation.27 This is supported by the documentary analysis conducted in this study, which demonstrated the limited breadth and depth of documents used in virtual clinics. When available, these documents provided hospital staff with a framework for the delivery of safe and consistent patient care. However, these have yet to be developed for all virtual clinics to ensure a standardized manner across the hospital.3
A multidisciplinary approach was also identified in this study as a key factor in improving the efficiency and effectiveness of virtual clinics. The qualitative data showed that including other health care professionals from the outset and providing them with relevant education would improve overall communication and engagement between the healthcare disciplines in these clinics. The early engagement of all relevant stakeholders can be critical to the successful planning of virtual clinics.17 Furthermore, the focus group advocated that providing education and training both for nurses and across teams and disciplines would contribute to building a team approach and develop peer support. In their review of nurse-led virtual clinics, Almeida and Montayre23 recommended that nurses undertake advanced communication and technology skills training as these skills are key to delivering high quality nurse—led virtual consultations that are both safe and efficient. Initiatives such as the HSE Telehealth Roadmap 2024–2027 and the guidelines for the management of virtual clinics provided by the HSE demonstrate awareness of the issues that need to be addressed in order to support nursing staff and other health professionals who are running or facilitating virtual outpatient clinics in Ireland.3,28 These resources need to be fully supported by nurse managers in terms of ensuring that nurses have access to these programmes and have time built into their working day to complete all necessary training.
The findings of this study highlight to Nurse Managers that virtual care is an integral part of how a healthcare service can provide a more accessible and high-quality service for people with diabetes and potentially other chronic conditions. The success of nurse-led virtual services requires key infrastructure to support nursing staff and sustain this service, for example, administration support, reliable and consistent high-speed internet connectivity services, up-to-date software, including information technology (IT) hardware that links MDTs and EHRs. Clear policies, procedures, and guidelines need to be available and training/education for nurses running virtual clinics.
Education for undergraduate nursing students and postgraduate nurses related to virtual clinics, digital health, and informaticsis essential to maintain and sustain this type of nurse-led service. This will provide nursing students with the informatics skills, attitudes, and knowledge necessary to provide effective and efficient virtual care for patients.29,30 Lo et al.31 suggest that nurse managers also need digital health knowledge and skills in order to improve their capacity to lead the development of virtual services for patients.
The findings from this study suggest that further research is required to identify the financial implications and benefits of virtual clinics and their impact on service and patient outcomes, including their impact on the incidence of emergency department presentations, admission avoidance, and reduced hospital stays. A more detailed exploration of the patient experience of nurse-led virtual clinics is needed, and its impact on healthcare outcomes.
This study provides objective and subjective data that identify issues linked to nurses’ practices in developing, delivering, and facilitating virtual clinics in a hospital in Ireland. The study took place in one acute care hospital setting, which limits its generalizability. It did, however, involve 4 virtual clinics which were either nurse-led or were MDT clinics facilitated by nurses. It captured 341 patients’ level of satisfaction with an outpatient service that was delivered virtually and has applicability to similar healthcare settings. The impact of response bias in terms of social desirability or respondent acquiescence (i.e., providing the response that one feels is the “correct” or desired response) must be considered here. Response bias may lead to an overestimation of patient satisfaction, and the effect may be inversely related to responders with the lowest levels of satisfaction.32,33 Nonetheless, this study provides relevant information to nurse managers in relation to nurses’ involvement in the development and implementation and barriers and facilitators to nurses continued involvement in providing virtual clinics. The exploration of cost implications in our study was not feasible. Analysis of the potential financial implications associated with virtual clinic delivery needs to be addressed in further studies, for example, in relation to reduced waiting lists and the avoidance of hospital admissions and general practitioner (GP) visits. Another issue which this study did not address was whether the non-attendance rate was lower for virtual clinics, which also has cost and service implications.
A further strength of this study is the development and reliability testing of the VCSQ, a questionnaire that is solely focused on patient satisfaction with virtual clinics. Reliability testing demonstrated that this survey tool was highly reliable. The large sample size is comparable to other research exploring patient satisfaction with outpatient clinics. Conversely, the relatively small sample size in relation to the focus group conducted is recognized as a potential weakness of this study, albeit a result of the low number of nurses involved in virtual clinics at the research site. The optimal size for focus groups is generally considered to be between 6 and 8 participants, and although they can work well with as few as 3 participants, smaller focus group numbers may limit discussion, and subsequently limit findings.34–36 A larger sample may have enriched the findings by providing additional viewpoints. However, the focus group data was analyzed using inductive thematic analysis to limit the possibility of researcher bias, and the focus group was facilitated and moderated by 2 members of the research team who were both independent of the research site to ensure objectivity. This approach strengthens the veracity of the findings despite the small sample size.
There is evidence that virtual clinics, particularly nurse-led virtual clinics, can be an effective and efficient way to provide care to patients. However, when developing and expanding this type of patient service, nurse managers need to provide supportive structures for nurses, such as dedicated administrators, IT support and infrastructure, education/training, and relevant policies/procedures. Without this, the focus on patient care can be negatively impacted and potentially increase the administrative workload on nurses.