The availability of Big Data and the increase in the computing power of machines in recent years have led to significant advances in the field of AI. These advances have given rise to a series of discussions and debates, mainly speculative, attempting to anticipate and identify the consequences and contributions of these advances for work [Zouinar, 2020]. Thus, while some people express fears of a widespread replacement of employees by AI-based systems, others believe that the major impact would be more reconfiguration rather than automation and the mass disappearance of jobs [CoE, 2017]. Another perspective that argues against replacement is to emphasize complementarity between AI and humans: AI could help workers (for example, by taking on repetitive or tedious tasks, by facilitating tasks or by “increasing” their capacities for action), just as humans could help AI systems through what they could teach them [Daugherty & Wilson, 2018]. There are still few empirical studies making it possible to understand the actual uses of AI-based systems and their real consequences in work situations.
In addition to these forward-thinking discussions, there are several questions to be answered: how are these systems deployed and used in real work situations? What do they make possible? What, specifically, does their introduction change in professional activities? What challenges and problems do they raise for workers? The aim of this article is to provide answers to these questions through two case studies in the field of radiology which, in particular, is having to face up to this technological development [Anichini & Geffroy, 2021].
The two case studies show that the uses and deployment of AI systems have varying impacts, depending on the type of system and the organizational context.
From a methodological point of view, a qualitative approach, based on interviews and observations, has been implemented. The first case concerns a Voice Recognition (VR) dictation system in a hospital medical imaging department (Figure 1). The system automatically transcribes the dictated text. Interviews and observations were carried out in real work situations in this department involving a variety of professionals: radiologists, technologists and secretaries, as well as computer scientists and healthcare executives.
Figure 1: Use of VR Dictation by a Radiologist
The second case concerns a Fracture Detection (FD) system (Figure 2) used in conventional radiology (X-ray imaging). Several radiologists and some emergency physicians were interviewed and observed during their work in three private practices and three hospitals, which are different establishments from the one in which the VR dictation system was studied.
Figure 2: the Fracture Detection System.
The Case of Voice Recognition (VR) Dictation
The rollout of VR dictation in the imaging department concerned coincides with the plan to reorganize the department’s secretariat. The reorganization is directly related to the high workload of secretaries in terms of transcription of radiological reports that have been dictated and recorded by radiologists. Integrated into the new version of their image archiving and transmission system rolled out at the time of the secretariat reorganization project, VR dictation provided an opportunity to address the problem of significant delays in transcription. The implementation of this system required a rethink of the way in which work is organized. While, prior to its implementation, secretaries transcribed and corrected reports recorded on cassettes or in digital format, the dictation system now performs the transcription task and the radiologist often corrects their report personally (Figure 3). This new way of organizing work was guided by the objective of saving time in the report production process.
Figure 3: Distribution of Tasks with and without VR Dictation
Time is Saved but there are Difficulties in terms of Use for Radiologists
The use of VR dictation by a radiologist is part of a rather complex and repetitive activity, namely the creation of a report. In order to analyze, cross-reference and summarize a range of information, this activity involves going back and forth between the clinical case of the patient, the request and questions of the doctor requesting the radiological examination and the images from the examination. The report is therefore created progressively, switching between the analysis and summary of these different pieces of information. Within this complex process, the radiologist may experience some difficulties in using voice recognition dictation. For example, they may find it necessary to systematically check and correct the transcribed text. Indeed, several radiologists have commented on inadequate system performance: these are cases of incorrectly transcribed words or grammatical errors. Some of them found the way in which the system learns – which is necessary for its proper functioning and the improvement of its performance – to be rather restrictive. First, the radiologist must configure their voice profile to allow the system to recognize their voice. However, what is most problematic are certain voice profile (re)configuration requests from the system that are not always received at a convenient time:
“What also bugs me is that it very often asks me to record a text so it can recognize my voice profile (…) sometimes it is not a problem, but sometimes you want to start up the computer because you are being asked for an opinion over the phone and there’s this thing that happens at startup.” (radiologist 1)
Despite these difficulties in using the system, several benefits have been highlighted by radiologists. First of all, it helps to eliminate delays in the transcription of reports, thus improving the care of patients who can obtain their reports quickly. Overall, the review and correction steps for the reports are simpler compared to the previous situation, in which back and forth between secretary and radiologist could take several days. Another benefit of using VR dictation could be seen in the formatting and construction of the report itself. In other words, with the interface of this system, radiologists can now view, adapt and better reorganize their ideas in real time.
A Transformed Collective and a Risk of Deterioration in the Quality of the Reports
However, the introduction of VR dictation is not without consequences on activities and the way in which work is organized on a more global level. Indeed, this system has opened up possibilities for reorganizing work in the face of organizational problems affecting the department (workload and transcription delays). Its introduction has helped to redefine the way in which work is organized between secretaries and radiologists. The new way in which work is organized resulted in the disappearance of the bulk of the secretaries’ work, i.e. the work of transcribing the reports. This development has been received poorly by some of them. This task – known as typing work – helped them to develop their medical knowledge:
“Even in people’s illness, despite the misfortune and very serious things, we were still learning about anatomical anomalies. We were gaining knowledge. Now, we no longer have this learning opportunity.” (secretary 1)
Changes are also evident at collective level: before VR dictation, a report was akin to collective work produced jointly by radiologists and secretaries, through which a solid relationship, particularly one of trust, was built between the two. According to the interviewees, with the new way of working, that relationship has now been lost. It takes another form and involves other tasks such as ad-hoc discussions or the management and organization of radiologists’ schedules by secretaries. In any event, this transforms their relationship:
“It was a relationship of trust as well (…) we are so used to certain turns of phrase with physicians, because we more or less always type for the same physicians (…) it is also a rapport and, well, we miss that.” (secretary 2).
In addition, the risk of deterioration in the quality of the reports is mentioned by some radiologists, as well as some secretaries, who note multiple transcription problems. For example, poorly recognized words that can change the meaning of sentences or make them unintelligible. These transcription problems can undermine the feeling among radiologists of a “job well done”:
“Sometimes you can let something slip by that doesn’t mean anything at all because you’ve reread it, but you have tired eyes and then you’ve skipped a sentence, so something bad slips through, which does not mean anything (…) this makes it necessary to make more condensed reports; previously I could use eloquent turns of phrase, now I no longer do that.” (radiologist 2)
In addition, it would seem that this task of re-reading and continually monitoring the text transcribed by the system tends to increase the workload of these radiologists.
The case of the Fracture Detection system (FD)
Unlike the VR dictation system which handles the automatic transcription task, the fracture detection system does not complete a task as such. It provides additional analysis to detect the presence or absence of fractures on conventional (X-ray) radiology images. In this manner, it plays a role in the interpretation of the images to be analyzed. According to the radiologists interviewed and observed for this second case, the introduction and use of this system meet different objectives in relation to the socio-organizational issues and the evolution of their profession:
- to monitor the evolution, particularly the technological evolution, of radiology, to participate in rather than undergo that evolution and to remain competitive and attractive: “not to fall behind”;
- to address the problem of the shortage of radiologists (particularly in hospitals) and the increase in their workload, leading to the deterioration of working conditions (with a risk to health and performance), which can lead to errors and reduced performance in image interpretation and diagnosis;
- to help emergency physicians with an initial reading of conventional radiology images, which are not always interpreted by radiologists due to an excessive workload.
Cautious confidence in the system and a risk of time being wasted in the work
By offering automatic analysis of radiological images, this system provides three types of information that are fed back to the radiologist without requiring any specific processing: absence of a fracture, doubt regarding one or more fractures and presence of one or more fractures. The radiologist can then dismiss or confirm the system’s analysis in a few seconds. However, this information does not relieve them of the interpretation work that they must carry out (analyzing all the relevant elements, looking at all the images, zooming in, zooming out etc.). Without the system, the radiologist would be alone in front of the images, having to judge which elements to observe and in which order. With this system, the interpretation of images is often guided and oriented by the information it provides. However, the radiologist is always the only one who assesses the relevance of this information with regard to the clinical case and the questions of the requesting physician. On this point, it turns out that the information provided by the system is not always useful or relevant. For example, the system’s doubts would often concern lesions with no impact on the patient’s healthcare. The inclusion of those elements in the report depends on the radiologist’s judgment regarding the clinical relevance of the case analyzed. Some radiologists explain that they would not have paid attention to the lesions detected without the use of the system. Another example is the case of examinations to monitor the evolution of fractures. In this case, the system systematically detects a fracture. It is therefore efficient but irrelevant and not useful for the radiologist who must perform an analysis and give their assessment regarding the evolution of this fracture and not its presence.
The use of the system also raises issues relating to radiologists’ confidence in it. This confidence seems to go through several phases and to develop as the system is used and put to the test. Some radiologists have expressed a tendency to first interpret images independently and then check the results of the system. For some, this trend fades over time as they begin to understand the performance and limitations of the system: instead of interpreting the images alone, they will then go straight to the results of the system in cases in which they are looking for one or more fractures and for which they know that the system performs well. Therefore, radiologists also mention the risk that, at certain times, this confidence may become over-confidence. Using the system requires maintaining control over the interpretation of the images and questioning the information provided to avoid this pitfall. However, this requirement raises another potential instance of time being wasted. This is the case, in particular, in the case of false positives, which can cause the radiologist to have doubts. Thus, they would tend to spend time searching, examining the images and areas indicated by the system when, in reality, there is no fracture, for example. In addition, the radiologist can always perform additional examinations (for example, CT scan, ultrasound scan etc.) if there is a discrepancy between their opinion, the results of the system and the patient’s clinical case.
In the “current”[1]version under observation, the system can only detect fractures, whereas interpreting radiological images means analyzing many other elements in order to reach a diagnosis. In this respect, the increase in radiologists’ confidence in the system also entails a risk of neglecting other aspects of the images and remaining focused on the information provided by the system alone, as this radiologist explains: “sometimes, people have so much confidence in [the FD system] that they no longer look at dislocations” (radiologist 3).
While being mindful of a risk of over-confidence and the need for monitoring, for some radiologists, the main contribution of the system is to protect them against possible oversights in the interpretation of the images. This is why this system is mostly described as “a safety net” or “a second reader”, making it possible to reduce the stress on the radiologist, caused by a fear of missing something on the images (which are sometimes difficult and numerous to analyze) and the risks that this raises in respect of their liability. The system would thus find a place, in particular, when radiologists are working in sub-optimal conditions, such as when working at night or when tired.
While one of the initial objectives of using the system was to save time in the process of interpreting images, this saving is difficult to assess in practice. This is because contributions do not always translate into significant time savings:
“For now, it’s almost a waste of time in the workflow, we really use it more for quality than for speeding things up. Since the business model at the outset was pretty much ‘we will save radiologists time’, etc. So far, this is not what we’ve experienced.” (radiologist 4)
Potential help for emergency physicians and radiographers
Indeed, the way in which the work of radiologists is organized is not directly transformed by the introduction of the system (at the time of our observations) but, according to some interviewees, its functions could open up possibilities for the reorganization of the work of emergency services and to help emergency physicians. For example, in the event of doubts, instead of waiting for the interpretation of the images by a radiologist, the emergency physician can refer to the results of the system in order to have their diagnosis confirmed and make an appropriate decision on treatment. It is also proposed that access to the results of the system be examined for radiographers during image acquisition. The objective would be to allow these radiographers to alert the radiologist in certain cases without waiting for them to consult the images from the examination. Currently, they can certainly do so in case of emergencies identified during image acquisition, but this is not systematic.
Conclusion
Recent advances in AI have raised many questions about its impact on work and organizations. These questions have been the subject of many examinations, mainly speculative, which focus either on damaging consequences or on the benefits. The two empirical studies presented here on the use of voice-recognition dictation and the fracture detection system in the medical imaging sector allow us to provide nuance and clarification to these examinations. Firstly, socio-organizational issues constitute a major element in the motivations and decisions to introduce an AI system, which often opens up a range of possibilities to reconsider the ways in which work is organized. The introduction of these systems is not neutral and can result in several direct or indirect transformations of work activities. In the case of radiology, the use of VR dictation by radiologists and the redefined way in which work is organized result in a change in the activity involved in the creation of a radiological report, which also results in significant consequences for the work of secretaries (e.g. loss of skills development). In the case of the FD system, even though this system does not play a direct role in the radiologist’s diagnosis, the information it provides seems to have an influence on the interpretation of the images. Thus, the consequences are more along the lines of the transformation of certain parts of the activities rather than a “replacement” of professionals, such as radiologists whose work of interpreting images cannot be reduced to the mere detection of fractures, for example.
Finally, in some situations, the introduction and use of AI systems seem to come up against two paradoxes: (i) instead of saving time, their inadequate performance or low relevance to the context of the activity may instead cause the worker to lose time when they need to verify, control and monitor the results of the system; (ii) instead of improving quality and performance, the use of an AI system could lead to an increase in workload and a decrease in performance, introducing errors that the introduction of the system specifically aims to reduce.
The two case studies show that the uses and deployment of AI systems have varying impacts, depending on the type of system and the organizational context. Contrary to promises made, these impacts are not always beneficial. This once again underscores the importance of anticipating and evaluating these impacts as early as possible and as closely as possible to the real work context, involving the people concerned in order to work together in the transformation of their work. The assessment of these impacts should be an ongoing process within organizations, while retaining opportunities to question the benefit of AI systems in the relevant work activities at any time.
[1] The FD system will evolve, incorporating other functions such as the detection of other anomalies, such as dislocations.
Sources :
Anichini, G. & Geffroy, B, “L’intelligence artificielle à l’épreuve des savoirs tacites. Analyse des pratiques d’utilisation d’un outil d’aide à la détection en radiologie”, Sciences sociales et santé, 39(2), 2021, pp. 43–69.
COE, Automatisation, numérisation et emploi. Tome 1: Les impacts sur le volume, la structure et la localisation de l’emploi, 2017, Paris.
Daugherty, P.R. & Wilson, J. Humans + Machine: Reimagining Work in the Age of AI, 2018, Harvard Business Review Press, Boston.
Zouinar, M. “Évolutions de l’Intelligence Artificielle : quels enjeux pour l’activité humaine et la relation Humain‑Machine au travail ?”, @ctivités, 17–1, 2020.
Tamari Gamkrelidze
Moustafa Zouinar
