Full article title Development of a core competency framework for clinical informatics
Journal BMJ Health & Care Informatics
Author(s) Davies, Alan; Mueller, Julia; Hassey, Alan; Moulton, Georgina
Author affiliation(s) University of Manchester, University of Cambridge, The Faculty of Clinical Informatics, Health Data Research UK
Primary contact alan dot davies at ucl dot ac dot uk
Year published 2021
Volume and issue 28(1)
Article # e100356
DOI 10.1136/bmjhci-2021-100356
ISSN 2632-1009
Distribution license Creative Commons Attribution-NonCommercial 4.0 International
Website https://informatics.bmj.com/content/28/1/e100356
Download https://informatics.bmj.com/content/bmjhci/28/1/e100356.full.pdf (PDF)

Abstract

Objectives: Up to this point, there has not been a national core competency framework for clinical informatics in the U.K. Here we report on the final two iterations of work carried out towards the formation of a national core competency framework. This follows an initial systematic literature review of existing skills and competencies and a job listing analysis.

Methods: An iterative approach was applied to framework development. Using a mixed-methods design, we carried out semi-structured interviews with participants involved in informatics (n = 15). The framework was updated based on the interview findings and was subsequently distributed as part of a bespoke online digital survey for wider participation (n = 87). The final version of the framework is based on the findings of the survey.

Results: Over 102 people reviewed the framework as part of the interview or survey process. This led to a final core competency framework containing six primary domains with 36 subdomains containing 111 individual competencies.

Conclusions: An iterative mixed-methods approach for competency development involving the target community was appropriate for development of the competency framework. There is some contention around the depth of technical competencies required. Care is also needed to avoid professional burnout, as clinicians and healthcare practitioners already have clinical competencies to maintain. Therefore, how the framework is applied in practice and how practitioners meet the competencies requires careful consideration.

Introduction

The healthcare sector in many countries is facing increasing demand as people live longer and healthier lives.[1] The public’s expectation of healthcare is also increasing and is tempered by various financial constraints. The healthcare sector has lagged behind other sectors regarding its adoption and use of digital technology. In the U.K., the Topol review was carried out to assess how the healthcare workforce can be prepared for the digital future. The review makes many recommendations on the use of genomics technology, robotics, artificial intelligence (AI), and digital medicine, including the training and education of healthcare professionals in such areas.[1] At the cutting edge of this digital upskilling of the workforce are informaticians from clinical, health, and social care disciplines.

The American Medical Informatics Association (AMIA) defines clinical informatics as "the application of informatics and information technology to deliver healthcare services."[2] The U.K. Faculty of Clinical Informatics (FCI) defines a clinical informatician as someone who "uses their clinical knowledge and experience of informatics concepts, methods, and tools to promote patient and population care that is person-centred, ethical, safe, effective, efficient, timely, and equitable."[3]

As of yet there are no U.K.-based overarching competency frameworks aimed at multiple informatics disciplines. Existing frameworks tend to focus on specific domains such as nursing informatics or bioinformatics.[4] The U.K. FCI was created to provide support for clinical informaticians, including those with clinical roles in the health and social care domains applying informatics in practice. It is the intention of the FCI to provide and accredit competencies for informaticians. This includes accreditation of the U.K.’s National Health Service (NHS) Digital Academy program which aims to create digital leaders for the digital transformation of the NHS. The present study forms part of a program commissioned by the FCI to create a national competency framework for clinical informaticians in the U.K.

Competency describes the behaviors, characteristics, skills, attitudes, and knowledge application used to successfully achieve something. Competence therefore is the achievement of a single competency or multiple competencies. A core competency framework describes the essential set of competencies required to achieve competence in a specific area. Many competency frameworks aimed at various clinical informatics disciplines currently exist, such as the ELIXIR (European Life-science Infrastructure for Biological Information) and TIGER (Technology Informatics Guiding Education Reform) frameworks for bioinformaticians and nurses. This paper reports on the methods used to generate and refine the U.K. FCI’s Core Competency Framework (CCF), which covers the core competencies required to develop clinical informaticians’ professional competencies, and to provide a process for the FCI to provide accreditation for training and education programs and individual clinical informaticians.[5]

Background

The Core Competencies Project spanned two primary phases. The first part of the work explored the definition of the professional attributes of a clinical informatician. The FCI carried out the first phase of the work (Figure 1), which consisted of three reports based on (1) discovery[6], (2) validation[7], and (3) consultation around the output competences of a clinical informatician.[3] This included defining clinical informatics, clinical informaticians, inclusivity, professional boundaries, and the functional domains (scope) of clinical informatics practice. This work was carried out using qualitative methods (e.g., interviews) with the faculty’s membership. The second phase (reported in this paper) involved an iterative process consisting of three separate iterations. The first iteration involved the combination of findings from both academia and industry in the form of an analysis of over 50 informatics job postings and a systematic literature review which explored the commonality of informatics competencies across different clinical informatics domains (such as medicine, nursing, pharmacy).[4] Following synthesis of this information, an initial draft competency framework was generated, which was then presented to 15 informatics experts in one-to-one semi-structured interviews and adapted following feedback. The amended version was presented for wider evaluation through a digital survey with 87 participants. In light of the survey results, the final version of the competency framework[5] was updated and disseminated publicly on the FCI’s website.

Fig1 Davies BMJHealthCareInfo2021 28-1.jpg

Figure 1. Overview of the core competencies project. The focus of this paper is the reporting of the final two iterations and dissemination from phase 2, as seen in the darker highlighted section.

Figure 1 provides an overview of the iterative steps followed to create the final core competency framework. This paper reports on the interviews and survey (iterations 2 and 3 of phase two). The principle research questions were:

  • RQ1: Was the chosen mixed-methods approach appropriate for generating a core competency framework?
  • RQ2: How do participants feel the framework should be applied and to whom?
  • RQ3: What competencies were considered "core" and how do they fit in with an evolving profession?

Iteration 1: Interviews

Methods

Semi-structured interviews were carried out with participants (n = 15) involved in various aspects of informatics. Participants were sent a copy of the interview schedule (see Box 2 in the supplied supplementary material) and the framework prior to the agreed interview date. Due to the COVID-19 lockdown, the interviews were carried out online using Zoom and audio recorded. Transcribed interviews were analyzed using framework analysis.[8] Framework analysis involves five stages consisting of familiarization with the data, identification of the thematic framework, indexing, charting, and mapping/interpretation. We followed the Standards for Reporting Qualitative Research (SRQR) guidelines.[9] The interview quotes were then stratified into themes by one researcher, and the quotes were independently matched to the primary themes by another. The percentage agreement between both researchers was 85.6%, with an unweighted Cohen’s kappa showing substantial agreement (κ = 0.67).

The sample

A purposive sample was obtained through the FCI, who sought to contact a wide variety of its members and associates with different professional backgrounds. Participants were from a broad set of backgrounds (Table 1). Participants were involved in informatics at various levels, from being the sole informatician in an organization through to Chief Clinical Information Officers (CCIOs) and digital leads. We omit the current roles of participants to maintain anonymity. Sectors of employment included the NHS, adult social care, and the prison system. Age and other demographic details were not requested. The clinical backgrounds were also varied and included general practice, urological surgery, pathology, and oncology. Nurses represented multiple clinical areas across their careers in both the NHS and private sector (including nursing homes).

Table 1. Demographics and background summary showing the number and percentage of interview participants
Background n (%)
Biomedical scientist 1 (6.6)
Physiotherapist 1 (6.6)
Paramedic 1 (6.6)
Software engineer 1 (6.6)
Pharmacist 2 (13.3)
Senior lecturer (bioinformatics, social care) 2 (13.3)
Nursing 3 (20)
Medical (three general practitioners, one consultant) 4 (26.6)

Researcher characteristics and reflexivity

Following the SRQR guidelines[9], details of the researchers' characteristics and personal attributes that may influence the research are detailed. Alan Davies (AD) has a background in computer science (PhD) and nursing science (BSc). AD works as a senior lecturer in health informatics and health data sciences and was previously a software engineer/data scientist in industry and a former cardiac nurse. AD did not personally know any of the interviewees that took part in the study, with the exception of one lecturer whom he knows in a professional capacity. AD’s research paradigm is situated in post-positivism. Julia Mueller (JM) has a background in health psychology (MSc, PhD). JM formerly worked as a lecturer in healthcare sciences teaching a course unit on digital public health, and currently works as a research associate in behavioral weight management. JM did not personally know any of the interviewees. JM’s research paradigm is situated in constructivism.

Results

A total of eight hours and 25 minutes (M = 33.66, SD = 12.28) of interview data were generated. Given the nature and topic of the interviews the results are reported in two main ways. The first set of results pertains to the required changes to the composition of the framework and to specific areas such as a specified competency. This does not require any interpretive themes and is reported descriptively. The second set of results uses framework analysis to identify themes around the potential application of the framework, who it is for, and how the participants see the informatics profession changing in the future.

Requested changes to the framework

Table 2 provides an overview of the suggested changes to the framework, broken down by topic and subtopic. This shows how many participants mentioned this topic and how many times.

Table 2. Main topics and subtopics of suggested change with number of participants requesting change and number of times mentioned by participants
Topic/subtopic Participants References
Framework composition
Language and terminology 12 55
Missing competency 12 35
Representative of core competency 9 31
Target level/granularity 13 38
Specific changes
Change to competency 9 51
Change to diagram or preamble 10 20
Change to structure, order or grouping 7 18

Framework composition

Language and terminology

It was mentioned (n = 2) that the wording should make specific reference to "social care" rather than just "healthcare":

... you just get people looking at it, and thinking, that doesn’t apply to me. And then they never revisit it. So, it’s that first impression of appearing as welcoming as they actually are, and showing from within the competencies, that they recognize that it’s not about a job title, it’s about the roles you are doing. – P2 Social workers, the social care sector tends to notice these things, if they feel it’s too contextualized within healthcare than social care ... And you have some healthcare professionals who really lean more towards working in the community for instance. So their roles overlap much more with their social care, social work side of the system. And so it would be helpful for that to be reflected as well. – P8

Missing competency

Table 3 summarizes what participants viewed as missing competencies.

Table 3. Summary of missing competencies identified by participants
Participant Missing competency
P1 1. Choosing between types of hardware and technology.
2. How data is transmitted between systems.
3. Accessing ethical and legislative requirements.
P4 1. How to access and use evidence bases (ie, collating evidence for a start-up).
2. Communication as part of change management (e.g., workshops, comms teams, weekly meetings, and teaching sessions).
3. Ethics and clinical safety.
P5 1. More on governance and regulatory frameworks.
2. Validation and verification of software.
3. Audit cycles.
4. Presenting data visually to patients (ie, infographics).
5. Communicating with stakeholders and getting management buy-in.
P6 1. Agile project management.
P7 1. Postgraduate teaching and supervising.
P8 1. Understanding integration between systems (i.e., clinical and social care systems).
2. Ethical decision-making.
P11 1. Understanding more about others roles.
P12 1. Decision-making should have its own domain.
2. Action behavioral change.
P14 1. Leadership resilience.
P15 1. Clinical improvement as its own section (research and evaluation of what we are doing).

Three participants made reference to the fact that AI and machine learning only receive a nominal mention:

... there’s a very nominal mention of, sort of, AI in there. I think, machine learning could be stronger, in the frameworks, I think, it’s something that people might be expected to start to look at in the clinical informatics kind of role. – P5

The target level

As the framework was intended to list the core competencies grouped by domain only, none of the versions of the framework presented were stratified into different levels of competence. Thirteen participants discussed the level or granularity of the competencies. Participants were mostly concerned that although they thought some competencies were "core," they were unsure about them being "entry level" competencies:

I wasn’t sure whether 1.4 [health administration and services] a, b, or c, at an entry level needed to be. I think they do need to be as someone progresses as a clinical informatician but they struck me as the sort of things that you wouldn’t say examine a nurse who was newly qualified or even a newly qualified doctor or a biomedical scientist, for that matter. – P1

Three participants specifically identified the "leadership and management" competency as one that was particularly affected by level. One participant pointed out that clinical informaticians may need to cultivate leadership and management skills at an earlier career point, due to the nature and novelty of the role:

... some of the informaticians that we’ve worked with have actually said ... that those sort of leadership skills actually are, you know, needed more rapidly than anticipated. Because, they’re thrown into quite, sometimes quite high-level meetings ... and from that sense they can also have quite a bit of influence, maybe, with some quite big decisions sometimes. – P5

Representation of a "core" competency Whether or not participants thought a competency was core or not seemed to depend on their role and experience showing the variety of work and experiences of informaticians:

If you look at all of them they’re not for every clinician; some clinicians have more inclination towards using some of them than others. So the current seven [domains], if you look at it in detail, they go way beyond what would be called for all of us. – P7

Seven participants agreed that all or most of the competencies presented were "core" and that it was not necessary to be an expert in them all, but rather to have at least an understanding or awareness of them:

It could be unfair for us to expect people to be experts across all of those domains ... so I think the core competencies have to have an understanding that, yes, you should have a basic understanding across all of these domains but we don’t expect you to be an expert in all of them. – P12

Application of the framework and impact on the profession

For this section, we applied framework analysis to examine themes surrounding the intended application of the framework, impact on the profession, and barriers. Several participants indicated an interest in how the framework might be applied and how people might meet the competencies:

P12: So there needs to be some line that we need to meet and the core competencies should be that, but we just need to be sensitive about how we’re ensuring our members are meeting the competencies.

Interviewer: It sounds like you’re saying a light touch is preferable.

P12: Yeah. We don’t want to beat them with an exam stick, you know. These people are already professionals, so we need to, I don’t know, acknowledge the fact that they are already professionals within their own right and experts in their own right and this is an additional bit and not scare them off by putting them through hours and hours of examinations or whatever.

Another mentioned how this could be used to provide flexible portfolio training routes for clinicians and other healthcare practitioners:

Now what’s interesting is that within that portfolio route there’s no underpinning framework, or nothing anywhere near as detailed as this. And, I think, this would be a really useful competency framework that might underpin flexible portfolio training routes for physicians. Nurses are also starting to come through and people that want to take time out of their regular day job to also diversify into, sort of, informatics type roles – P5

Participants pointed out that some practitioners work more or less in a single competency domain or domains:

I mean some people, their job description will purely be one domain because that’s where they will fit. I was trying to think of a CCIO. There’s probably a CIO in an organization, would this description apply to someone’s job, and I think to be fair, it’s a fairly good description of what you would expect someone to have to have to do that role. – P3

Some barriers were also identified; one from the perspective of those involved in social care work, which seemed to work in two directions:

P2: I think the barriers, certainly from a social care perspective, are the fact that you have to be a registered clinician, and there is no registration process for most people working within social care ... the vast majority of people that work in social care are professionals without a professional registration.

Interviewer: It also sounded to me like there was actually a barrier coming from the other direction as well, with people from social care actually doing these roles, but not identifying as such?

P2: Hundred percent, yeah, 100 percent. There are people in social care, who, if they were in an NHS trust, would have CCIO, CNIO responsibilities, but they do not have them in social care, because the sector does not recognize that as a role yet. So, it’s a two-way barrier, it’s not a one-way barrier at all.

It was generally felt that the framework should be open to as many professional groups as possible, the caveat being that they should also have a clinical role. It was felt that there was a greater distinction between bioinformatics and the other types of health informatics. Some participants suggested that in the future, pathways to becoming clinical informaticians may change. Currently, those with medical/clinical training acquire informatics skills subsequently, but future pathways may involve information technology (IT) and chief information officers acquiring clinical knowledge:

... people who are getting up to be heads of IT, chief information officers as opposed to chief clinical information officers, could probably start to pick up the levels of clinical knowledge to be a clinical informatician so long as it’s not about the clinical interpretation ... – P1

Iteration 2: Survey

Following incorporation of interviewees’ feedback into the framework, the amended version was presented for wider evaluation through a digital survey.

Methods

A bespoke digital survey data collection tool (Figure 2) was created using Python Flask to allow participants to view the survey side-by-side with the competency framework for easy comparison. The online survey was piloted with the research team and internally reviewed. Most of the questions were based on those asked in the interviews, with the addition of rating scales (Likert items) for generating summary data. Participants were identified by the FCI and asked to forward the survey to other relevant professionals.


Fig2 Davies BMJHealthCareInfo2021 28-1.jpg

Figure 2. Screenshots of the survey data collection tool. The framework can be viewed and scrolled on the right of the screen during the primary survey so that participants can reference the framework easily as they complete the survey.

The survey was hosted by the University’s Research IT team. Survey data were analyzed using descriptive statistics (means, percentages), and by using sentiment analysis using the Bing lexicon.[8] Sentiment analysis is a computational method for assessing opinions in textual data and whether or not the writer expresses a positive, negative, or neutral sentiment.

The sample

A total of 87 respondents completed the survey, having between 0 and 49 years of experience in informatics (M = 12.8, SD = 10). The median age of respondents was 50 years (range: 29–73 years). Table 4 shows the breakdown of respondents by informatics discipline.

Table 4. Demographics and background summary showing the number and percentage of survey respondents
Informatics discipline n (%)
Clinical informatics 60 (62.5)
Public health informatics 12 (12.5)
Other informatics 9 (9.4)
Pharmacy informatics 8 (8.3)
Nursing informatics 7 (7.3)
Gender n (%)
Male 57 (65.5)
Female 29 (33.3)
Prefer not to say 1 (1.1)

The "other informatics" backgrounds identified (as mentioned in Table 4) included:

  • research informatics
  • imaging informatics
  • clinical coding (e.g., Healthcare Evaluation Data (HED), SNOMED CT)
  • engineering
  • knowledge management
  • allied health professional (AHP) in NHS
  • urgent care informatics
  • dental Informatics
  • little experience in informatics

Results

We present the results of the survey, which includes the Likert item questions followed by the open-ended questions (see Box 3 in the supplied supplementary material). Figure 3 shows the responses for self-rated informatics expertise. This was rated on a scale from 1 (novice) to 5 (expert). The percentage of responses can be seen on the y-axis, with the actual number of respondents in the bars.


Fig3 Davies BMJHealthCareInfo2021 28-1.jpg

Figure 3. Responses to the the question "How would you rate your informatics expertise? (1 = novice, 5 = expert)." Note that the number of respondents is displayed on the bars.

A summary of the questions asked for the Likert items can be seen in Table 5.

Table 5. Likert item questions, with score ranges
Question Score (1–5)
1. How well do you think these competencies represent those required by professionals to work in the field of clinical informatics in the U.K.? 1 = not well to 5 = very well
2. These competencies represent core competencies. 1 = strongly disagree to 5 = strongly agree
3. The framework strikes the right balance between clinical and technical areas of clinical informatics. 1 = strongly disagree to 5 = strongly agree
4. How easy to understand do you think the competencies in the framework are? 1 = very difficult to 5 = very easy
5. How well do you think the competencies in the framework are presented? 1 = very badly to 5 = very well
6. How many of these competencies do you currently cover? 1 = none of them to 5 = all of them

Figure 4 shows the results for each of these questions per informatics background.


Fig4 Davies BMJHealthCareInfo2021 28-1.jpg

Figure 4. Likert question responses per question by informatics background.

The remaining questions were open-ended questions (see Table 6 for an overview).

Table 6. Free text open-ended questions
Question
7. If there are any competencies that you think are missing from the framework, please briefly describe them.
8. If there are any competencies in the framework you think are not "core" competencies, please indicate which ones by their number(s), followed by a brief reason.
9. Would you suggest any changes to the terminology used in the framework? If so, please briefly describe.
10. Do you have any comments on the overall structure or grouping of the framework competencies?
11. Do you have any comments about any individual competencies in the framework? If so please state the number of the competency and then make your comment.
12. Is there anything else that you would like to mention?

Competencies missing from the framework

Thirteen respondent’s (32.5%) stated that they could not identify any missing competencies. Of those identified as missing, competencies suggested included aspects of diplomacy and negotiation, clinical safety of systems, medical device regulations, and knowledge of research governance. "Cybersecurity" was mentioned by several respondents (n = 3).

Odd that there is no mention of cybersecurity at all, and little on confidentiality and information governance. Burying them under Data security and governance (2.6) is insufficient.

Some element of historical lessons learned in informatics was also suggested (n = 2).

I would add a historical lessons section on previous successful and not successful implementations of health informatics (e.g., NPfIT, Choose & Book, eRS, PACS/DICOM, Wachter review, Topol review) which require to learn [from] past success and failures.

One respondent mentioned that an awareness of programming/software development is not sufficient and that practitioners should actually cultivate skills to create custom-built solutions.

If I think about some of the health informaticians/clinical bioinformaticians I have had contact with over the last few years, many of them discuss actually doing software development roles, for example, making apps/interactive interfaces and also requiring programming skills to do so. I think this is not strong enough in the framework. I don't think it’s sufficient to have an awareness of these things/or just be able to evaluate them. I think they actually need the knowledge and skills to build solutions for themselves (particularly if you consider a financially strained NHS where potentially external solutions will not be affordable). Some bioinformaticians have also had to undertake roles such as server migration (in genomics) and implementing cloud solutions; again I don't think these are alluded to anywhere. [Cloud] solutions should definitely be mentioned.

Are the competencies "core" competencies?

Although some participants recommended adding software development skills when asked what competencies might be missing from the framework, others (n = 2) suggested this may not be a core competency:

I think there are many current active clinical informaticians who struggle to fulfill all of these competencies currently. One good example would be the fields of advanced coding/visualization/modelling and AI. I think as an initial set of "core" competencies, only basic information on these may be appropriate. More advance training may allow some parties to specialize in that field. (Analogous with, for example, a basic surgical training, then specialization say as an orthopedic surgeon, as opposed to an ophthalmologist).

Another participant pointed out that the requirement should be to have an awareness of—rather than being able to practice—these skills:

While the range is pretty comprehensive, it aligns with competencies in other countries, and the requirement is often at the "awareness" rather than the practitioner level, so I think these are OK.

Terminology used in the framework

Of the 37 participants who answered this question, 18 (49%) accepted the terminology presented and did not request any changes. Of those who did request changes, the majority were minor. The changes suggested were quite varied, with little to no agreement, and focused on different aspects:

I would like to see the concept of clinical assurance and clinical risk management in Domain 4. Also more emphasis on governance—risk ownership, project assurance, project delivery, product ownership and maintenance—in Domain 6. I think all terms are accessible to those working in the field, but might be difficult for those who are thinking of changing into a career in informatics, particularly those with limited experience.

Comments on the overall structure or grouping of the framework competencies

Of the 46 respondents answering this question, 28 (61%) had no comment on the structure or thought the structure was good and did not require any changes. The majority of responses indicated minor changes to the organization of the competencies, with no significant homogeneity of opinion.

I really liked it. I think AHPs would find it helpful in understanding their competencies in this field of practice and would support a career pathway for them into the professional cohort. The language is familiar and readable.

Another respondent considered the utility of identifying how one would progress from the essential core competencies through to expert level:

I think it would be useful to identify a progression from "core" to "expert." This would support the identification of a learning program to support a career path in clinical informatics and support the progression from associate to fellow within the faculty.

Comments about individual competencies in the framework

The specific changes to competencies suggested by respondents can be seen in Table 7.

Table 7. Specific changes proposed to competencies (NB: competencies from second iteration of the framework)
Competency Suggested change
2.1 (a) Demonstrates knowledge of key information technology components including hardware and software, and how they can be used in health and social care settings. 2.1 (a) Should be at the level of "Analyzing" rather than "Knowledge" for core competencies.
5.2 (a) Demonstrates understanding of models for effective knowledge acquisition and storage, including strengths and limitations. 5.2 (a) Needs to include something on dissemination.
5.2 (b) Understands the transformation of knowledge (from generation to modelling) into clinical decision tools. 5.2 (b) Do you want to talk about computable knowledge?
4.1 (a) Applies quality improvement and process engineering to facilitate business and clinical transformation, measuring and analyzing appropriate outcomes. 4.1.(a) Please emphasize that clinical safety covers the deployment and use of health information technology systems, not just their manufacture.
4.3 (a) Understands the organizational and human factor challenges to effective use of health information systems and technologies and can apply appropriate methods to address these and ensure maximum user engagement and widespread adoption. 4.3. (a) One bullet point for behavioral change is not really enough!
3.1 Methodologies and applications. Domain 1 and competencies therein, need to include a global perspective as a second bullet point. 3.1 needs to mention databases, data access/storage options. The database (or other) also fundamentally affects the ability to create the analyses. Only mentioning R/Python is akin to saying you need a steering wheel to drive without mentioning the car’s engine.
2.2 Working with project stakeholders. 2.2 I think the inclusion of human factors and user-centered design at this stage might be useful (it is referred to in the change section 4).
2.1 (d) Can identify appropriate technology to resolve healthcare problems and contribute to the development of good practice to do this. 2.1 (d) "Can identify appropriate technology to resolve healthcare problems and contribute to the development of good practice to do this": what do we mean by this and particularly resolving healthcare problems?
2.4 (a) Demonstrates knowledge of the range of technology for transmitting information (e.g., messaging between systems) and clinical standards (e.g., standards for structuring clinical information) for information needed to support the creation of interoperable systems, and promotes their importance to drive an integrated delivery care model. 2.4 (a) "Clinical standards" should read "clinical information standards" as without context, "clinical standards" could mean something different.
3.1 (a) Is able to explain the methodological concepts of basic descriptive statistics, probability, predictive modelling and artificial intelligence (AI) and know when and how to use them to solve health and social care, and clinical practice and research problems.
3.1 (b) Is able to demonstrate how data quality effects analysis, and resulting clinical and healthcare insights, and how important it is to improve to derive maximum potential from its utilization.
3.1 (c) Understands how technologies (e.g., R, Python, Github) facilitate the analysis, display of results and reproducibility of analyses.
3.1 (a–c) It would be very thoughtful about exactly how much you would expect basic, core clinical informaticians to be able to explain about methodological concepts of modelling and AI. Or indeed expect a robust understanding of R, Python, Github.

Another respondent suggested defining broad skills in an "assumptions" section as one could consider skills like project management as not being specific to clinical informatics:

Suggest defining some of the broad skills in an assumptions section that is "assumed that each individual has the following competency areas not specific to CI" for example, project management skills, quality improvement skills, and governance skills, and only including those skills specific to CI.

The final CCF

Overall, participants were generally positive about the framework. This is quantified by sentiment analysis of the survey responses using the Bing lexicon[10] which showed a net sentiment score of 60.33% (negative=70, positive=283). The final domains and subdomains included in the framework can be seen in Figure 5. The final version of the framework can be viewed on the FCI’s website.


Fig5 Davies BMJHealthCareInfo2021 28-1.jpg

Figure 5. Domains and subdomains in the final competency framework.

Competencies are defined hierarchically by complexity using Bloom’s original Taxonomy[11], progressing from level 1 to 6 consisting of remembering (knowledge), understanding (comprehension), applying, analyzing, synthesizing and creating, and finally evaluation. Such competencies can be graded against the taxonomy.[12] The CCF has four principle areas: health, data and technologies, leadership and management, and people. There are six domains, comprising of health and well-being in practice, IT and systems, working with data and analytical models, enabling human and organizational change, decision making, and leading informatics teams and project. Each domain contains a number of sub-domains (n = 36) which contain the individual competencies (n = 111).

The rationale for acting on a suggestion to make a change to the framework was based on several factors, including the number of people expressing a similar desire for change, whether the detail provided was sufficient to understand the suggested change, and the relevance to clinical informatics in general. Following collation of data (systematic review, interviews, and survey) the authors determined the final competencies at each stage through internal consensus based on the synthesized data.

Discussion

The use of an iterative approach for competency definition is advocated by Greenhalgh and Macfarlane.[13] Our approach was similar to that of Jidkov et al., which used an iterative method to develop 20 universal health informatics competencies for postgraduate medical education. The approach featured a literature review, content analysis, and expert review.[14] As opposed to a purely topic-based approach, competency approaches have been gaining more attention in both nursing and medical education.[12] This is not without some issues, however, such as problems related to identifying priorities, as well as defining and measuring how competencies are met.[15] As such, one should aim to strike a balance between covering the most salient areas and keeping the number of competencies to the smallest number required. Another threat to competency models is that of clinician burnout in postgraduate education.[16] As pointed out in the responses, there is a potential additional burden on clinical informaticians as they already have professional membership as clinicians and allied health professionals, with codes of conduct and clinical competencies to maintain. Consideration is required in terms of how clinical informaticians will be required to meet the competencies defined in the framework. Several participants indicated that a light touch is preferable to more intense examinations. It should also be recognized that participants may work deeply in one (or several) domains and not have experience spanning all the domains depending on their role.

The mixed-methods approach chosen appears to be appropriate for generating a core competency framework, as it allowed us to take a systematic and structured approach to co-designing the framework with no major issues. The combination of qualitative and quantitative methods contributed to the robustness of the final output. Specific frameworks for certain informatics subdomains already exist, such as TIGER for nurses[12] and ELIXIR[17] for bioinformaticians. Additionally, there are international frameworks and recommendations for informatics education such as HITComp[18] and the International Medical Informatics Association (IMIA) updated recommendations.[19] Although there is some overlap between the CCF and competencies identified in such international frameworks, the CCF adopts a U.K.-specific perspective, taking into consideration the unique characteristics of U.K. health and social care. Considering this local perspective is crucial to the development of competency frameworks.[4][6][12] The Topol review specifically identifies the FCI as ideally positioned to recruit, retain, and credential NHS data science specialists.[1]

Participants felt that the framework should be open to as many professional groups as possible. It was also highlighted that practitioners may work in depth in a few domains and may not span all the domains. However, it was considered that at entry level, an awareness of the other domains would be sufficient and that practice-level skills may not be required in all the domains. The participants generally felt that all the competencies were "core" competencies, depending on the level that a practitioner was expected to engage with them. There was some contention in terms of the more technical aspects (specifically around domain 3, "working with data and analytical methods") where some felt that informaticians should have more extensive skills (i.e., being able to build software), whereas others put forward that clinicians may struggle to meet this requirement. As it stands, the framework only stipulates an awareness of such issues rather than practical experience.

The evolving technical requirements of informatics and the overlap with the profession of data science is a key consideration. Douglas Fridsma, the former president and COE of AMIA, discussed how such similarities may be operationally defined via the development of core competencies.[20] There may be some overlap in areas of analytical and computational skills in healthcare data science, as seen in a recent content analysis of healthcare data scientists job postings[21] and informatics. Although data science may involve a deeper dive into analytical methods and processes, informatics spans a wider remit and can be involved with decision-making, communication, and implementation of data analysis and health systems. The boundaries between such professions may be blurred at times, but as Fridsma points out, core competencies can help to distinguish professions as well as define how they overlap.

Another point raised was the importance of topics such as AI and ML. Both AI and ML are mentioned prominently in the Topol review[1] and are receiving serious consideration and funding for their application to healthcare. AI is not a new concept and has been around since the 1950s.[22] Many algorithms are referred to as a black box because the exact workings of certain algorithms (e.g., neural networks) can be complex and unpredictable.[22] Despite their current popularity, it is unclear whether AI and ML will continue to hold the same prominent position in the future. The framework therefore incorporates AI and ML into the broader category of "Analytical Methodologies and Applications" rather than a section in its own right. Many statistical methods are still widely used in medical contexts, and "statistical learning," which is focused on prediction as well as supervised and unsupervised modelling, spans the domains of machine learning and statistics.[23]

One of the challenges going forward will be the implementation of the framework. This will include tasks such as developing operational definitions for the competencies concerning their related knowledge, skills, attitudes, and behaviors, as well as defining how such competence will be measured for individuals.[15] The CCF provides the initial foundation for defining the requirements to be a clinical informatician in the U.K. and provides a starting point for professional development in this fast-moving area of healthcare.

As the interviews highlight, careful consideration is required when linking the framework to developmental stages and/or academic levels. Various stage models of professional development exist[24][25][26] and mostly originate from the field of cognitive psychology. These focus on moving through various developmental stages (e.g., novice, advanced beginner, competent, proficient and expert[27]). Stage models are often based on a variety of attributes, including attitudes, skills, and underpinning knowledge, which are supplemented with additional skills and knowledge from the workplace.[28]

Criticism of such models suggest that some aspects of professional skill development, such as the skill being developed in terms of understanding and practice, may be concealed, as the primary focus is on the developmental stages themselves.[28]

More recent work considers models that account for multiple dimensions such as progression of skills over time and embodied understanding of practice.[28] This reflects the difference between skill progression over time that is possible without such understanding of practice, which differentiates experienced and expert practitioners. Other models consider extra dimensions, such as functional and foundational competency domains and stages of professional development.[29]

It can also be difficult to ascertain exactly which skills, knowledge, and attitudes differentiate between being an expert or not. Given that there is a connection between the individual and their characteristics and their professional practice[28], applying a staged model can be complex, especially when considering applying this to the various clinical groups that form the clinical informatics community with their underlying cultures and contexts. This may have to be applied in slightly different ways to account for these variations in context.

The FCI have developed a working group that seeks to address such issues, including the mapping of the framework to academic courses, accreditation using the framework, and other details around its validation and implementation. Progress of the working group is reported publicly online via the FCI website, giving the option for more end users and stakeholders to comment and feedback on the implementation as well as future revisions.

Limitations

Interviews were conducted at the start of the COVID-19 pandemic, which may have impacted recruitment levels. It is possible that not all perspectives of all professional groups relevant to clinical informatics were represented. We were however able to engage with a range of participants that represented different roles, professional backgrounds, and experience levels. A further limitation is that the work is based in the U.K. and therefore may not be fully generalizable outside of the U.K. without adaptions for other contexts. Finally, the survey only received 87 responses, which given the size of the informatics community in the U.K., would not indicate a large take-up. This was partly due to the project’s limited time frame as well as the start of the COVID-19 pandemic, which meant many potential respondents prioritized pandemic-related work.

Conclusion

We used a mixed-methods iterative approach, using both qualitative and quantitative methods to develop a core competency framework for clinical informatics, consisting of 111 competencies spread over six principle domains with 36 subdomains. The domain around working with data and analytical methods seems to be the most contentious, due to variations in the extent to which practitioners are involved with the practical aspects of producing systems and coding. The overlap between fields like healthcare data science and informatics blurs at this point. Competency frameworks can be used to define these overlaps , as well as the differences. Finally, consideration and care is needed regarding the application of such frameworks for professionals that already have a primary clinical competency set to meet to avoid burnout and to ensure all groups can meet these competency requirements, even if their main area of work spans only one or a few of the primary domains. We recommend an annual review process for the framework moving forward to ensure its continuing relevance.

Supplemental material

Supplementary material 1 (PDF)

Acknowledgements

The authors would like to thank the Faculty of Clinical Informatics for funding this work and Dr. Daniel Corbett from the universities' Research IT department for hosting the survey. We would also like to thank all the people who gave their time to take part in the interviews and complete the survey, especially at this difficult and disruptive time.

Contributions

AD wrote the manuscript. AD and JM collected and synthesized the data. GM produced the draft competencies. All authors reviewed the competencies and reviewed and made edits to the manuscript.

Funding

This work was funded by the Faculty of Clinical Informatics.

Data availability

No data are available. To keep participants anonymous, original interview transcripts/recordings are not publicly available.

Ethics approval

The University of Manchester Research Ethics Committee reference 8985 and 9591.

Competing interests

None declared.

References

  1. 1.0 1.1 1.2 1.3 Health Education England (2019). "The Topol Review – Preparing the healthcare workforce to deliver the digital future". NHS. https://topol.hee.nhs.uk/. 
  2. American Medical Informatics Association. "Informatics: Research and Practice". American Medical Informatics Association. https://amia.org/about-amia/why-informatics/informatics-research-and-practice#:~:text=Clinical%20Informatics-,Clinical%20Informatics%20is%20the%20application%20of%20informatics%20and%20information%20technology,clinical%20informatics%20and%20operational%20informatics.&text=Clinical%20Informatics. Retrieved 02 September 2020. 
  3. 3.0 3.1 Hassey, A.; Jidkov, L.; Williams, J. (2020). "Report C: Consultation stage - Consultation Exercise and Output Competences for a Clinical Informatician". Core Competencies Project - Phase 1 Final Summary Report. p. 22. https://facultyofclinicalinformatics.org.uk/core-competencies-phase1. 
  4. 4.0 4.1 4.2 Davies, Alan; Mueller, Julia; Moulton, Georgina (1 September 2020). "Core competencies for clinical informaticians: A systematic review". International Journal of Medical Informatics 141: 104237. doi:10.1016/j.ijmedinf.2020.104237. ISSN 1872-8243. PMID 32771960. https://pubmed.ncbi.nlm.nih.gov/32771960. 
  5. 5.0 5.1 Moulton, Georgina; Davies, Alan; Mueller, Julia (22 July 2020) (in en). Protocol: Development of core competencies for Clinical Informatics in the UK. doi:10.5281/ZENODO.3956373. https://zenodo.org/record/3956373. 
  6. 6.0 6.1 Quinn, N.; Hassey, A.; Jidkov, L. (2020). "Report A: Discovery stage: Develop and Define the Professional Attributes of a Clinical Informatician – Final Report". Core Competencies Project - Phase 1 Final Summary Report. https://facultyofclinicalinformatics.org.uk/core-competencies-phase1. 
  7. Hassey, A.; Jidkov, L.; Williams, J. (2020). "Report B: Validation stage - Validation Study and draft Output Competences for a Clinical Informatician". Core Competencies Project - Phase 1 Final Summary Report. https://facultyofclinicalinformatics.org.uk/core-competencies-phase1. 
  8. 8.0 8.1 Ritchie, J.; Spencer, L. (1994). "Qualitative data analysis for applied policy research". In Bryman, A.; Burgess, B. (in English). Analyzing Qualitative Data. Routledge. pp. 173–94. ISBN 978-0-203-41308-1. OCLC 1020495928. https://www.worldcat.org/title/mediawiki/oclc/1020495928. 
  9. 9.0 9.1 O'Brien, Bridget C.; Harris, Ilene B.; Beckman, Thomas J.; Reed, Darcy A.; Cook, David A. (1 September 2014). "Standards for reporting qualitative research: a synthesis of recommendations". Academic Medicine: Journal of the Association of American Medical Colleges 89 (9): 1245–1251. doi:10.1097/ACM.0000000000000388. ISSN 1938-808X. PMID 24979285. https://pubmed.ncbi.nlm.nih.gov/24979285. 
  10. Liu, Bing (2015). Sentiment analysis: mining opinions, sentiments, and emotions. New York, NY: Cambridge University Press. ISBN 978-1-107-01789-4. 
  11. Bloom, Benjamin S (1956) (in English). Taxonomy of educational objectives. Handbook 1, Handbook 1,. London: Longmans. ISBN 978-0-582-32386-5. OCLC 604022335. https://www.worldcat.org/title/mediawiki/oclc/604022335. 
  12. 12.0 12.1 12.2 12.3 Hübner, Ursula; Shaw, Toria; Thye, Johannes; Egbert, Nicole; Marin, Heimar; Chang, Polun; O’Connor, Siobhán; Day, Karen et al. (1 June 2018). "Technology Informatics Guiding Education Reform – TIGER: An International Recommendation Framework of Core Competencies in Health Informatics for Nurses" (in en). Methods of Information in Medicine 57 (S 01): e30–e42. doi:10.3414/ME17-01-0155. ISSN 0026-1270. PMC PMC6193400. PMID 29956297. http://www.thieme-connect.de/DOI/DOI?10.3414/ME17-01-0155. 
  13. Greenhalgh, T.; Macfarlane, F. (1 April 1997). "Towards a competency grid for evidence-based practice". Journal of Evaluation in Clinical Practice 3 (2): 161–165. doi:10.1046/j.1365-2753.1997.00082.x. ISSN 1356-1294. PMID 9276593. https://pubmed.ncbi.nlm.nih.gov/9276593. 
  14. Jidkov, Lydia; Alexander, Matthew; Bark, Pippa; Williams, John G; Kay, Jonathan; Taylor, Paul; Hemingway, Harry; Banerjee, Amitava (1 March 2019). "Health informatics competencies in postgraduate medical education and training in the UK: a mixed methods study" (in en). BMJ Open 9 (3): e025460. doi:10.1136/bmjopen-2018-025460. ISSN 2044-6055. PMC PMC6475211. PMID 30928942. https://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2018-025460. 
  15. 15.0 15.1 Reid, W. Michael; Dold, Claudia Jennifer (2017 Jan/Feb). "Leadership Training and the Problems of Competency Development". Journal of public health management and practice: JPHMP 23 (1): 73–80. doi:10.1097/PHH.0000000000000456. ISSN 1550-5022. PMID 27598708. https://pubmed.ncbi.nlm.nih.gov/27598708. 
  16. Dauphinee, W. Dale (1 March 2020). "Building a core competency assessment program for all stakeholders: the design and building of sailing ships can inform core competency frameworks". Advances in Health Sciences Education: Theory and Practice 25 (1): 189–193. doi:10.1007/s10459-020-09962-1. ISSN 1573-1677. PMID 32030572. https://pubmed.ncbi.nlm.nih.gov/32030572. 
  17. Larcombe, L.; Hendricusdottir, R.; Attwood, T.K.; Bacall, F.; Beard, N.; Bellis, L.J.; Dunn, W.B.; Hancock, J.M. et al. (21 June 2017). "ELIXIR-UK role in bioinformatics training at the national level and across ELIXIR" (in en). F1000Research 6: 952. doi:10.12688/f1000research.11837.1. ISSN 2046-1402. PMC PMC5521157. PMID 28781748. https://f1000research.com/articles/6-952/v1. 
  18. Omni Micro Systems. "HITComp Competencies". HITComp. Omni Micro Systems. http://hitcomp.org/competencies/. Retrieved 27 March 2020. 
  19. Mantas, John; Ammenwerth, Elske; Demiris, George; Hasman, Arie; Haux, Reinhold; Hersh, William; Hovenga, Evelyn; Lun, K. C. et al. (7 January 2010). "Recommendations of the International Medical Informatics Association (IMIA) on Education in Biomedical and Health Informatics. First Revision". Methods of Information in Medicine 49 (2): 105–120. doi:10.3414/ME5119. ISSN 2511-705X. PMID 20054502. https://pubmed.ncbi.nlm.nih.gov/20054502. 
  20. Fridsma, Douglas B (1 January 2018). "Data Sciences and Informatics: What’s in a name?" (in en). Journal of the American Medical Informatics Association 25 (1): 109–109. doi:10.1093/jamia/ocx142. ISSN 1067-5027. PMC PMC7647139. PMID 30099538. https://academic.oup.com/jamia/article/25/1/109/4837306. 
  21. Meyer, Melanie A. (1 May 2019). "Healthcare data scientist qualifications, skills, and job focus: a content analysis of job postings". Journal of the American Medical Informatics Association: JAMIA 26 (5): 383–391. doi:10.1093/jamia/ocy181. ISSN 1527-974X. PMC 7787353. PMID 30830169. https://pubmed.ncbi.nlm.nih.gov/30830169. 
  22. 22.0 22.1 The Lancet Respiratory Medicine (1 November 2018). "Opening the black box of machine learning" (in en). The Lancet Respiratory Medicine 6 (11): 801. doi:10.1016/S2213-2600(18)30425-9. https://linkinghub.elsevier.com/retrieve/pii/S2213260018304259. 
  23. James, Gareth; Witten, Daniela; Hastie, Trevor et al., eds. (2013). An introduction to statistical learning: with applications in R. Springer texts in statistics. New York: Springer. p. 426. ISBN 978-1-4614-7137-0. OCLC 828488009. https://www.worldcat.org/title/mediawiki/oclc/828488009. 
  24. Ericsson, K. Anders; Smith, Jacqui, eds. (1991). Toward a general theory of expertise: prospects and limits. Cambridge ; New York: Cambridge University Press. ISBN 978-0-521-40470-9. 
  25. Hoffman, Robert R., ed. (1992) (in en). The Psychology of Expertise. New York, NY: Springer New York. doi:10.1007/978-1-4613-9733-5. ISBN 978-1-4613-9735-9. http://link.springer.com/10.1007/978-1-4613-9733-5. 
  26. Sternberg, Robert J.; Ben-Zeev, Talia (2001). Complex cognition: the psychology of human thought. New York: Oxford University Press. ISBN 978-0-19-510771-5. 
  27. Dreyfus, Hubert L.; Dreyfus, Stuart E.; Athanasiou, Tom (1986). Mind over machine: the power of human intuition and expertise in the era of the computer. New York: Free Press. ISBN 978-0-02-908060-3. 
  28. 28.0 28.1 28.2 28.3 Dall’Alba, Gloria; Sandberg, Jörgen (1 September 2006). "Unveiling Professional Development: A Critical Review of Stage Models" (in en). Review of Educational Research 76 (3): 383–412. doi:10.3102/00346543076003383. ISSN 0034-6543. http://journals.sagepub.com/doi/10.3102/00346543076003383. 
  29. Rodolfa, Emil; Bent, Russ; Eisman, Elena; Nelson, Paul; Rehm, Lynn; Ritchie, Pierre (2005). "A Cube Model for Competency Development: Implications for Psychology Educators and Regulators." (in en). Professional Psychology: Research and Practice 36 (4): 347–354. doi:10.1037/0735-7028.36.4.347. ISSN 1939-1323. http://doi.apa.org/getdoi.cfm?doi=10.1037/0735-7028.36.4.347. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. A few grammar and spelling errors were also corrected. In some cases important information was missing from the references, and that information was added. The original citation 20 appears to be a duplicate of citation 6 and was not included in this version. The original citation 23 appears to be a duplicate of citation 1 and was not included in this version.