1. Introduction
Providing healthcare for certain patient groups is complicated due to difficulties with cooperation, communication, health conditions, and social context, amongst others [
1]. Examples of groups affected include the dependent elderly, persons with disabilities, young infants, persons with mental health problems, incarcerated persons, homeless persons, etc. Such patient groups may be considered as having special needs in health care, as services must be adapted if high-quality, appropriate treatment is to be provided. Adaptation of services typically requires additional expertise, equipment, and time, and thus imposes additional costs on healthcare providers. In many health systems, dentistry is financed on a “fee per item” basis, that is, the dentist is paid a set fee by the state for a defined item of treatment. This payment system is considered to be incompatible with access to care for persons requiring additional time or expertise in dentistry [
2]. To compensate for this problem, many public health systems provide salaried services for special care patients but this implies exclusion from the mainstream system, and the creation and financing of suitable and sufficient local salaried services. In addition, when commissioning services, it is difficult to identify and quantify needs when the target population is ill-defined and hard to reach [
3,
4].
In order to encourage the provision of care for populations with special healthcare needs, systems must be able to identify those persons requiring adapted care and must ensure appropriate financial compensation to the service provider for potential costs engendered. Although there is little consensus about the definition of “complexity of care” in the literature [
5], it is clear that patient diagnosis alone does not adequately capture differences in care complexity [
6]. In 2010, the British Dental Association proposed a special care case mix tool to address this problem [
3]. Different healthcare systems and healthcare cultures require different tools, however, and the development of country-specific “case-mix” tools is growing (although these remain unpublished as yet).
In France, the vast majority of dentists (89%) work in general practice within the primary public healthcare system, on a fee-per-item basis [
7]. There is no specific salaried service for special care dentistry. In order to encourage access to adapted dental care within the primary healthcare system, a financial supplement for dentists for each treatment session with a patient with a severe disability has recently been introduced. To justify the financial supplement, dentists are asked to complete the French Case Mix tool (FCM) (in French the FCM is entitled APECS—échelle des Adaptations pour une Prise En Charge Spécifique en odontologie). This tool was originally designed to identify the degree of adaptation necessary for appropriate dental treatment to be provided for all types of special care patients. The FCM had not been validated on its introduction into the dental fee structure.
The aim of the current study was to investigate the validity and psychometric properties of the French Case Mix tool in the identification of patients with special healthcare needs in dentistry. The specific objectives were to describe the development of the FCM; to describe pilot studies undertaken to improve content validity; to undertake a test–retest study to evaluate criterion validity, intra-dentist and inter-dentist reproducibility, and interpretability; and to retrospectively analyze national data from patient records to verify internal consistency, floor, and ceiling effects, and construct validity.
2. Materials and Methods
2.1. Development of the FCM
A group of experts from the French Association for Disability and Oral Health (SOSS Santé Orale et Soins Spécifiques,
www.soss.fr, accessed on 30 January 2023) (termed “the national association” from here on) developed a preliminary version of the French Case Mix tool (FCM). A working group of three special care specialists, one in hospital practice, one in university hospital practice, and one in general practice, drew up an initial draft of the FCM following an informal scoping review of the relevant literature. This draft was debated, reworked, and reworded during a dedicated session of the Executive Board of the national association (made up of 12 persons including dental professionals, medico-social professionals, and patient representatives). This version underwent three waves of piloting by practitioners in two different specialist hospital practices and in one general practice. These pilot projects led to changes in wording and presentation. The draft FCM was presented to the Disability Working Group of the French Social Security Department working on the new financial measures for dental treatment (Caisse Nationale d’Assurance Maladie). Further amendments were made by the experts of this group including the addition of the American Society of Anesthesiologists classification for the evaluation of medical health [
8].
The definitive version of the FCM is given in
Supplementary Materials. It consists of seven domains: Communication; need for sedation, general anesthesia (GA) or other facilitatory techniques; cooperation; medical health; dental risk factors; autonomy; administrative coordination. The FCM is designed to be completed by the dentist at the end of each treatment session. The dentist gives a score of no adaptation, minor adaptation, moderate adaptation, or major adaptation for each of the seven domains. If the treatment session involving a patient with a disability is scored as having required moderate or major adaptation in at least one domain, the dentist is eligible for a financial supplement.
2.2. Test–Retest Evaluation
A group of five experienced special care specialists from the national association with different practice backgrounds designed twelve virtual case studies, detailing treatment sessions with different types of patients and different levels of adaptation to standard dental care. Each case study was accompanied by a photograph of the fictional patient and a detailed description of the patient encounter, including all necessary information to complete the FCM. The experts completed the FCM for each case study individually and then reached a consensus through debate to create a set of “gold-standard” responses. The limits of the “gold- standard” are recognized in that, in practice, a dentist’s experience of the complexity of care is subjective and will vary according to his/her skill and experience in special care dentistry.
All members of the national association were solicited by email to take part in the test–retest study and to encourage their colleagues to participate in a snowballing process. All dentists responding to the call for participation were given a secure link to a website presenting the twelve virtual case studies. The dentists completed the FCM in relation to each case study. After 14 days, they were sent up to three email reminders to re-complete the FCM for the same case studies. The sample size was fixed according to COSMIN guidelines [
9]. Participants were included until at least 50 test–retest data sets had been collected, in accordance with Terwee et al. [
10] who suggest that a positive rating for reliability can be given when the ICC or weighted Kappa is at least 0.70 in a sample size of at least 50 participants.
2.3. National Data
All members of the national association were solicited to share data. FCM forms were retrieved from patient files of participating dentists, along with anonymized sociodemographic data (patient age, gender, patient with disability/dependent elderly/dental anxiety). Data collection took place for patients who had been treated between May 2019 and May 2020. FCM forms were included if they were fully completed. There were no exclusion criteria regarding patient characteristics (age, medical history, or social context for example). For each patient session, the dentist evaluated whether complex care had been required. Dentists described their type of practice (private/other), their target population (special care/general population), their affiliation to a special care coordination network, and their special care specialist status. The sample size was fixed according to COSMIN guidelines [
9], and it was hoped that at least 50 dentists would provide up to 100 FCM forms each.
The protocol was certified by the Institutional Review Board of the University Hospital of Clermont Ferrand (Ref: 2019/CE38).
2.4. Statistical Analysis
The psychometric properties of the FCM were evaluated following COSMIN guidelines [
9], using criteria described by Terwee and al. in 2007 [
10]. Data from the test–retest and the national study were analyzed separately. Statistical analysis was performed using Stata13 (StataCorp, College Station, TX, USA). A summary of the analysis methodology is given in
Table 1.
To compare socio-demographic characteristics, either the Chi2 or Fisher exact test was used.
Content validity was sought by analyzing the feedback given by dentists during the pilot. In particular, the time taken to complete the survey and the acceptability of the domain concepts were reported.
Internal consistency was assessed using Cronbach’s alpha. Item rest correlation, average inter-item correlation, and labeled alpha were calculated for each domain. The inter-item correlation matrix was also analyzed. It was important that all domains measured different concepts, so that how individual domains related to the overall scale was examined. A Cronbach’s alpha score between 0.7 and 0.8 was expected. The ideal range of average inter-item correlation should be 0.15 to 0.50.
Floor and ceiling effects were evaluated based on eligibility for the financial supplement (a score of moderate or major in at least one of the seven domains of the FCM). For each item, the percentage of patients needing minor and major adaptations was calculated.
Construct validity was evaluated by analyzing the agreement between the dentist’s opinion of care complexity and eligibility for the financial supplement using the weighted Kappa coefficient. In order to further investigate this concept, the Item Response Theory (IRT) was applied to estimate the properties of the domains and the utility of each domain. More precisely, the IRT Model was used to produce item information characteristic curves to evaluate whether the four-point Likert scale was relevant to the level of adaptation in each domain.
In terms of criterion validity, the correlation between the dentist’s scores and the “gold standard” replies in the test–retest study was measured, with a positive rating if the correlation was at least 0.70.
The test–retest study allowed for the investigation of reproducibility. The concordance coefficient (weighted Kappa) and concordance percentage were used to evaluate the intra-dentist agreement. A concordance coefficient less than 0.4 was considered low, above 0.7 good, and above 0.8 excellent. The inter-dentist agreement was calculated using the Intra-Class Correlation Coefficient (ICC). Close to 1, the ICC indicates a high degree of similarity between the responses of practitioners, whereas when it is close to 0, it shows responses that are less similar.
Interpretability was investigated by analyzing the dentist’s replies against the “gold standard” in the test–retest study in terms of whether the case study was considered eligible for the financial supplement or not.
To analyze the reliability of the domains to identify patients requiring adaptation, a receiver operating characteristic curve was constructed. The cut-off of at least one domain with a moderate or major score in order to be eligible for the financial supplement was an arbitrary decision. The area under the curve (AUC) was, therefore, calculated for the number of items with a moderate or major adaptation score in order to discuss this decision. AUC is an effective way to summarize the overall diagnostic accuracy of the test. The lower limit on the confidence interval is expected to be <0.7 [
11]. The positive predictive value (PPV) and the negative predictive value (NPV) were examined to investigate whether the FCM agreed with the dentist’s appraisal of care complexity.
It was not appropriate to evaluate responsiveness in this study as the FCM is designed to be completed retrospectively after each treatment session, as a one-off event, not to measure change over time.
4. Discussion
French policy makers introduced a compensatory financial measure for dental practitioners treating patients with “severe disability”. The French Case Mix Tool is written into the public fee scale and appears in all French dental payment software as a measure to justify this financial compensation but it had not yet been evaluated on introduction. This study describes the development and validation of the FCM through three phases: pilot studies, a test–retest study, and national data collection. The FCM tool was found to be valid and to have good psychometric properties. It was able to identify patients requiring adaptation for their dental care with very good reliability. Internal consistency was high and the seven domains demonstrated consistency with each other while remaining independent constructs. The agreement between the dentist’s opinion of the complexity of care and the FCM was very high, and the four-point Likert scale was found to be appropriate. The test–retest data gave evidence of inter-dentist and intra-dentist reproducibility and criterion validity. It was thus confirmed that the tool was fit for the purpose of retrospectively identifying dental treatment episodes where significant adaptation to standard care had been required.
The longest-standing measure of the complexity of patient care is the British Dental Association Case Mix tool (BDA) [
3,
12]. This tool is largely used to commission salaried public special care dental services in the UK [
13]. No other published examples of case mix tools were found in the literature although the authors have knowledge of projects in both Australia and Malaysia, and the International Association of Disability and Oral Health (iADH) is planning to produce an international case mix tool. The nearest type of measure in the literature is represented by sporadic attempts to anticipate difficulty for certain patient groups, such as a tool to predict cooperation for dental patients with autism [
14], those with Alzheimer’s disease [
15]; or children with a disability [
16]. Other tools have been used to estimate specific patient treatment needs and are used for commissioning care, for example, the IOTN for orthodontic services [
17] or the IOSN for sedation services [
4,
18].
As far as the authors are aware, none of the above tools have been used to justify financial compensation at the individual patient level, as is the case in France. The original BDA case mix consisted of six domains with weighted ratings. It was designed to measure the relative complexity of care between different patients or of the same patient at different points in time. The outcome of the FCM, on the other hand, is measured in dichotomous terms: a score of moderate/severe in any one domain is sufficient to trigger justification of financial compensation, related to a specific treatment episode. As the domains have been shown to be independent, this system would seem fairer than using a weighted score in this context. Following the initial publication of the BDA tool, a simplified version was proposed, consisting of five domains [
13,
19]. It is possible that with ongoing evaluation of the FCM, it may be possible to simplify the tool in a similar manner in the future, although the completion time for the current tool was already reported at under one minute. It should also be underscored that the reply to the simple question of “Do you think this patient requires complex care or special care dentistry?” was highly correlated with the result of the FCM tool. It might, therefore, be appropriate to do away with the FCM altogether and consider that the dentist is competent to judge the level of complexity of care without the aid of a scale in the future. The FCM was not originally designed as a tool to ration access to financial compensation, but rather as a tool to quantify and qualify the adaptation to care necessary in special care dentistry. In this context, it was to be used primarily for research, epidemiology, or teaching. This debate is particularly pertinent in France, where medical practitioners are free to bill complex consultations at a higher financial fee without justification, but dental practitioners are asked to complete the FCM to justify the complexity.
Within the French dental healthcare system, the recognition of the additional expertise and time required to treat patients with special needs was welcomed as a major breakthrough. In a system dominated by general practitioners working in individual practices, it was hoped that the provision of a financial supplement, over and above the fee-per-item payment, would give incentive to dentists to treat patients from these populations. The FCM was originally designed to identify all situations where adaptation is necessary, including patients with severe dental anxiety, young children, homeless persons, persons with mental health problems, and the dependent elderly, for example, although the national study reported here did not explicitly describe all these groups. Unfortunately, the financial measure is currently limited solely to patients with “severe disability”, as this population was the explicit target of the new measure. The national association hopes that the financial measure will be extended to all patient groups requiring adapted care in the future in order to cover the full scope of special care dentistry. No figures are yet available to describe the demand for the financial supplement amongst French dentists, but anecdotal evidence is that uptake has been very poor. This is partly due to the lack of visibility of the new measure, which was introduced as a very small feature of the entire overhaul of the dental fee scale. In addition, its introduction coincided with the COVID-19 pandemic and was little discussed, even in the specialist media. For these reasons, further time and analysis are needed to evaluate the real public health impact of providing financial compensation for the additional expertise and time required to provide care for certain patient groups.
This study has certain strengths and limitations. The FCM is the first of such tools to undergo a stringent, structured investigation into its validity and psychometric properties. Unfortunately, this evaluation came after the measure had already been introduced into French law which gives a rather upside-down example of policymaking. Although every effort was made by the national association to ensure wide consultation in the development of the tool, the last word was given to the French Social Security Department which imposed certain amendments. The test–retest evaluation depended on the notion of a “gold standard”, which is arbitrary in the case of an individual practitioner’s perception of the complexity of care, as stated in the Method section. The analysis of the national data is strengthened by the large sample size. However, data were collected from members of the national association who were therefore aware of the complexities of special care dentistry. The results may have been different if data had been retrieved from general practitioners with no special interest in these patient populations. Finally, as mentioned above, the demographic categories available to characterize patients in the national study were unnecessarily restrictive.