1. Introduction

The diagnosis of tuberculosis in children remains a challenge. Symptoms are non-specific and, in the early phases of the disease, up to 50% of the children may be asymptomatic [1,2]. In spite of the advances in tuberculosis diagnosing, no test is highly accurate in children [3]. The presence of acid-fast bacilli on sputum smear microscopy–the standard diagnostic criteria for adults–is rare in childhood. Recently, there has been an increasing interest in defining diagnostic criteria for patients with negative smear and culture [4]. In Brazil, a scoring system was adopted to diagnose tuberculosis in childhood in 2002 [5]. In childhood, bacteriology is positive in only 5% of cases and culture in 25% [6]. Diagnosis in children therefore must be based on clinical and radiological findings, tuberculin test results, epidemiological links, and response to treatment using algorithms or personal experience [6,7].

Diagnostic scoring systems for the diagnosis of tuberculosis were developed, but none are used routinely. All scores have limitations [8–10]: low sensitivity [11,12]; inclusion of smear or culture or histopathology as a criteria [10,11,13], or response to antibiotic therapy or anti-tuberculosis treatment as a step to diagnosis, and postponing the diagnosis of tuberculosis [10,14]. The scoring system adopted in Brazil in 2002 to diagnose childhood tuberculosis in children and adolescents aged 15 years or less has rarely been evaluated [15–18] and was tested retrospectively, mainly with hospitalized patients, which has clear limitations [9,15,17]. The study reported in this paper aims to identify the accuracy of the scoring system proposed by the Ministry of Health to diagnosis tuberculosis in children aged 15 years or less with negative smear and culture attending outpatient clinics.

2. Methods

The study was conducted during the period 2003–2005, in a Reference Center for infectious diseases in João Pessoa Paraíba – Brazil. It is a phase III validation study, according to the Sackett and Haynes (2002) classification [19], using a cross-section design. In phase III studies, all individuals enrolled are suspected of having the disease and the investigation is carried out to verify if the test (the scoring system) distinguishes those with and without the target disorder. It was a delayed-type cross-sectional study in which the follow-up procedure was aimed at retrospectively assessing the health status at time zero, as a substitute for establishing the reference standard diagnosis of tuberculosis immediately at time zero itself [20].

Children and adolescents were considered suspects of tuberculosis according to the guidelines of the Ministry of Health [5]. In short, tuberculosis suspects were those children and adolescents who had been symptomatic for more than 15 days or who had contact with a respiratory symptomatic patient.

The reference standard for the diagnosis of tuberculosis was a blind and independent review of the medical records, radiology and tuberculin test by two experts in childhood tuberculosis disease. There was an independent assessment of the scoring and the reference standard. The experts were blind with regard to the result of the scoring for each patient, and the interviewer responsible for the application of the score was blind to the diagnosis established by the experts. Experts and the investigator responsible for the application of the score were not informed as to the diagnosis given by the attendant physician or if patients had started treatment for tuberculosis. Disagreement between the two experts was solved by a third expert who would review all discordant diagnoses. Two experts were specialists in pulmonology (by the Brazilian Society of Pulmonology and Tisiology), and one in pediatric infectious diseases (by the Brazilian Pediatric Society) and had experience in attending patients at the hospital and at the outpatient pediatric infectious disease and pediatric pulmonology clinics. Children were classified as positive by the reference standard if, after a period of six months, they had the diagnosis of tuberculosis by two (of three) experts. The steps for selection and follow-up of patients are summarized in Fig. 1.

Fig. 1

Flow diagram of the steps for selection and follow-up of patients.

Smear microscopy for acid-fast bacilli and culture for Mycobacterium tuberculosis was requested of all patients. A total of 68 patients were able to expectorate voluntarily and had the smear microscopy and culture performed, four of them had a positive smear and none had a positive culture. The four patients with a positive smear were excluded from the study.

The sample included 163 patients aged less than 15 years attending the outpatient clinics suspected of having tuberculosis disease. A total of 56 cases were diagnosed as tuberculosis disease according to the reference standard. The data were collected in a standard questionnaire applied by the investigator to the patient or caregiver. Sensitivity, specificity, predictive values, likelihood ratios and pre- and post-test probability were calculated for two score cut-off points: 30 and 40 points. All children in the study were followed up for 6 months, with complete treatment for those diagnosed with TB disease by the attending physician. None of the patients included in the study had, nor developed during the follow-up period, any sign or symptom of extra-pulmonary tuberculosis; lymph node, pleural, or other organs or systems involvement were not diagnosed. No child died. The scoring system is demonstrated in Table 1.

This project was approved by the Ethics Committee of the Hospital Universitario Lauro Wanderley Universidade Federal da Paraiba (registration CEP/HULW 371/2004).

3. Results

The study was conducted on 163 patients suspected of having tuberculosis, of which 60 were considered to have tuberculosis according to the expert review. The average and median age of the children and adolescents were, respectively, 88.69 (SD 46.28) and 91.1 months, with a minimum age of 0.3 and a maximum of 189 months. Of the total of 163 children, 7 (4.29%) were less than 12 months in age, 43 (26.38%) were aged 12–59 months, 66 (40.49%) were aged 60–119 months, and 47 (28.83) were aged 120 months or more.

For 30 as the cut-off (Table 2), the sensitivity was 78.57% (95%-CI: 65.56–88.41%), specificity 69.16% (95%-CI: 59.50–77.73%), positive predictive value (PPV): 57.14% (95%-CI: 45.35–68.37%), negative predictive value (NPV): 86.05% (95%-CI: 76.89–92.58%), likelihood ratio (+): 2,55, pre-test probability: 34.36% and post-test probability (+): 57.14%. Using 40 points as the cut-off (Table 3), the values were sensitivity: 48.21% (95%-CI: 34.66–61.97%), specificity: 87.85% (95%-CI: 80.12–93.37%), PPV: 67.50% (95%-CI: 50.87–81.43%), NPV: 76.42% (95%-CI: 67.93–83.61), LR (+): 3.87, pre-test probability: 34.36% and post-test probability (+):76.42%.

The concordance between the two experts was low (observed agreement: 61.7%, kappa of 15%). A third expert reviewed just the discordant diagnoses (n = 49). In the analysis of this subgroup, the observed agreement between the third expert and expert #1 was of 41.3% and between the third expert and expert #2 was of 74.5%.

4. Discussion

Sensitivity of 78% and specificity of 69% are low, but in the acceptable range, for a cut-off point of 30. The low expert’s concordance (observed agreement: 61.7%, kappa of 15%) may constitute a limitation of this study, although it does highlight how difficult it is to diagnosis tuberculosis in childhood. Disagreement between the two experts was solved by a third expert who reviewed all discordant diagnoses.

Some characteristics of the patients may be behind the agreement rate observed, such as the occurrence of patients with mild symptoms, with symptoms lasting less than 15 days, with normal chest radiograph, or with the presence of non-specific lesions without cavitation. Approximately 80% of the patients had a history of previous contact with a patient with tuberculosis.

Outpatient clinic studies using experts’ reviews as a reference standard are rare. Tuberculosis validation studies are, generally, hospital-based, with bacteriology and histopathology as the gold standard. This would obviously not be appropriate in this case where there is a search for a solution to the diagnosis of bacteriologically negative cases [11]. According to Sackett’s classification, this would result in limited external validity of the findings because of selection and spectrum bias [19].

There are a few studies discussing the accuracy and characteristics of the approaches used to diagnose tuberculosis in childhood. Hesseling et al. [8] analysed the current approaches to the diagnosis of childhood tuberculosis and concluded the urgent need to develop new diagnostic approaches for childhood tuberculosis. Carreira et al. [15] analysed the WHO, Kenneth-Jones and Keith-Edwards criteria using the Instituto de Pediatria e Puericultura Martagão Gesteira criteria as the gold standard. The Keith-Edwards showed higher values of sensitivity and specificity (84% sensitivity and 97% specificity). However, as this score used response to treatment with non-tuberculosis antibiotics, using this score delays tuberculosis diagnosis. Van Rheenen [21] analysed this score in a high HIV prevalence area, finding 88% sensitivity, 25% specificity, 55% positive predictive value, and 67% negative predictive value. The International Union Against Tuberculosis Lung Disease (IUATLD) point scoring system showed less than 70% sensitivity and specificity and a positive predictive value of 60 – 70% [22].

The scoring system used in Brazil was based on the other scores and on the author’s personal experience in the assistance of tuberculosis patients. The cut-off points used are empiric [7]. The adaptation was based on a decision not to require bacteriology (which is negative in most childhood cases and requires a laboratory and an expert) and not to require a response to non-tuberculosis treatment, which would delay the tuberculosis treatment. It was also validated in a hospital setting using gastric aspirate results as the gold standard: in this case sensitivity was 89% and a specificity of 86% [9]. These results are unlikely to be generalizable to outpatients.

Sant’Anna et al. [16], in a case-control study using a gastric lavage culture that is positive to Mycobacterium tuberculosis as the gold standard, report that the sensitivity of the score ranged from 58% to 89% and the specificity from 98% to 86%, with cut-offs of respectively ⩾40 or ⩾30.

Another study in a reference center in Bahia, Brazil, Sant’Anna et al., using as the gold standard radiologic aspect, clinical, epidemiological and follow-up data, report the use of the scoring system as an auxiliary element to diagnose tuberculosis in culture-negative children and adolescents [17].

Pedrozo et al. [18] also conducted a study to determine the efficacy of the Brazilian scoring system. It was a cross-sectional study performed in a pediatric pulmonology clinic, and a clinical diagnosis of tuberculosis was used as the reference standard. They concluded that the scoring system was valid for the diagnosis of pulmonary tuberculosis. However, in their study, there was a non-independent assessment of test and reference standard, the follow-up period was limited to 30 days, and they did not express accuracy through the usual parameters (sensitivity and specificity, positive and negative predictive values, or positive and negative diagnostic likelihood ratios).

This study showed that, using the point scoring system with 30 points as the cut-off, 80% (95%-CI: 70–90) of children with tuberculosis will be correctly diagnosed: a score higher than 30 provides a factor of 2.6 in favor of tuberculosis, whereas a score lower than 30 yields a factors of 3.4 (1/0.29) in favor of no tuberculosis. The pre-test probability of tuberculosis of 35% changes to 59% in case of a high score (> than 30), reflecting the informativeness of the scoring system and its potential impact on the diagnosis of childhood tuberculosis by the health care system.

HIV co-infection, if present, would interfere in the evaluation of the scoring system. Children and adolescents were not tested for HIV. However, the incidence of AIDS in Brazil, in 2003, was 5.15 per 100,000 inhabitants in the age group 0–4 years, 1.86 per 100,000 inhabitants for those in the age group 5–12 years, and 2.94 per 100,000 inhabitants for those in the age group 13–19 years [23]. Thus, it is not likely that it would influence the results of this study.

The results of this study favor the recommendation of the use of this point scoring system in Brazil and similar sites. The values of sensitivity and specificity are similar to those reported in the literature. In this setting, the cut-off of 30 points can be used to define tuberculosis diagnosis. However, the discriminatory power of the point scoring system may vary across settings as the frequency of different signs and symptoms vary as patients presenting to healthcare facilities with different levels of complexity differ. Therefore, it is recommended that this phase III study be replicated in different settings attended by patients.

5. Conclusion

The results of this study support the use of this point scoring system in Brazil and similar sites with the cut-off of 30 points. However, as the discriminatory power of the point scoring system may vary across settings, it would be useful to replicate this study in different settings.

Acknowledgements

The authors thank all the staff of the Reference Center and Hospital of Infectious Disease Clementino Fraga João Pessoa-Paraíba-Brazil. One author was partially supported by CNPq (Scholarship 308311/2009-4 to R.A.A.X.).