Reproducibility in Echotracking Assessment of Local Carotid Stiffness, Diameter and Thickness in a Population-based Study (The STANISLAS Cohort Study)
- DOI
- 10.2991/artres.k.200314.001How to use a DOI?
- Keywords
- Reproducibility; STANISLAS Cohort; wall track system; ART.LAB
- Abstract
Background: Carotid Intima Media Thickness (IMT) and stiffness are associated with cardiovascular events. The study aims were to perform a head-to-head comparison of the Wall Track System (WTS) and ART.LAB for carotid IMT, distension and diameter echotracking measurements as well as inter- and intra-observer reproducibility.
Methods: Echotracking measurements were performed with WTS and ART.LAB in 188 participants from the STANISLAS Cohort (mean age 47 ± 14 years). Inter-observer reproducibility analysis was performed in 60 patients consecutively included among the STANISLAS Cohort and two other ongoing cohorts, in whom measurements were successively performed by three operators.
Results: The relative differences between WTS and ART.LAB in artery diameter measurements were minimal (mean difference −1.8%) while the differences in IMT and distension measurements were 6.1% and 4.3%, respectively. The Bland and Altman plots for diameter, distension and intima media thickness showed no measurement bias between ART.LAB and WTS. The internal reproducibility for carotid diameter was good with the two devices (≈2% SD). The ART.LAB performed ≈2-fold better than WTS for IMT internal reproducibility (5% vs. 12.5%, p < 0.0001) and distension internal reproducibility (6.3% vs. 12.4%, p < 0.0001). The inter- and intra-observer reproducibility for carotid diameter and IMT was good for both devices. Complete results were obtained in 1:50 min in ART.LAB and 11:13 min with WTS.
Conclusion: ART.LAB and WTS show good agreement, with good inter- and intra-observer reproducibility with the two devices. Nevertheless, internal reproducibility of ART.LAB is better and measurements are easier to perform, favouring this device for carotid intima media thickness and stiffness measurements.
- HIGHLIGHTS
- •
Our results show a good agreement between ART.LAB and WTS echotracking devices.
- •
In our hands, internal reproducibility of ART.LAB is better than WTS.
- •
Carotid stiffness and thickness measurements are easier/quicker with ART.LAB than with WTS.
- •
- Copyright
- © 2020 Association for Research into Arterial Structure and Physiology. Publishing services by Atlantis Press International B.V.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
1. INTRODUCTION
The local arterial stiffness, diameter and Intima Media Thickness (IMT) measured at the site of the common carotid artery have been associated with cardiovascular events by certain medical authorities [1–9].
Local arterial diameter, stiffness and IMT are usually determined by a B-mode vascular ultrasound system coupled with a video-image analysis [10] or by direct analysis of the Radiofrequency (RF) - signal by an echotracking device [1,11]. The Wall Track System (WTS) is a high-resolution, echo-locked artery wall tracking system using radiofrequency and requires both high technical skills and a considerable amount of time to acquire the data and manage raw files [1,11,12]. In contrast, the ART.LAB is an automated evolution of WTS which requires no subjective post-processing. The comparability of the measurements derived from these two techniques has been insufficiently studied.
The Reference Values for Arterial Measurements Collaboration has published reference values for both carotid intima media thickness measured with echotracking [13] and local arterial stiffness [14] using data from different centres and techniques including B-mode- and RF-based devices. In addition, this meta-analysis encompasses data stemming from various centres using different devices and techniques [6]. Hopefully, future collaborative projects will continue to merge data from various horizons. While studies comparing ART.LAB and video analysis have been published [10], data regarding the agreement between ART.LAB and other techniques remain scarce. Better assessment of the agreement between measurement methods of IMT (especially WTS and ART.LAB) along with their reproducibility would be helpful before proceeding to collaborative efforts [15]. Good agreement and reproducibility are indeed typically the two requirements of data-merging.
The aims of this study were to perform a head-to-head comparison of the carotid IMT, diameter and distension measurements provided by the WTS and ART.LAB systems as well as their inter- and intra-observer reproducibility.
2. MATERIALS AND METHODS
2.1. Populations
Our study protocol included four phases, namely (1) the comparison of the measurements derived from the WTS and ART.LAB systems, (2) the devices’ internal reproducibility analysis, (3) the inter-observer reproducibility analysis and (4) intra-observer reproducibility. For the first and second phase of the study protocol, both IMT measurements were performed in 188 consecutive participants from the fourth examination of the STANISLAS Cohort (Clinical Trial: NCT01391442) [16–18] by a senior imaging engineer (as the goal of this phase was to provide an optimal comparison of the results provided by the two devices). For the third phase of the protocol, 60 participants were consecutively included across three ongoing cohorts simultaneously conducted in our centre, in whom measurements were successively performed by three operators (one senior engineer and two research nurses). For the fourth phase of the protocol, intra-observer reproducibility was assessed in 19 participants solely for the WTS. Below is the description of the three cohorts from which the participants were included from.
- (1)
The majority of the patients were from the STANISLAS Cohort [18–20] which is a single-centre familial longitudinal cohort comprised of 1006 families (4295 subjects) from the Nancy region recruited in 1993–1995 at the Centre for Preventive Medicine. This cohort was established with the primary objective of investigating gene–gene and gene–environment interactions in the field of cardiovascular diseases. The families were deemed initially healthy, free of declared acute and/or chronic illness, in order to assess the effect of genetics on the variability of intermediate phenotypes on the transition toward disease [19,20]. From 2011 to 2016, 1705 survivors of the original cohort underwent their fourth examination at our department. Carotid echotracking was routinely performed at the fourth examination (N = 301).
- (2)
The Ingenious Hypercare Network of Excellence [21] investigates the genetic, genomic and proteomic basis of hypertension and susceptibility to hypertension-related end organ damage [renal damage and heart failure (N = 3)]. Carotid echotracking was included in the study protocol.
- (3)
R2C2-II is a longitudinal study following R2C2 [22,23] and investigating the role of the renin angiotensin aldosterone system and cardiac matrix remodelling in the mechanism of transition toward heart failure in abdominal obesity (N = 11). Carotid echotracking was also included in the study protocol.
All research protocols were approved by Local Ethics Committee (Comité de Protection des Personnes Est III - Nancy - France) and all study participants gave a written informed consent to participate. The informed written consents were previously approved by the Local Ethics Committee.
2.2. Echotracking Devices
The non-invasive investigations were performed in a controlled environment at 20 ± 1°C after a 10-min rest in supine position. Carotid diameter, distension and intima media thickness were measured on the right common carotid artery in immediate succession with the two devices according to the respective study protocols.
2.3. Wall Track System
The WTS (ESAOTE, Maastricht, The Netherlands) [11,12] measures carotid diameter, distension and IMT in one acquisition on one RF line. The radiofrequency signal is analysed directly from one M-line perpendicular to the artery, on the two-dimensional B-mode image captured with a 7.5-MHz probe (ACUSON AU5 ultrasound system with LA13A probe) and through an acquisition card. Operators have no live view of the artery once the acquisition process is launched. The RF-line is placed 1–2 cm from the carotid bifurcation and four measurements (six beats/measurement are collected) are carried out per participant. After completing the acquisition, the operator performs the post-processing of each file (i.e. placement of callipers): callipers are placed by the operator on the RF line at the level of the arterial walls and the signal is processed by the algorithm to calculate IMT, diameter and distension. Briefly, lumen-intima and media-adventitia boundaries are detected beat-to-beat by the onboard algorithm comparing signal amplitude and the derivative of mean amplitude signal distribution with the adjacent signal (i.e. noise) on the RF line using a predefined threshold [12].
2.4. ART.LAB
The ART.LAB (ESAOTE, Maastricht, The Netherlands) [24], coupled with an ESAOTE Mylab70 XVG sonographer equipped with a LA435 18 mHz linear probe, is an evolution of the WTS and measures diameter and intima media thickness on 128 RF lines (B-Mode, up to a 4-cm zone). ART.LAB also measures diameter and distension on 14 RF lines (FB-mode). The program requires no post-processing procedures and the operator has a live view of the artery during the recording. The system displays the instantaneous standard deviation of the recorded parameters thus providing a live quality control. Carotid diameter, distension and IMT were measured at the site of the common carotid artery with the ART.LAB system, starting at 0.5 cm from the bifurcation. One Bm file and one FBm file were acquired per participant (six beats collected per file), with the carotid diameter calculated as the mean of Bm and FBm diameters. The ART.LAB algorithm is an automated and modernized version of the WTS algorithm [24]. The data window is placed and set (size, height, and centre) by the operator and the signal detection begins on the RF-line from the centre of the window toward the edge searching for the lumen-intima and media-adventitia boundaries. Diameter, IMT and distension are automatically calculated by the program (Diameter and IMT in B-mode, Diameter and distension with a higher acquisition frequency in FB-mode). Because the calliper placement step is managed automatically by the ART.LAB software, the method is notably easier to use for operators with limited experience.
3. STUDY PROTOCOL
3.1. Head-to-head Comparison of Measurements with ART.LAB and WTS
Head-to-head comparison between ART.LAB and WTS was performed in 188 participants in immediate succession by a senior operator. The ART.LAB and WTS assessments were carried out in random order. In addition, the time needed to complete the exam and obtain the final results was assessed in 15 participants by a senior operator (EB).
3.2. Internal Reproducibility of the Devices
In our standard study protocol, four measurements were obtained per participant with WTS. For each measurement, a log file was created containing the diameter, IMT and distension measured at each cardiac beat. The variability in diameter, distension and IMT was then compared between all acquired beats.
Only two measurements were obtained with ART.LAB, one B-mode measuring diameter and IMT and one FB-mode measuring diameter and distension. The parameters were measured at each of the six recorded cardiac beats. Diameter variability was assessed using Bm and FBm files, while distension and IMT variability was obtained only with FBm and Bm, respectively. The device’s internal variability was considered to be the beat-to-beat variability for each parameter within the Bm and FBm files (diameter, IMT, distension).
For the two devices, results are expressed as the mean value of the parameters, the mean Standard Deviation (SD) of all the values acquired and the SD in percentage of the mean value.
3.3. ART.LAB and WTS Inter-observer Reproducibility
Inter-observer reproducibility was assessed with the two echotracking devices on different participants. Measurements for each device were performed successively by two operators. A senior engineer (EB), trained in vascular echocardiography and responsible for the cardiovascular ultrasound laboratory was considered to be the gold standard. Three trained research nurses were compared to the gold standard. Reproducibility was assessed in 20 participants with the ART.LAB and 60 consecutive participants with the WTS.
3.4. Intra-observer Reproducibility
Intra-observer reproducibility was assessed with the WTS in 19 participants for the senior operator and the three research nurses. Measurements were carried out twice by two operators in alternation. The first value obtained was considered to be the gold standard. The probe was removed and repositioned between each measurement.
ART.LAB intra-observer reproducibility was not assessed given that the latter is expected to be very good since ART.LAB measures the parameters on six consecutive heart beats with a visual live control of the standard deviation of the parameters. However, diameter was in effect measured twice (in B-mode and FB-mode, without removing or replacing the probe) and intra-observer reproducibility could be calculated for diameter.
3.5. Statistical Analysis
Baseline characteristics of the participants are described as means and SDs for continuous variables, and proportions for categorical variables.
Internal variability of WTS and ART.LAB was estimated by the mean value, mean SD, and SD in percentage of mean value. Differences between the two devices were assessed by paired t-test (the first order risk was set at 0.05). The mean absolute and mean relative differences between the two devices were also calculated.
Inter-observer and intra-observer reproducibility was investigated using the Intra-class Correlation Coefficient (ICC). Inter-observer agreement was classified as poor (ICC ≤ 0.40), fair to good (ICC = 0.40–0.75) or excellent (ICC > 0.75).
The mean absolute difference and the mean relative difference were estimated between measurements using the WTS to determine reference values. WTS was taken as the gold standard given that this method has been used for years in our department and in particular for a major portion of the participants assessed in the three protocols.
Bland and Altman plots [25] were drawn for inter-observer reproducibility and for device comparisons. Comparison data are expressed as mean difference ± standard error and mean difference in percentage of the gold standard value. Agreement was considered good if the mean difference versus gold standard was <10%.
All statistical analyses were performed using SAS 9.3 (SAS Institute, USA).
4. RESULTS
The present population, comprised of subjects from the three ongoing studies, were middle-aged (47 ± 14 years) with a high prevalence of cardiovascular risk factors (50% had hypercholesterolemia, 26% had hypertension). Mean vital signs were within the normal range. The main characteristics of the population are presented in Table 1.
| N | Mean ± SD OR [n (%)] | Median (IQR) | |
|---|---|---|---|
| Demographics | |||
| Age (years) | 315 | 47 ± 14 | 52 (34–60) |
| Male gender | 315 | 143 (45.4) | |
| Weight (kg) | 307 | 74.7 ± 16.7 | 72.7 (61.1–85.0) |
| Height (m) | 308 | 1.69 ± 0.09 | 1.68 (1.62–1.75) |
| Bodymass index (kg/m2) | 307 | 26.2 ± 5.2 | 25.1 (22.4–29.5) |
| Vital signs | |||
| Heart rate (beats/min) | 293 | 64 ± 9 | 63 (58–59) |
| SBP (mmHg) | 293 | 119 ± 16 | 117 (107–129) |
| DBP (mmHg) | 293 | 69 ± 9 | 68 (62–74) |
| Risk factors | |||
| Hypertension | 308 | 81 (26.3) | |
| Diabetes | 293 | 15 (5.1) | |
| hypercholesterolemia | 306 | 154 (50.3) | |
| Current or ex-smoker | 308 | 63 (20.5) | |
| WTS | |||
| Diameter (mm) | 253 | 6.638 ± 782 | 6.544 (6.106–7.055) |
| Distension (μm) | 253 | 468 ± 164 | 445 (343–564) |
| IMT (μm) | 253 | 644 ± 152 | 620 (530–720) |
| ART.LAB | |||
| Diameter (mm) | 250 | 6.840 ± 812 | 6.665 (6.319–7.294) |
| Distension (μm) | 244 | 447 ± 148 | 446 (335–549) |
| IMT (μm) | 248 | 598 ± 132 | 590 (495–688) |
IQR, interquartile range; SBP, systolic blood pressure; DBP, diastolic blood pressure; WTS, wall track system; IMT, intima media thickness.
Baseline characteristics of the study participants
4.1. Head-to-head Comparison of Measurements with ART.LAB and WTS
Results with regard to the comparison between WTS and ART.LAB measured in 188 participants are shown in Table 2. The differences in artery diameter measurements using the two devices were minimal (mean absolute difference −1.8%) while the relative differences in IMT and distension measurements were 6.1% and 4.3%, respectively. The Bland and Altman plots for diameter, distension and intima media thickness showed no measurement bias between ART.LAB and WTS (Figure 1).
| Parameter | Reference value* | Mean absolute difference ± SD (μm) | Mean relative difference ± SD (%) |
|---|---|---|---|
| Diameter (mm; n = 188) | 6.679 | −118.5 ± 247.2 | −1.8 ± 3.7 |
| IMT (μm; n = 186) | 647 | 46.2 ± 98.9 | 6.1 ± 14.3 |
| Distension (μm; n = 181) | 459 | 22.5 ± 79.8 | 4.3 ± 18.1 |
Reference value is the value obtained by the senior engineer with the wall track system.
IMT, intima media thickness.
Comparison between ART.LAB and wall track system
Bland and Altman plots for comparison of carotid diameter, intima media thickness and distension between the wall track system (gold standard) and ART.LAB.
4.2. Internal Reproducibility of the Devices
The internal reproducibility for carotid diameter was good with the two devices (≈2% standard deviation of the mean value). The ART.LAB performed 2.5-fold better than WTS (5% vs. 12.5%, p < 0.0001) for IMT reproducibility. Similarly, the internal reproducibility of ART.LAB was ≈2-fold better than that of WTS (6.3% vs. 12.4%, p < 0.0001) for the distension measurement (Table 3).
| ART.LAB diameter | WTS diameter | ART.LAB IMT | WTS IMT | ART.LAB distension | WTS distension | |
|---|---|---|---|---|---|---|
| Mean value (μm) | 6636 | 6489 | 582 | 638 | 445 | 491 |
| Mean SD (μm) | 137 | 105 | 29 | 80 | 28 | 61 |
| SD in % of mean value | 2.1 | 1.6 | 5.0 | 12.5* | 6.3 | 12.4* |
p < 0.0001 vs. ART.LAB respective value.
Internal variability of the wall track system and ART.LAB
4.3. Inter-observer Reproducibility
The inter-observer reproducibility for carotid diameter and IMT was good irrespective of the device used (all <5%, Table 4). Distension exhibited lower inter-observer reproducibility with both devices.
| Operator | n | Parameter | Reference value* (μm) | Mean absolute difference ± SD (μm) | Mean relative difference ± SD (%) | Intra-class correlation coefficients (95% CI) |
|---|---|---|---|---|---|---|
| WTS | ||||||
| 1 | 58 | Diameter | 6423 | 66.3 ± 337.6 | 0.9 ± 5.7 | 0.95 (0.91–0.97) |
| 2 | 55 | Diameter | 6354 | −32.8 ± 300.6 | −0.6 ± 4.8 | 0.93 (0.88–0.96) |
| 3 | 56 | Diameter | 6290 | −76.8 ± 304.3 | −1.3 ± 4.9 | 0.94 (0.90–0.97) |
| 1 | 59 | IMT | 613 | 25.3 ± 112.3 | 1.7 ± 16.5 | 0.87 (0.78–0.92) |
| 2 | 55 | IMT | 569 | 2.0 ± 63.2 | −0.7 ± 11.3 | 0.88 (0.79–0.93) |
| 3 | 55 | IMT | 573 | −0.2 ± 51.2 | −0.1 ± 8.9 | 0.92 (0.86–0.95) |
| 1 | 58 | Distension | 493 | −3.2 ± 100.4 | −3.8 ± 22.3 | 0.90 (0.83–0.94) |
| 2 | 54 | Distension | 511 | −11.7 ± 90.4 | −3.5 ± 17.2 | 0.93 (0.88–0.96) |
| 3 | 55 | Distension | 527 | 39.3 ± 84.4 | 7.3 ± 16.2 | 0.95 (0.92–0.97) |
| ART.LAB | ||||||
| 1 | 20 | Diameter | 6881 | 126.3 ± 248.8 | 1.7 ± 3.7 | 0.97 (0.93–0.99) |
| 2 | 21 | Diameter | 7216 | 120.7 ± 330.0 | 1.5 ± 4.0 | 0.97 (0.92–0.99) |
| 3 | 20 | Diameter | 6832 | 337.3 ± 321.9 | 5.0 ± 4.8 | 0.95 (0.87–0.98) |
| 1 | 20 | IMT | 596 | −7.4 ± 85.4 | −2.8 ± 14.2 | 0.89 (0.73–0.96) |
| 2 | 21 | IMT | 617 | −20.9 ± 55.2 | −4.2 ± 9.4 | 0.97 (0.92–0.99) |
| 3 | 20 | IMT | 559 | −10.2 ± 76.4 | −3.2 ± 14.6 | 0.81 (0.51–0.92) |
| 1 | 20 | Distension | 503 | −18.6 ± 58.3 | 4.8 ± 12.2 | 0.95 (0.87–0.98) |
| 2 | 21 | Distension | 440 | −30.1 ± 75.9 | −8.1 ± 19.6 | 0.92 (0.81–0.97) |
| 3 | 20 | Distension | 502 | −43.8 ± 111.2 | −11.2 ± 25.5 | 0.71 (0.28–0.89) |
Reference value is the value obtained by the senior engineer with the WTS.
IMT, intima media thickness; CI, 95% confidence interval.
Inter-observer reproducibility of the wall track system and ART.LAB
The ICCs for WTS parameter assessment with expert reading as gold standard were classified as excellent for all operators (0.87–0.95). The ICCs for ART.LAB parameters assessment were classified as good (distension for one operator) to excellent (0.71–0.97).
4.4. Intra-observer Reproducibility
The intra-observer reproducibility of WTS was classified as good to excellent for all parameters, ranging from 0.65 to 0.98 (Table 5). Higher ICC coefficients were observed for the expert reader (all ICC > 0.90).
| Operator | n | Parameter | Reference value* (μm) | Mean absolute difference ± SD (μm) | Mean relative difference ± SD (%) | Intra-class correlation coefficients (95% CI) |
|---|---|---|---|---|---|---|
| 0 | 19 | Diameter | 6295 | 74.8 ± 163.7 | 1.2 ± 2.6 | 0.98 (0.934–0.990) |
| 1 | 19 | Diameter | 6150 | −28.6 ± 365.9 | −0.6 ± 6.3 | 0.88 (0.698–0.955) |
| 2 | 19 | Diameter | 6341 | −33.0 ± 307.7 | −0.6 ± 4.8 | 0.90 (0.743–0.962) |
| 3 | 19 | Diameter | 6232 | −148.3 ± 387.1 | −2.5 ± 6.5 | 0.86 (0.638–0.946) |
| 0 | 19 | IMT | 552 | 8.0 ± 50.3 | 0.8 ± 9.0 | 0.91 (0.753–0.963) |
| 1 | 19 | IMT | 519 | −36.5 ± 80.6 | −7.8 ± 16.8 | 0.65 (0.100–0.866) |
| 2 | 19 | IMT | 545 | −8.6 ± 52.7 | −2.3 ± 9.5 | 0.86 (0.646–0.948) |
| 3 | 19 | IMT | 562 | 30.1 ± 69.7 | 4.5 ± 11.2 | 0.88 (0.676–0.952) |
| 0 | 19 | Distension | 530 | −20.2 ± 75.7 | −5.0 ± 14.2 | 0.95 (0.874–0.981) |
| 1 | 19 | Distension | 550 | 21.8 ± 144.5 | −1.1 ± 32.0 | 0.77 (0.393–0.910) |
| 2 | 19 | Distension | 550 | −1.2 ± 94.1 | −3.1 ± 20.7 | 0.91 (0.771–0.966) |
| 3 | 19 | Distension | 496 | 14.2 ± 119.5 | −0.6 ± 21.8 | 0.87 (0.651–0.948) |
Reference value is obtained by the wall track system. Operator 0 is the senior engineer.
IMT, intima media thickness; CI, 95% confidence interval.
Intra-observer reproducibility of the wall track system
Regarding ART.LAB reproducibility, 60 patients were measured by the senior operator using B- and FB-mode; the diameters extracted from this method were highly correlated with a linear correlation coefficient of 0.983.
4.5. Amount of Time Needed to Perform the Exams
The time needed to obtain the complete results was assessed in WTS and ART.LAB. The analysis revealed a substantial difference between the two devices: final results were obtained in 1:50 min (SD 00:21) in ART.LAB while 11:13 min (SD 02:11) were needed to obtain the results with WTS.
5. DISCUSSION
Overall, in this study, we documented a good agreement between ART.LAB and WTS measurements as well as good internal, inter- and intra-observer reproducibility in carotid structural and functional assessment with the two devices. However, the ART.LAB device appeared more reproducible (reproducibility 6% or less for all considered variables) and required less time (<2 min vs. more than 11 min), skills and knowledge to measure local carotid stiffness and structure. These findings strengthen the validity of the pooled analysis of data derived from a mixed set of ART.LAB and WTS measurements [13,14] and promotes the use of the ART.LAB or ART.LAB-derived technology in the future given its better internal properties and ease of use.
5.1. Comparison of Measurements Provided by the ART.LAB and WTS Devices
To the best of our knowledge, we provide the largest head-to-head comparison of WTS and ART.LAB measurements in a population setting. We took advantage of the STANISLAS Cohort in which both measurements were performed in immediate succession in 188 participants to undertake this head-to-head comparison. The difference in the measurements using both devices were minimal (distension: 4.3%; diameter: −1.8% and IMT: 6.1%) and no bias was shown in Bland and Altman plots. WTS measures on one RF-line at 1–2 cm from the bifurcation while ART.LAB measures on a long arterial segment starting from 0.5 cm below the bifurcation. In this segment where diameter is not homogeneous, regional differences in distension have already been shown between carotid bulb and common carotid artery [26] and can explain part of the differences observed between the two devices. In addition, carotid IMT is usually heterogeneous near the bifurcation [27] and WTS only provides a one-line mean IMT (although we performed four measurements) while ART.LAB provides a 128 lines/4 cm mean IMT, thus less prone to bias in instances of local anatomical and inter-cycle heterogeneity of intima media thickness.
5.2. Internal Reproducibility of the Devices
While internal variability was closely similar between ART.LAB and WTS in terms of diameter assessment, ART.LAB showed greater superiority for IMT (5% vs. 13%) and distension (6% vs. 12%) measurements. This improvement in accuracy between the two generations of devices can be partly explained by both technological and detection algorithm improvements, and also by the fact that ART.LAB allows a live control of RF-line positioning during acquisition, enabling the operator to achieve better stability, whereas the WTS screen is frozen during acquisition time.
5.3. Inter- and Intra-observer Reproducibility
Inter- and intra-observer reproducibility are critical points in long-lasting research protocols such as the STANISLAS cohort [19,20] or protocols requiring repeated assessments usually by different operators. Accordingly, a validation study within the STANISLAS cohort was recently conducted in which cardiac ultrasound and diastolic function measured by echocardiography were proven to be highly reproducible [18]. Likewise, we assessed herein the inter- and intra-observer variability of WTS and ART.LAB measurements. The two devices exhibited excellent inter-observer and good to excellent intra-observer reproducibility. Dogan et al. [28] also compared inter-observer reproducibility of IMT measurements by video analysis in RADIANCE 1 and 2 (NCT00134264 and NCT00136981) studies with ICC coefficients ranging from 0.678 to 0.908. However, the target population differed from the present study with most participants having dyslipidemia and thus explaining a higher mean IMT than that found in our population.
5.4. Duration of Acquisition
The substantial difference observed herein in terms of examination time is due to the fact that WTS requires long and complicated post-processing to obtain the final results. Moreover, the acquisition process is longer in WTS since our quality assurance procedure requires four acquisitions per patient. Once the images are acquired, the operator must carefully place the callipers to allow an accurate detection of the arterial layers and obtain the final result. The degree of skill (and time) required to achieve accurate results using this technique has been discussed by others [1]. In contrast, the ART.LAB results were obtained in <2 min which compares very favourably against the 11 min needed to obtain the WTS results.
5.5. Clinical and Research Implications
Carotid IMT, diameter and stiffness is a useful cardiovascular risk marker although the high accuracy offered by RF-based technology is critical for clinicians who wish to use the latter in their daily practice. Indeed, its widespread use is limited by several difficulties. In the present analysis, we provide strong evidence for the superiority of ART.LAB over WTS given its better internal reproducibility and quicker acquisition and reading time. From our perspective, precise carotid properties measurements have the potential to become a widespread (i.e. not limited to a few pathophysiology experts or clinical trialists) and clinically relevant tool only with reliable measurement devices (such as ART.LAB or other validated techniques not assessed in the present study).
In addition, given that joint analyses of existing databases are becoming more frequent, it is critical to determine whether datasets using different IMT measurement devices (i.e. either the WTS or ART.LAB device) can be safely merged together. We provide strong evidence for the relevance of such merging without any modification in initial IMT measurements. Nevertheless, since the internal reproducibility of the WTS is larger than that observed with the ART.LAB system, the degree of measurement fluctuation is higher with WTS, thus probably reducing associations with relevant clinical outcomes. The extensive reproducibility analysis conducted in the present study strongly supports future collaborations/meta-analyses of IMT studies and hopefully will strengthen the validity of their output.
5.6. Limitations
Certain limitations should be acknowledged in the present study. When the study was conducted, WTS had long been used in the department while ART.LAB was a new acquisition. Thus, the comparison of inter-observer reproducibility between WTS and ART.LAB could be biased by the difference in experience of the nurses in the use of the devices. Notwithstanding the latter, in participants seen in the three protocols, the intra-observer reproducibility of the senior engineer with WTS was better than the internal reproducibility of the device (mean relative difference: diameter 1.2%, IMT 0.8% and distension −5%; intra-observer ICC for diameter: 0.98, distension: 0.95, IMT: 0.91). Intra-observer reproducibility was not calculated for the ART-LAB system but the diameter was in effect measured twice (B- and FB-Mode) and intra-observer reproducibility for the diameter was very good (correlation coefficient of 0.983). Finally, the studied population comprised a sizeable proportion of cardiovascular risk factors. A different reproducibility could potentially be observed in other settings.
6. CONCLUSION
Overall, the present study shows good agreement between ART.LAB and WTS measurements as well as good inter-and intra-observer reproducibility with the two devices. These findings strengthen the validity of analyses (e.g. meta-analyses) performed on data derived from both ART.LAB and WTS measurements. However, the ART.LAB device exhibited a better internal reproducibility and required less time and operator skill set. Hence, ART.LAB can appropriately replace WTS and allow non-expert manipulators to quickly perform a reliable measurement of carotid structure and function. With this technology, local arterial stiffness assessment is thus more technically affordable to a larger number of healthcare providers. This device will hopefully help the widespread use of IMT measurement, which could translate into better cardiovascular risk stratification.
CONFLICTS OF INTEREST
Prof. Zannad has received fees for serving on the board of Boston Scientific; consulting fees from Novartis, Takeda, AstraZeneca, Boehringer-Ingelheim, GE Healthcare, Relypsa, Servier, Boston Scientific, Bayer, Johnson & Johnson, and Resmed; and speakers’ fees from Pfizer and AstraZeneca. Prof. Rossignol has received board membership fees from Novartis, Relypsa, and StealthPeptides. Dr. Girerd and Dr. Ferreira have received board membership fees from Novartis. The other authors have no relationships to disclose relevant to the contents of this article.
AUTHORS’ CONTRIBUTION
EB contributed in conceptualization of the study, data collection and drafting of the manuscript. NG contributed in supervision of the project, editing of the manuscript. JPF contributed in editing of the manuscript. IL contributed in statistical analysis. FZ and PR contributed in supervision of the project, conceptualization of the study.
FUNDING
The indicated studies were sponsored by the Nancy CHRU. They were supported by the Programme Hospitalier de Recherche Clinique (PHRC), by a public grant overseen by the French National Research Agency (ANR) as part of the second “Investissements d’Avenir” programme (reference: ANR-15-RHU-0004), and Contrat de Plan Etat Région Lorraine and FEDER, La Région Lorraine, and la Fondation de Recherche en Hypertension artérielle, and by the EU FP6 program Ingenius Hypercare.
ACKNOWLEDGMENTS
The authors deeply thank the entire Clinical Investigation Centre staff and collaborators (S1) all of whom are involved in the daily management of the STANISLAS Cohort and especially the research nurses (Lydie Poinsignon, Nathalie Robin, Michèle Schmartz) involved in the echotracking measurements, supervised by Erwan Bozec.
SUPPLEMENTARY MATERIAL
Supplementary data related to this article can be found at
Footnotes
REFERENCES
Cite this article
TY - JOUR AU - Erwan Bozec AU - Nicolas Girerd AU - João Pedro Ferreira AU - Ichraq Latar AU - Faiez Zannad AU - Patrick Rossignol PY - 2020 DA - 2020/03/20 TI - Reproducibility in Echotracking Assessment of Local Carotid Stiffness, Diameter and Thickness in a Population-based Study (The STANISLAS Cohort Study) JO - Artery Research SP - 5 EP - 12 VL - 26 IS - 1 SN - 1876-4401 UR - https://doi.org/10.2991/artres.k.200314.001 DO - 10.2991/artres.k.200314.001 ID - Bozec2020 ER -