Quality of Reporting in Human Aortic Tissue Research – A Systematic Review
- DOI
- 10.2991/artres.k.191106.003How to use a DOI?
- Keywords
- aortic tissue; reproducibility; reporting
- Abstract
Ex vivo human tissue is a valuable research resource. However, if vital methodological information such as anatomical location, tissue processing procedures, or donor characteristics are not reported in scientific literature to a high standard, studies utilising ex vivo human tissue can be difficult to replicate. Furthermore, data analysis and interpretation based on these studies can be challenging. In this systematic review, we focus on the reported use of human aortic tissue in research. The human aorta is a complex tissue, with embryological, biochemical and biomechanical variations along its length, which alter with age, and differ between genders and ethnicities. The aorta therefore serves as an excellent case study for examining the importance of high quality and robust reporting of methodology when utilising human tissue samples, for reliable interpretation and reproducibility. In this systematic review, we sought to critically analyse scientific papers published between 1980 and 2017 which utilised human aortic tissue to determine whether the methodological information provided would be sufficient for replication, comparison with other studies and interpretation. Eight databases (Springerlink, ScienceDirect, PMC, PLoS, JSTOR, Pubmed, Web of Science, Scopus) were mined for articles that contained the search term ‘human aortic tissue’ from January 1980 to August 2017. Following review, 143 full-text articles were selected, data extracted, tabulated and analysed. The review highlighted several areas where reporting of human aortic tissue use was insufficient for replication and thorough data interpretation. The use of control tissue was often poorly explained and in many cases, omitted completely. Sample size was largely difficult to calculate and 30% of studies did not provide this information. Age/gender information was absent in 30% of studies. Tissue storage and handling information was present in 78%, and 75% of studies gave information about statistical analyses but few gave enough information for replication. Overall the quality of reporting in many studies was deemed to be of a low standard for replication and reliable interpretation of the reported findings. Here we propose five simple recommendations for the reporting of human tissue with the primary aim of improving reproducibility and transparency in the sector, avoiding bias and maximising output.
- Copyright
- © 2019 Association for Research into Arterial Structure and Physiology. Publishing services by Atlantis Press International B.V.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
1. INTRODUCTION
Basic research into the mechanisms that underpin clinical pathology is a vital step in understanding disease mechanisms and in the development of therapeutics. Much of this work relies on the use of model systems and the use of ex vivo human tissue. Many studies have highlighted problems in general scientific reporting and transparency; it is estimated that 85% of research is wasted by poor practice throughout the process [1]. Inadequate reporting prevents proper replication and comparison of data, and valuable resources that are often difficult and time-consuming to obtain, are squandered. Worryingly, poor reporting in some scenarios can directly affect patient care [2,3]. Although this information is primarily from clinical trial data, it seems likely that these trends are observed across multiple scientific disciplines. In the wider community this has led to the development and publication of various reporting guidelines such as Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for systematic reviews [4], Animal Research: Reporting of In Vivo Experiments for animal studies [5] and Consolidated Standards of Reporting Trials for randomised trials [6]. The Enhancing the QUAlity and Transparency of health Research (EQUATOR) network has a searchable library for these guidelines and provide toolkits and training to encourage the correct and appropriate reporting to maximise output and reliability of studies (https://www.equator-network.org/).
Animal models are extremely important in biomedical research for both understanding the pathophysiology of disease processes as well as for developing therapeutic interventions. For example there are well-established models for arterial diseases including hypertension and atherosclerosis [7] However, there can be challenges with the use of animal models because in many cases they fail to recapitulate the complexity of some diseases [8]. There can be relatively poor predictive ability associated with translation from animal studies to human trials, resulting in a high drop-out rate at failed clinical trials [9,10]. Human tissue is therefore an appealing alternative to animal models; however, obtaining appropriate human samples and suitable control tissues is challenging [11].
Age, race and gender are factors that significantly alter human biochemistry and physiology [12,13]. It is logical therefore that, where possible, samples compared across different patient groups should be age, ethnicity and gender matched to minimise the impact of these variables on any study in question. Furthermore, different sample handling and processing techniques required for experimentation can affect their properties. Collectively, if these factors are not taken into consideration during analysis it can result in erroneous interpretation of results. The difficulty of obtaining appropriate control tissue for human tissue studies make the scenario of the perfect age/gender matched control tissue from disease-free patients that have been handled/treated identically a near impossibility [11]. Within the EQUATOR library, there is a single guideline document identified from the search term ‘human tissue’ entitled ‘Biospecimen reporting for improved study quality’ or BRISQ [14]. BRISQ highlights some of these issues, in particular those relating to handling and storage of human samples and puts forward very comprehensive guidelines for the reporting of these parameters. However, BRISQ does not cover some of the other key areas that are critical for interpretation and replication of studies involving human tissue. For example, sample size, use of control tissue and data analysis are not specifically covered in the BRISQ guidelines. The present review aims to appraise a subset of research articles which use human aortic tissue in basic research as an exemplar for the wider discussion on human tissue use.
The human aorta is an embryologically complex tissue. Formation of the heart occurs early in embryonic development, a pool of cardiac precursor cells fuse in the midline and form a beating linear tube, as it elongates through the proliferation of precursor cells it loops to form the four-chambered heart with a single outflow tract [15]. This single outflow tract is connected to the aortic sac from which the pharyngeal arches derive, connecting the heart to the dorsal aortae (Figure 1, left). There are five pairs of arches that are subsequently remodelled. The first and second arch pairs involute; the third arch pair transforms into the common carotid and the proximal internal carotid arteries. The left fourth arch becomes part of the aortic arch, whilst the right fourth arch forms part of the right subclavian artery. Proximal portions of the sixth arch arteries form the right and left pulmonary arteries respectively. The distal portion of the right sixth arch involutes, whereas the distal part of the left sixth arch forms the ductus arteriosus. The dorsal aortae reorganise; the right branch into the right subclavian and the left branch becomes the aortic arch and descending aorta [15–17]. As demonstrated in Figure 1, the fully formed aorta comprises tissue derived from five distinct cell populations.
Embryonic development of the aorta highlighting the different embryological origins of the aortic anatomy.
In addition to different embryological origins, regions of the aorta have different mechanical [18,19] and biological properties [20,21] that change with age [19] and gender [22]. For these reasons, it is critical that accurate reporting is in place to ensure correct interpretation and understanding of the data presented. Although the comments herein are related to the aorta specifically, the issues raised are relevant to all human tissue research.
We hypothesised that reporting and methodology surrounding the use of human aortic tissue would prove insufficient for thorough replication and comparison across studies, and that the use of appropriate control samples would be lacking. We aimed to highlight areas in experimental design and reporting of information that require improvement and put forward simple recommendations to permit comparison across studies and enhance translation from bench to bedside.
2. MATERIALS AND METHODS
2.1. Search Strategy and Article Selection
A schematic overview of the review protocol is given in Figure 2. To identify published articles that involved the study of human aortic tissue, we searched eight databases (Springerlink, ScienceDirect, PMC, PLoS, JSTOR, Pubmed, Web of Science, Scopus) for articles that contained the search term ‘human aortic tissue’. This search term was designed to identify articles that used human aortic tissue in research (Table S1). The review aimed to take a snapshot of research pertaining to human aortic tissue samples and assess the reporting, rather than to exhaustively assess all work on human aortae. As such we only used the single search term. The inclusion and exclusion criteria were agreed by all authors prior to review and are detailed in Table 1.
Schematic overview of the systematic article selection. Article numbers at each stage are presented in square parentheses.
Inclusion
|
Exclusion
|
Inclusion/Exclusion criteria
The initial review process, search result consolidation, duplication removal and abstract screening was managed using Covidence software (Melbourne, Australia). Abstract screening and subsequent full-text review and data mining was performed by two reviewers, and differences of opinion were resolved through discussion. A further thirty papers were randomly selected for assessment by a third reviewer.
2.2. Data Evaluation
The basic characteristics of the selected articles such as author information, publication year and journal classification according to Journal Citation Reports (JCR), were tabulated. We also categorised the techniques used into biomechanical, biochemical or imaging and combinations thereof. Where possible the mean number of patient samples used per group/per experiment were recorded. The inclusion of age/gender information, tissue handling/storage information and statistical methods were also recorded and tabulated (Table S1).
A key aim of this review was to assess the quality of experimental controls. During the review of each article, the type of experimental control was recorded and then subsequently divided into four categories; non-diseased human tissue, comparison study, no controls used and other. Details of these categories can be found in Table 2.
1. Human non-disease tissue
|
2. Comparison study
|
3. No controls used
|
4. Other
|
Control categories
3. RESULTS
Primary database searches yielded 623 articles for initial abstract review. Following abstract screening, 210 papers were selected for full paper review and a total of 143 full-text articles were fully analysed and included in the review (Figure 2). The primary reason for exclusion after abstract screening was the lack of human aortic tissue use (e.g. computational modelling of the aorta where no experimental work on tissue samples had been conducted, or where animal tissue was utilised and compared to human tissue data from other studies).
3.1. Characterisation of Subject Articles
From our study it is apparent that there is an overall upward trend in the number of papers being published that use aortic human tissue, from 1980 to 2017 (Figure 3A). The articles came from a wide range of journals, spanning across 25 different classifications. Most of the papers came from the following four JCR classifications; surgery, peripheral vascular disease, engineering-biomedical and pathology (Figure 3B). Assessment of the research discipline as biomechanical, biochemical or imaging revealed that 62% of the articles fell into a single discipline, 38% crossed at least two disciplines and overall 75% of the articles included some biochemistry (Figure 3C).
Overview of the paper characteristics included in the study. The number of studies published by year (A), journal classification (Journal Citation Reports) (B) and the breakdown of primary experimental techniques used in the articles as a Venn diagram (C).
3.2. Tissue Source and Control Tissue
The tissue samples used in the studies reviewed were obtained from three sources; during surgery, at autopsy or from transplant organs (heart/kidneys). Most tissues were from surgery or autopsy or a combination thereof (31%, 30% and 22% respectively) (Figure 4A).
Tissue source and control tissue information. Breakdown of tissue source (A) and the control categories (B).
A key aim of this review was to explore the use of different controls in experiments involving human aortic tissue. To simplify the discussion, the use of controls within the review articles was divided into four categories; healthy human aortic tissue, comparative studies, no controls used and other. The details of each grouping can be found in Table 2. Within this cohort, 98 studies used non-diseased human tissue as a control, 10 were comparison studies between disease groups and 28 studies fell under the category of ‘no controls’ (Figure 4B).
Whilst the number of studies using non-diseased human tissue is high, these divide into several sub-categories with overlap in some instances. Most of the studies in this category compared diseased tissue with non-diseased tissue from autopsy or transplant organs. This option represents a good model with regards to comparing healthy and diseased tissue; however, a major drawback can be the differences in tissue handling. In most of the studies where autopsy tissue was used, the time of tissue collection post-mortem was either not stated or was hugely varied. It has long been established that DNA remains relatively stable during the post-mortem interval; however, RNA and proteins level are much more labile [23–25]. Thus, comparison of post mortem tissue with tissue collected from patients during surgery must be approached with caution.
In some of the studies the authors compared diseased tissue with tissue collected from a different non-diseased site during surgery such as punch biopsies from coronary artery bypass or strips of aorta taken during valve replacement surgery. Although the aorta in these instances maybe largely normal, these patients have clinically significant disease and thus are not likely to truly represent healthy tissue. Similarly, many studies took a single cohort of patients e.g. all autopsy cases and then used histology to classify the patients into groups, with one of the groups representing disease-free or ‘normal control’. This has benefits with regards to age/gender matching and equal group size; however, it again raises the question of whether the tissue can truly be considered a ‘normal’ sample if the donor has some level of cardiovascular disease.
Furthermore, it was noted during the analysis that in some cases ‘comparative’ tissues were collected from different sites e.g. thoracic aorta with infrarenal aorta. Different regions of the aorta have different embryological origins [16], cellular and matrix organisation [26] as highlighted above. It is important to understand these differences and to take them into consideration when comparing across different regions; to ensure that differences and study findings being reported are a true reflection of a disease process, and not anatomical location differences. These caveats may not be relevant to some studies; however, clear reporting of all relevant details including tissue location and detailed storage information will allow the community to appraise studies with all the relevant information to make appropriate conclusions from these studies.
The ‘no control’ group was particularly broad comprising pilot studies or cell isolation studies of healthy individuals, and in other instances where the investigators did not deem control tissue to be necessary. However, despite this variation most of the studies within the ‘no control’ group (Group 3) did not include any controls, or the information was not available or insufficient to appraise. There were only seven studies in the final group (Other).
3.3. Patient Numbers
Another key aim of this study was to investigate the sample size. Due to the difficulty in obtaining human tissue, the number of patient samples used in human studies is relatively low compared to animal studies and can often be underpowered. Publications reviewed here are no exception; where possible the number of patients per group per experiment was extracted from the articles and represented in Figure 5. The mean number of patients per group per experiment was 9.6 with a range of 1–23. In some cases, the low sample size was justified because the study’s aim was either method development or a proof of concept; further investigations must contextualise the findings presented based on sample size, and to avoid any inference to the larger population until further independent validation is obtained.
Analysis of patient numbers used in studies shown as box plot. The plot shows the mean number of patients used per experiment per group, where it was possible to determine this information. Variable distribution is shown as blue box containing the interquartile ratio (first and third quantiles) with the median (central line) and mean (dashed line) shown, whiskers represent the 5–95% range respectively.
Another important point to consider is the correct standardisation of sample size reporting. A large percentage of manuscripts tend to report sample size as a cumulative figure for all of the samples used in the different experiments presented in the manuscript. While technically correct, this might mislead the readers as it implies that the power of the study is much larger than it is. A more correct approach would be to report the sample size used per test, as this accurately informs about their power. In many publications, the number of samples varies for different experiments/tests. However, the rationale for this is often lacking. This can point towards simple limitations in experimental design or budget but can also point towards biased presentation of results by selecting the specific samples that best fit the hypothesis.
In cases where it was possible to determine n numbers, in many cases, the information had to be calculated or determined from graphs, or from information presented within figure legends, and this information was not readily accessible within the methods section. The average reader is not likely to read in this much detail and thus there is again scope for misinterpretation.
Strikingly, it was not possible to extract sample size information from 30% of papers. The primary reason for this was unclear division of the number of samples from patients into different experimental groups. For example, a study may have collected 50 samples from 12 patients and then re-categorised them based on histological findings into four groups. This made it difficult to ascertain how many patients contributed to the samples in each group. It could for example be possible that the samples for an entire group came from a single patient.
It was also interesting to note that in many cases the number of patients in the study group was significantly higher than the corresponding control group. It is also noted that n number was particularly difficult to determine for imaging studies where representative images are presented, but no indication provided as to how many samples or patients the reported result was observed in.
3.4. General Reporting Issues
In addition to the parameters stated above, there are several other key factors that need to be reported to put the research into the appropriate context and provide information that is important for reproducibility including: patient metadata storage and handling information and details of statistical analyses.
In the papers assessed in this review, 30% of the studies presented no age or gender information. As alluded to earlier, age and gender information is key to the interpretation of data and if not taken into account can generate bias. Many studies detail changes in the aorta associated with age and the corresponding alterations in geometry [22], biochemical and mechanical properties [18]. Many of these also differ between genders. There is a gender-related discordance in the incidence and outcomes of aortic aneurysm [27] and dissection [28], which is yet to be fully explained. However, there are suggestions that hormonal differences play a role and that there may be differences in elastin and elastolytic enzymes between the sexes [29].
Almost all the studies failed to detail other potential confounding factors such as Body Mass Index (BMI), history of hypertension, statin use, diabetes etc. Ethnicity data were noticeably absent from all but a handful of studies. Epidemiology data has identified differences in the prevalence of aortic disease between ethnicities. For example the incidence of abdominal aortic aneurysm in men in the Asian population is reportedly 10-fold lower than that of Caucasian men [30]; African-Americans are at significantly lower risk of developing severe aortic stenosis than Caucasians [31]. Physiological differences have also been reported such as such as thicker aortic wall [32] and higher pulse wave velocity measurements [33,34] in African-Americans when compared to Caucasians. Although these differences are smaller than the contributions made by age and gender [32], they are nonetheless factors to consider in analysis, particularly given the marked difference in prevalence between ethnic groups.
Storage and handling information was provided in 78% of the studies reviewed here. However, in most of the studies, the information was insufficient for replication of the procedure. It is well established that storage and handling of tissues can affect their biological properties. Practically, it is highly likely that tissues will be stored or processed prior to analysis; however, it is critical that these processes and procedures are consistent within a study as far as possible and well reported. There is evidence that freezing aorta samples and storage prior to analysis does not affect the biomechanical properties of the aorta [35]. Nevertheless, subtle differences in protocol have been shown to alter properties. For example, samples stored at different temperatures exhibited altered biomechanical properties [36–38].
Regarding statistical tests, 75% provided information about all or some of analyses performed. However, a largely shared caveat was the lack of evidence to support the choice of statistical tests (such as assumptions of normality or homoscedasticity) that classically defaults to Student’s t-tests and ANOVA with no consideration for more suited non-parametric alternatives when the general hypotheses could not be met. Furthermore, several graphical representations lack uncertainty measures (e.g. error bars or standard deviation values), which are paramount to assess the validity of the data presented. Most publications reviewed in this work do not provide the data used for the analysis nor full disclosure of patient/animal metadata, which makes it impossible to reproduce the analysis or test further hypotheses. It is also noteworthy that several methodological publications (e.g. optical imaging techniques) do not provide evidence on the internal validation of their method to account for bias in quantification/detection within technical replicates.
3.5. Limitations of the Study
This review represents a snapshot in time of research papers pertaining to the study of human aortic tissue. Although the search term was broad some papers may have inadvertently been omitted if they did not include the exact term. We have used human aortic tissue as an exemplar tissue, but we strongly believe that many of the pitfalls highlighted here are relevant to research in other human tissue; however, we also recognise that this focussed review does not provide that evidence.
In order to probe some of these issues it was necessary to classify studies into categories. This categorisation is by nature subjective; we have tried to limit this by obtaining a consensus between at least two authors for all classifications made.
4. CONCLUSION
In general, the reporting of all key information that is required to replicate a study and compare data across different studies was poor. In very few cases the information was easily obtained; however, in the vast majority the required information was presented in a disjointed manner, for example, spread across a number of sections and within the supplementary information.
We acknowledge that it is often beyond the control of the research group to obtain perfectly matched comparison groups in a study (e.g. equal numbers in each sub-group). However, this should be recognised in a limitations section within the article so that the reader is fully aware and can consider these limitations when assessing the relevance of the data presented and design of future studies.
4.1. Recommendations
The aim of this review is to appraise a subset of research articles which use human aortic tissue in basic research as an exemplar for the wider discussion on human tissue use. This review has highlighted several common pitfalls that we feel are not addressed within the guidelines currently available within the EQUATOR library. In order to improve the reproducibility and transparency for the studies involving human tissues, there is a definite need for guidance that can be adopted by the broader scientific community and embraced by the peer-review system. Below (Table 3), we outline some basic and achievable points for reporting studies involving the use of human aortic tissue specifically but have broad relevance to all studies involving human tissue.
| 1. Clearly describe the cohort(s) used in the methods. |
| 2. State the site of tissue collection. |
| 3. Provide patient demographic data. |
| 4. Make all raw and processed data available. |
| 5. Describe and discuss the limitations of the study. |
Recommendations
4.1.1. Clearly describe the cohort(s) used in the methods
This should include detail about tissue source (cadaveric/transplant etc) and sample size, including the number of pieces per patient, for each experiment. If division of tissue samples is complex, we suggest including a table outlining each experiment and the corresponding patients involved. If samples are excluded explicitly, state the reason for exclusion.
4.1.2. State the site of tissue collection and give full details of storage and handling information
This is particularly important for aortic tissue; Figure 6 details possible division of the aorta that takes account of some of the regional differences highlighted in Figure 1. Most of the studies discuss aortic tissue as either, root and ascending, arch or descending (Figure 6A). Whilst better than no division at all, this description is limited and given the considerable variations in properties along the aortic length, a more detailed description would be advantageous. Taking account of the embryological and haemodynamic differences, we propose division of the aorta into a minimum of 10 sections (Figure 6B). There is of course scope for further division particularly of the descending thoracic aorta to correspond to intercostal branches. However, this limited improvement would be significantly better than existing reporting.
Possible division of the aorta into reporting sections. Division of the aorta into three sections, most commonly observed in the studies reviewed herein (A). Proposed division incorporating embryological and haemodynamic differences which could affect mechanical and biochemical properties (B).
4.1.3. Detail the patient demographic data in a table including age/gender/race/BMI/other clinical confounding factors
Metadata is key to the wider interpretation of data. As we acknowledge earlier, it is incredibly difficult to obtain perfectly matched disease and control samples especially in the study of cardiovascular diseases where often the only source of ‘healthy tissue’ is cadaveric. Accordingly, it is particularly important to outline the patient metadata to enable the reader to interrogate potential limitations in matching and the implications for the findings. As well as giving the opportunity to draw further hypotheses to test within the different demographics presented.
4.1.4. Make all raw and processed data available
The benefit of this is twofold. First, it increases the amount of data available for the community to use, making the most of this scarce resource and encourages collaboration. Second, it promotes transparency and enables thorough replication.
4.1.5. Describe and discuss the limitations of the study
When working with human samples there are unavoidable limitations that all researchers must face. However, to minimise unwanted bias and patient misclassification it is necessary to provide a proper description and discussion of limitations to help less-specialist readers interpret data correctly and highlight additional areas for future research.
Many of these points are universally applicable to human tissue studies in general. Accordingly, we have registered our intent with the EQUATOR network to develop simple reporting guidelines for the use of human tissue in research.
CONFLICTS OF INTEREST
The authors declare they have no conflicts of interest.
AUTHOR CONTRIBUTIONS
Y-HC and HD conceived and performed the systematic review. EC provided statistical expertise. RB and MF provided a surgical perspective. JM, RA and MF re-reviewed a subset of articles and crossed checked data. HD wrote the manuscript with input from all authors.
FUNDING
This work was supported by
SUPPLEMENTARY MATERIAL
Supplementary data related to this article can be found at
Footnotes
REFERENCES
Cite this article
TY - JOUR AU - Ya-Hua Chim AU - Eva Caamaño-Gutiérrez AU - Rashmi Birla AU - Jillian Madine AU - Mark Field AU - Riaz Akhtar AU - Hannah Angharad Davies PY - 2019 DA - 2019/11/18 TI - Quality of Reporting in Human Aortic Tissue Research – A Systematic Review JO - Artery Research SP - 3 EP - 10 VL - 25 IS - 1-2 SN - 1876-4401 UR - https://doi.org/10.2991/artres.k.191106.003 DO - 10.2991/artres.k.191106.003 ID - Chim2019 ER -