Institutional Research Evaluation Model (IREM): A framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia

Mazen Hassanain; Shirin Anil; Ayman Abdo

doi:10.1016/j.jegh.2016.03.002

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Volume 6, Issue 4, December 2016, Pages 249 - 256

Institutional Research Evaluation Model (IREM): A framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia

Authors

Mazen Hassanain^a^{, b}^,mhassanain@ksu.edu.sa, Shirin Anil^c^{, d}^,shirin.anil@yahoo.co.uk, Ayman Abdo^e^{, *}^,aabdo@ksu.edu.sa

^aDepartment of Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia

^bHubert Department of Global Health, Emory University, USA

^cInstitute of Public Health, College of Medicine & Health Sciences, UAE University, Al Ain, United Arab Emirates

^dPublic Health Solutions Pakistan, Islamabad, Pakistan

^eDepartment of Medicine, College of Medicine, King Saud University, P.O. Box 2925 (59), Riyadh 11472, Saudi Arabia

Peer review under responsibility of Ministry of Health, Saudi Arabia.

^*Corresponding author.

Corresponding Author

Ayman Abdoaabdo@ksu.edu.sa

Received 4 August 2015, Revised 4 February 2016, Accepted 9 March 2016, Available Online 16 April 2016.

DOI: 10.1016/j.jegh.2016.03.002 How to use a DOI?
Keywords: Impact factor; Research impact; Research policy
Abstract: Increased financial and human resource constraints for research and development (R&D) imply rigorous research evaluation to guide the research policy for wise allocation of resources. In this study, we developed a conceptual framework called the “Institutional Research Evaluation Model” (IREM) to evaluate the quality of research and its determinants. The IREM was then applied to a medical institution to study its applicability in Saudi Arabia. The IREM consists of five levels: duration decision; choice of research quality indicators [impact factor (IF), article influence scores (AIS), citations per paper (CPP), and publication in indexed journal]; trend indicators (numbers of publications, study design, subject); data extraction; and statistical techniques to determine the factors affecting impact of research. Application of the IREM to the College of Medicine, King Saud University (CMKSU) for research evaluation from 2003 to 2013 revealed that during this duration, 1722 studies were published, the highest in 2013 (n = 314) and 85.5% (n = 1472) in indexed journals (p < 0.001). The mean IF was 2.6, mean AIS 1.16, and mean CPP 10.06. IF was positively associated with duration, indexation, CPP, and subject being human genetics at multivariable linear regression. The IREM is an applicable basic tool for institutional research evaluation which can guide the research policy.
Copyright: © 2016 Ministry of Health, Saudi Arabia, Production and hosting by Elsevier Ltd.
Open Access: This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

1. Introduction

Research and development (R&D) in the field of medicine supports medical advances to improve the quality of life, as well as life expectancy [1]. This in turn increases the economic development of the nation [1]. Academic institutions are the hub of R&D. In USA, more than 60% of basic research takes place in the universities [2]. The most effective form of research organization is academia, where collaborations and outsourcing can take place for widespread research [2].

R&D requires a lot of financial and human resources. In 2013, about 1.5 trillion USD was spent for R&D globally [2]. The expenditure of R&D as percentage of gross domestic product (GDP) has increased from 2.5% in 2005 to 2.8% in 2012 for USA, which contributed 29% of share in global R&D spending in 2012 [2]. On the contrary, countries like Kingdom of Saudi Arabia spend only 0.058% of their GDP on R&D [3]. According to the Scopus database, approximately 22,338 research documents were produced in the country from 1996 to 2013 with an average of 8.42 citations per document [4]. This makes the ranking of the Kingdom fourth, among the Middle Eastern countries in terms of production of research in the field of medicine [4]. The R&D spending on biomedical research is decreasing in western countries like USA, Canada, and Europe and increasing in Asian countries like China, India, Japan, and Korea [5]. Universities receive 10 times more funding for research as compared to other organizations in USA [2].

The limited resources for R&D require rigorous research evaluation to pave the way forward for research policy so that R&D funds can be utilized effectively for high impact research outcomes [6]. Although research evaluation methods have been devised to compare research outputs at national and international levels [3,7,8], intrainstitutional systematic research quality and trend evaluation models are lacking. In this study, we aimed to develop a conceptual framework to evaluate the trends and impact of organizational research and called it the Institutional Research Evaluation Model (IREM). As mentioned above, academia/universities play a pivotal role in R&D, and we applied this model for research evaluation at the College of Medicine, King Saud University (CMKSU), Saudi Arabia as a case study. Research policy was suggested based on its findings.

2. Materials and methods

2.1. Development of the IREM

Before development of the conceptual framework for the IREM, the goal and objectives were explicitly pronounced for the IREM. The goal of the IREM was to provide institutional research evaluation to guide the research policy for R&D. The objectives were to give a concise and clear concept for the quality assessment of R&D of an institution in quantitative terms. The IREM consists of five levels or steps (Fig. 1):

•
Level 1: make decision about the duration of research evaluation. The researcher can define the time period, for example, 1 year, 2 years, 5 years, or 10 years. If the evaluation is to be done for the first time, it should be done for a longer duration. Evaluation can be done periodically thereafter.
•
Level 2: choose the research quality indicators to be used in research evaluation. There is a range of indicators for assessing the quality of research [9], produced by the institution. Some of these indicators are:
- ○
  Impact factor: Garfield and Sher developed the indicator “impact factor” (IF) in the early 1960s to select the journals for the Science Citation Index (SCI) [10]. IF is defined as the number of citations of publications in the journal in 2 years to the number of published articles in the journal in the same 2 years [10]. IFs are available from the Journal Citation Report (JCR) on the Web of Knowledge and SCI [9].
- ○
  Article influence score: article influence score (AIS) measures the average influence of each of the journal’s article over the first 5 years after its publication [11]. It is also available from the JCR.
- ○
  Cites per paper: citations per paper (CPP) is defined as the number of times or the frequency with which the published article is cited [8]. CPP measures the performance or the impact of individual articles. CPP can be retrieved from Google Scholar. The average CPP can be calculated as the total number of citations for all the papers in a particular duration divided by the number of papers (articles) during the same duration.
- ○
  Publication in indexed journal: publication of an article in the indexed journal increases its visibility and availability of the findings of the study to the global research community [12]. The critical elements for indexation of the journal include its scope and coverage, quality of content, quality of editorial work, production quality, and audience to name a few [13]. The journal’s indexation in MEDLINE can be extracted from the National Library of Medicine (NLM) catalog. This index includes 5625 journals, of which 5080 are those indexed in Index Medicus [14].
•
Level 3: after finalizing the research quality indicators, select the trend indicators. These consist of:
- ○
  Number of publications: this tells about the research output of an institution in terms of frequency of publication per year for the entire duration that has been agreed upon by the researcher at Level 1.
- ○
  Study design: this can range from observational study (case report, case series, cross sectional, case control, cohort) to intervention studies (randomized or nonrandomized) and reviews and meta-analysis [15].
- ○
  Subject of research: subject or the specialty to which the research belongs can be determined to study its trends over time. If the academia is a medical institution for example, the subjects can be the medical specialties like basic sciences, medicine, surgery, and genetics.
•
Level 4: data are extracted at this level. Before extracting the data, indicate the inclusion and exclusion criteria. Various search engines can be used to extract the data; common ones consist of, but are not restricted to, PubMed, Google Scholar, SCOPUS, and Web of Science [16]. The data extraction technique should include institutional affiliation and search by list of authors from that institution, as well maximizing the data mining [17]. Two independent individuals should extract the data to minimize the chances of selection bias.
•
Level 5: the above four levels will lead to data collection on which statistical analyses can now be performed to gauge the quality and trend of research produced by the institution over a specified duration. This can be done with any of the statistical software like SPSS (IBM Corp., Armonk, NY), SAS (SAS Institute Inc., Cary, NC, USA), Stata (StataCorp., TX, USA) and R (R Foundation for Statistical Computing, Vienna, Austria) [18]. Descriptive statistics can be followed by statistical tests to compare means over time, like analysis of variance (ANOVA) and regression analysis to find out the determinants of the research impact of the institution.

2.2. Application of the IREM

The IREM was applied for the research evaluation for the CMKSU, which was founded in 1967 and is one of the oldest universities of Saudi Arabia. It has two teaching hospitals: King Abdul Aziz University hospital became affiliated to it in 1968 and King Khalid University hospital in 1981 [19].

•
Application of Level 1: research evaluation was done for the CMKSU from 2003 to 2013.
•
Application of Level 2: IFs for the journals were retrieved from the JCR 2013 [20], and the SCI [21]. The AIS of each journal was retrieved from the JCR 2013 [20]. CPP were obtained from Google Scholar. Journals’ indexations in MEDLINE were extracted from the NLM catalog [14].
•
Application of Level 3: trend indicators used for research trends of the CMKSU were number of publications, study design, and subject area.
•
Application of Level 4: studies were included if they met the following criteria: (1) published manuscripts by the CMKSU in English language from January 1, 2003 to December 31, 2013, in indexed and unindexed journals; (2) principal authors from the college of medicine and its affiliated hospitals (King Khalid university hospital and King Abdul Aziz university hospital, Riyadh, KSA); (3) if the Principal author was from the college of the KSU other than the college of medicine, then at least one coauthor should be from the college of medicine, KSU; and (4) if the principal author was from the institution other than the KSU, then at least one coauthor should be from the KSU, college of medicine, or its allied hospitals. The following were excluded: (1) books, project reports, manuals, or unpublished manuscripts/thesis documents; (2) conference proceedings in the form of abstracts or posters; (3) manuscripts/books not related to medicine; and (4) principal author and coauthors from the institution other than the CMKSU and its allied hospitals.

PubMed and Google Scholar were chosen as the search engines to retrieve publications in both the indexed and unindexed journals. To maximize the search from the CMKSU, a variety of search phrases like “College of medicine, King Saud University”, “King Abdul Aziz University Hospital, Riyadh”, “King Abdul Aziz hospital”, “King Khalid University Hospital” and “King Khalid Hospital” were used. In the case of authors not having used the above search words, a list of the most published authors at CMKSU was extracted from the Institute for Scientific Information (ISI) report of 2011 (by Thomson Reuters) and their publications searched. Data extraction was performed by two independent individuals to minimize selection bias.

•
Application of Level 5: the data were analyzed in SPSS version 14.0 [22]. Mean and standard deviation were computed for continuous variables like IF, AIS, and CPP. Frequencies and percentages were computed for categorical variables like study design and topic/subject of the study. Trends over the years (from 2003 to 2013) were plotted for both continuous and categorical variables. One way ANOVA was applied to look for the difference of IF, AIS, and CPP from 2003 to 2013 (duration in years taken as a categorical variable as required for this statistical test). The Chi-square test or Fisher exact test (where cell count was <5) was done to look for the difference of categorical variables like indexation of articles in MEDLINE. A p value of less than 0.05 was considered significant for all of the statistical tests performed. Univariable followed by multivariable linear regression modeling was done to determine the factors affecting IF. Variables having p < 0.05 at univariable regression were included in the multivariable model. Regression coefficient β and 95% confidence interval (CI) of β were calculated from the regression model.

3. Results

3.1. Findings from applicability of the IREM to the CMKSU

A total of 3752 articles were reviewed, of which 1722 were included; the description is elaborated in Fig. 2. The highest numbers of studies were published in 2013 (n = 314) and the lowest in 2003 (n = 55); of them, 85.5% (n = 1472) were in indexed journals (p < 0.001).

The IF of the journal was available for 92.6% (n = 1594) studies. The mean IF (standard deviation) was found to be 2.6 (4.1), the minimum being 0.018 for an unindexed journal called “Medical Channel” and the maximum 51.66 for an indexed journal “New England Journal of Medicine (NEJM)”. The highest average IF was found in the year 2011 (an average of 3.9), which was significantly higher than those found from the years 2003–2010 (p < 0.001), but was not significantly different from those found for 2012 and 2013 (Fig. 3). AIS was available for 70.9% (n = 1221) studies. The average AIS was found to be 1.16 (2.23), the minimum 0.02, and the maximum 21.49. The highest average AIS was found for the year 2011 (an average of 1.9), which was significantly higher than that for years 2003–2009 (p < 0.001), but statistically similar to that for years 2010, 2012, and 2013 (Table 1). The average cites per document was 10.06. The highest cites per document was found for the year 2004 (17.32), p < 0.001 (Table 1).

Year	Indexation (n)		Average CPP	Average AIS

	Yes	No
2003	52	3	10.18	0.33
2004	73	4	17.32	0.28
2005	78	8	16.88	0.60
2006	78	7	10.91	0.33
2007	79	6	12.40	0.66
2008	104	13	10.68	0.67
2009	113	27	11.24	0.71
2010	148	42	15.32	1.24
2011	250	38	15.14	1.90
2012	233	52	4.99	1.47
2013	264	50	1.51	1.36
Total	1472	250	10.06^*	1.16^*

AIS = article influence score; CMKSU = College of Medicine at King Saud University; CPP = cites per paper.

*

Average for all the years.

Table 1

Indexation, cite per paper, and article influence score for research produced by College of Medicine, King Saud University from 2003 to 2013.

The highest number of studies were observational in design (76.02%, n = 1309) followed by reviews and meta-analysis (13.94%, n = 240), intervention and experimental studies (9.18%, n = 158), and others including comments, surgical techniques, debates, and editorials (0.87%, n = 15). According to the content of the studies published from 2003 to 2013, the highest number was that of studies related to human genetics (17.8%, n = 307), followed by ophthalmology (7.5%, n = 130), and cardiovascular medicine (5.8%, n = 100) (Fig. 4).

Duration (years taken as continuous variable), indexation of the journal, CPP, study design being interventional study, and subject of human genetics were positively associated with IF, while study design being observational study and papers related to the subjects of medicine, surgery, ophthalmology, obstetrics and gynecology, and public health were negatively associated to IF on univariable linear regression. At multivariable regression, as the duration increased by 1 year, IF increased by 0.27 (95% CI 0.21–0.33); one point increase in CPP raised the IF by 0.068 (95% CI 0.06–0.07); indexation of the journal raised the IF by 1.30 (95% CI 0.69–1.92), and articles having the subject of human genetics increased the IF by 2.78 (95% CI 2.08–3.47), adjusting for study designs in the final model (Table 2).

Variables	Univariable regression		Multivariable regression

	β (95% CI β)	p	β (95% CI β)	p
Year (2003–2013)	+0.29 (+0.22 to +0.35)	0.001^*	+0.27 (+0.21 to +0.33)	0.001^*
CPP	+0.07 (+0.06 to +0.08)	0.001^*	+0.07 (+0.06 to +0.07)	0.001^*
Indexation	+1.99 (+1.25 to +2.66)	0.001^*	+1.30 (+0.69 to +1.92)	0.001^*
Study design
Observational	−0.77 (−1.24 to −0.30)	0.001^*	−0.38 (−0.87 to +0.12)	0.134
Interventional	+1.33 (+0.65 to +2.01)	0.001^*	+0.57 (−0.15 to +1.29)	0.123
Review or meta-analysis	+0.19 (−0.39 to +0.78)	0.513
Others	+0.38 (−1.68 to +2.44)	0.718
Subject
Basic sciences	+0.48 (−0.38 to +1.33)	0.278
Medicine	−0.73 (−1.13 to −0.32)	0.001^*	+0.04 (−0.57 to +0.65)	0.895
Surgery	−1.35 (−1.85 to −0.84)	0.001^*	−0.11 (−0.78 to +0.56)	0.745
ENT	−1.59 (−3.72 to +0.54)	0.143
Ophthalmology	−0.99 (−1.76 to +0.23)	0.011^*	−0.34 (−1.17 to +0.49)	0.426
Obstetrics & gynecology	−1.48 (−2.75 to −0.22)	0.022^*	−0.22 (−1.42 to +0.98)	0.720
Public health	−1.68 (−2.87 to −0.50)	0.005^*	−0.59 (−1.73 to +0.541)	0.304
Human genetics	+3.65 (+3.17 to +4.13)	0.001^*	+2.78 (+2.08 to +3.47)	0.001^*
Radiology	−1.29 (−2.73 to +0.14)	0.077

*

Statistically significant association.

Table 2

Determinants of impact factor at univariable and multivariable regression.

4. Discussion

In this study, the conceptual framework for institutional research evaluation was developed and implemented on one of the academic institutions. We consider this model (IREM) to be specific, as it achieved the objective of evaluation of research quality and trends in quantitative, and thus a subjective manner. The application of the IREM on medical academia revealed that it can measure the research impact of an institution in an effective way, giving an intraorganizational comparison tool.

The research evaluation for the CMKSU gave valuable insights into the research impact of the institution, and will guide its research policy, which was the goal of the IREM. The CMKSU has produced research papers in varied fields from 2003 to 2013, showing an increasing trend over time in terms of the numbers of publications. The average IF of the journal in which the studies were published increased significantly over time. It demonstrates that quality and impact of the medical literature produced by the CMKSU is increasing over time. However, this needs to be improved further as the CMKSU published only 1.3% (n = 22) studies in journals having an IF >15, which is much less than that found by Benamer and Bakoush [23] (83%) for studies published in the Arab world from 2001 to 2005.

The number of times the article is cited is considered as a measure of its impact on the society in general and the specialty from which it arises in particular [24]. It was found that the average CPP for the year 2003–2013 for studies produced by the CMKSU was 10.06, which is more than that found by Meo et al. [3] for all of the studies published in KSA from 1996 to 2011 (cites per document of 7), depicting an increase of 42.8%. It was seen that the average CPP was highest for 2004 and thereafter decreased over time. This may be due to the fact that the CPP of older published studies is more than that of newer studies. It may also indicate that the studies published in 2004 had good quality or important medical/public health issues as compared to the latter ones. However, it should be realized that citation distributions are skewed and their averages thus provide a limitation to its use in determining the impact of the published research [25]. Keeping this in view, the average IF and AIS of the journals in which the studies were published were also used as research quality indicators in the IREM, as these indicators normalize the distribution of citations over time [26]. With time, it has been realized that IFs may also be skewed over varied fields of science and hence weighted indicators to measure research quality, like “impact indicator” have been devised [26]. The IREM provides the basic framework for research evaluation and the choice of research quality indicators depends on the researcher.

The regression analyses revealed that publication of the article in an indexed journal increased the IF. Publication in an indexed journal should be encouraged by the institutional research policy. It was also seen that the interventional study design increased the IF at univariable analysis, emphasizing on such study designs for conduction of future research in the institution. Research related to human genetics also increased the IF significantly, thus the trend of research in genetics should continue at the CMKSU.

The strength of this study is that it not only developed the conceptual framework for research evaluation, but it was also applied to a medical academia as a case study to test its applicability. A number of indicators were utilized to evaluate the quality of research, which reflects its multidimensional approach. The CI of IF was wide for some years (when taken as a categorical variable), which was taken care of by taking it as a continuous variable in the univariable and multivariable regression analysis. Bias due to confounding to determine the factors affecting the research impact was taken care of by multivariable regression analysis. This tool utilized objective assessment for the quality of research. Future research can be done to find its correlation with the perception of the physician/researcher about the research quality of different institutions.

One of the limitations of this study is that the data extraction at level 4 of the IREM might have created selection bias due to different affiliations used by the authors of the research studies, which might have underreported the number of published researches. This bias was addressed by generating the list of authors from the institution and extracting the studies by author name also. Although this tool was designed to evaluate research impacts and trends for an institution, it can also be applied on an individual level to measure the professional progress in the research domain. The IREM could also be applied to compare research impact of Saudi Arabia with other countries. The authors could not do so due to the limited scope of this work. International research impact comparison by application of the IREM is an area for future research.

This study shows that the IREM can be applied for institutional research impact and trend evaluation in a systematic way. This basic framework can further be modified by the choice of indicators like cost effectiveness indicators and statistical techniques according to researcher’s requirement.

Conflicts of interest

All authors declare that they have no competing interest in the publication of this study. No funding was received.

Acknowledgments

SA designed the study, extracted the data, analyzed and interpreted it and drafted the manuscript. MH contributed toward conception of the study and formulating the research questions & objectives and review of the manuscript for its intellectual content. A.A. gave technical assistance for the revision of the manuscript critically for its intellectual content. All authors gave final approval for the manuscript to be published.

References

[1]KM Murphy and RH Topel, Measuring the gains from medical research: an economic approach, University of Chicago Press, Chicago, USA, 2010.

[2]Battelle, 2013 Global R&D Funding Forecast. Available at: <http://www.rdmag.com/sites/rdmag.com/files/GFF2013Final2013_reduced.pdf> [accessed 01.01.2015].

[3]SA Meo, AA Al Masri, AM Usmani, AN Memon, and SZ Zaidi, Impact of GDP, spending on R&D, number of universities and scientific journals on research publications among Asian countries, PLoS One, Vol. 8, 2013, pp. e66449.

[4]SCOPUS, SCImago journal & country rank (1996–2013). Available at: <http://www.scimagojr.com/countryrank.php?area=2700&category=0&region=Middle+East&year=all&order=itp&min=0&min_type=it> [accessed 02.03.2014].

[5]J Chakma, GH Sun, JD Steinberg, SM Sammut, and R Jagsi, Asia’s ascent–global trends in biomedical R&D expenditures, N Eng J Med, Vol. 370, 2014, pp. 3-6.

[6]L Allen, The art of evaluating the impact of medical science, Bull World Health Organ, Vol. 88, 2010. 04–04A

[7]J Allik, Factors affecting bibliometric indicators of scientific quality, Trames, Vol. 17, 2013, pp. 199-214.

[8]Z Li and Y-S Ho, Use of citation per publication as an indicator to evaluate contingent valuation research, Scientometrics, Vol. 75, 2008, pp. 97-110.

[9]MA Joshi, Bibliometric indicators for evaluating the quality of scientific publications, J Contemp Dent Pract, Vol. 15, 2014, pp. 258-62.

[10]E Garfield, Journal impact factor: a brief review, CMAJ, Vol. 161, 1999, pp. 979-80.

[11] Eigenfactor metrics in JCR Web. Available at: <http://wokinfo.com/media/pdf/EigenfactorFAQ.pdf> [accessed 02.03.2014].

[12]U Dhaliwal, N Singh, and A Bhatia, Masters theses from a university medical college: publication in indexed scientific journals, Indian J Ophthalmol, Vol. 58, 2010, pp. 101-4.

[13] Fact Sheet, MEDLINE Journal Selection. Available at: <http://www.nlm.nih.gov/pubs/factsheets/jsel.html> [accessed 02.03.2014].

[14] Number of Titles Currently Indexed for Index Medicus and MEDLINE on PubMed. Available at: <http://www.nlm.nih.gov/bsd/num_titles.html> [accessed 02.03.2014].

[15]M Woodward, Epidemiology: study design and data analysis, 3rd ed., CRC Press, Florida, USA, 2014.

[16]ME Falagas, EI Pitsouni, GA Malietzis, and G Pappas, Comparison of PubMed, Scopus, web of science, and Google scholar: strengths and weaknesses, FASEB J, Vol. 22, 2008, pp. 338-42.

[17]F Peng and A McCallum, Information extraction from research papers using conditional random fields, Inf Process Manage, Vol. 42, 2006, pp. 963-79.

[18]KB Keeling and RJ Pavur, A comparative study of the reliability of nine statistical software packages, Comput Stat Data Anal, Vol. 51, 2007, pp. 3811-31.

[19] Faculty of medicine. Available at: <http://medicine.ksu.edu.sa/index.php?lang=ar> [accessed 12.25.2013].

[20] Thomson Reuters Research Analytics Unveils 2013 Release of Its Journal Citation Reports. Available at: <http://thomsonreuters.com/press-releases/062013/2013-journal-citation-reports> [accessed 01.01.2014].

[21] SJR–SCImago Journal & Country Rank. Available at: <http://www.scimagojr.com/> [accessed 01.01.2014].

[22]IBM, SPSS, 14.0.0 edition, SPSS Inc, Chicago, 2005.

[23]HTS Benamer and O Bakoush, Arab nations lagging behind other Middle Eastern countries in biomedical research: a comparative study, BMC Med Res Methodol, Vol. 9, 2009, pp. 26.

[24]BA Jamjoom, AA Jamjoom, and AB Jamjoom, The most cited articles in the Saudi medical literature, Saudi Med J, Vol. 33, 2012, pp. 93-5.

[25]L Leydesdorff, L Bornmann, R Mutz, and T Opthof, Turning the tables on citation analysis one more time: principles for comparing sets of documents, J Am Soc Inf Sci Technol, Vol. 62, 2011, pp. 1370-81.

[26]L Leydesdorff, Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers, Scientometrics, Vol. 92, 2012, pp. 355-65.

<Previous Article In Issue

Download article (PDF)

Next Article In Issue>

Journal: Journal of Epidemiology and Global Health
Volume-Issue: 6 - 4
Pages: 249 - 256
Publication Date: 2016/04/16
ISSN (Online): 2210-6014
ISSN (Print): 2210-6006
DOI: 10.1016/j.jegh.2016.03.002 How to use a DOI?
Open Access: This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Cite this article

ris enw bib

TY  - JOUR
AU  - Mazen Hassanain
AU  - Shirin Anil
AU  - Ayman Abdo
PY  - 2016
DA  - 2016/04/16
TI  - Institutional Research Evaluation Model (IREM): A framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia
JO  - Journal of Epidemiology and Global Health
SP  - 249
EP  - 256
VL  - 6
IS  - 4
SN  - 2210-6014
UR  - https://doi.org/10.1016/j.jegh.2016.03.002
DO  - 10.1016/j.jegh.2016.03.002
ID  - Hassanain2016
ER  -

download .riscopy to clipboard