Evaluation of the certificate in emerging infectious disease research and the certificate in one health training programs, University of Florida
- DOI
- 10.1016/j.jegh.2014.10.004How to use a DOI?
- Keywords
- Education; Zoonosis; Epidemiology; Environmental health; Entomology; Food safety
- Abstract
In developing countries, public health professionals and scientists need targeted training and practical skills to respond to global emerging infectious disease threats. The Certificate in Emerging Infectious Disease Research was developed in 2008 to aid such professionals to respond to complex emerging disease problems. The short-course was modified slightly in 2013 and renamed the Certificate in One Health. To evaluate the immediate impact of the short-course, an online survey of 176 past participants from both the courses was conducted. The survey tool assessed the program’s process, impact, and outcome measures respectively via assessing the courses’ perceived strengths and weaknesses, perceived skills gained, and the participants’ current position, publication status, funding status, and educational attainment; 85 (48.3%) participants completed the survey. Reported program strengths included the curriculum, expertise of lecturers, and diversity of the training cohort. The principal reported weakness was the compressed academic schedule. The most frequently reported benefits included: epidemiological and biostatistical skills, followed by One-Health knowledge, and research skills. Twenty-eight percent of the survey respondents reported publishing one or more manuscripts since completing the course and 21% reported receiving research funding. The course appears to have had a positive, immediate impact on the students’ self-perceived knowledge and capabilities.
- Copyright
- © 2014 Ministry of Health, Saudi Arabia. Published by Elsevier Ltd.
- Open Access
- This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
As globalization spurs the international transport of people and goods, and countries expand their human resource base, supply chains and business models, more opportunities arise for the rapid emergence and transfer of infectious diseases across once-separated geographic regions. Important health problems that have already moved quickly across continents include multi-drug resistant bacteria [1,2] and novel zoonotic viruses [1,3]. To address these and other emerging public health concerns, there is a growing need to provide interdisciplinary training across international borders [4]. The Department of Environmental and Global Health at the University of Florida (UF) is helping to address these ongoing public health threats through the Certificate in One Health (COH) program. In the long-term, focused didactic and practical training programs, such as the COH may lead to improved national and regional planning and preparedness for public health emergencies, as well as more efficient and effective responses to emerging health threats. To evaluate the near-term impact of the “short-course”, a mixed-methods educational and training model for mid-career professionals from developing countries, members from the UF One Health Center of Excellence (OHCE) conducted an online, mixed quantitative and qualitative survey of the past certificate program participants, including those who did not complete all of the course modules. The survey team sought to better understand the perceptions of the course, including the training environment and pedagogical approach, in an effort to evaluate the process or delivery of the course, as well as gauge course impacts via skills gained and outcomes via a myriad of factors. As no pre-tests had been administered to the previous classes, a quantitative impact evaluation in terms of assessing pre- and post-training knowledge could not be performed.
1.1. Program origin
The COH initially started in 2008 as the Certificate in Emerging Infectious Disease Research (CEIDR) at the University of Iowa’s College of Public Health. In 2010 it was moved to the University of Florida. Over the years, the certificate program has received combined support from the Department of Defense’s (DoD) Global Emerging Infections Surveillance and Response System (GEIS), the Department of State’s (DoS) Biological Engagement Program (BEP), the US Centers for Disease Control and Prevention (CDC), and from self-funded individual trainees.
The program sought to engage and train mid-level international public health scientists, especially public health and laboratory professionals, to be better prepared to respond to future emerging disease threats. The program was designed to involve a relatively brief commitment for face-to-face didactic time through an intensive review of current laboratory and epidemiologic methods. The focused training for capable scholars did not demand long-term absence from their home countries and after the trainees returned home, they continued their training through additional online courses that were required for the students to receive the certificate.
To achieve a multidisciplinary approach to the understanding of emerging infectious diseases, the program evolved to align with educational and operational goals of the One Health Initiative [5]. In 2013, the certificate was officially renamed the Certificate in One Health and a major change in program format was accomplished by removing the course, Infectious Disease Epidemiology and replacing it with An Introduction to One Health Problem Solving.
1.2. Pedagogic methods and objectives
The COH is an annual (summer), 18-day (6 credit hour) intensive period on campus at UF with mixed teaching methods including didactics, small group problem solving, laboratory instruction, practical exercises, field work and discussions regarding One Health-related videos. Following the residential portion of the course, the students must complete a three-credit online course in environmental health during the concurrent summer semester. Students must complete all modules with an average grade of 3.0 on a 4.0-scale to receive their certificate.
Each class typically consists of 40–50 people, most of whom are nominated by the DoD or DoS. The nominees are contacted individually by COH program administrators and are asked to provide additional information in order to evaluate their readiness for a US-based academic program, including English language fluency. An independent selection committee at UF reviews the students’ information and selects those who are best qualified. The funding support includes tuition scholarships and travel funding; UF administrators assist students with requesting travel visas and other logistical arrangements. Self- or independently funded students who meet the basic qualifications are allowed to enroll if there are unfilled seats.
1.3. Literature review of similar programs
A web-based, non-systematic search for other educational programs suggested that the CEIDR/COH is unique in its approach in enhancing the capabilities of the global community of scientific professionals. The search identified training programs for emerging infectious diseases, but some programs required long-term absence and the possession of US citizenship [6]; others specialized in specific areas of emerging infectious diseases, such as tropical medicine and traveler’s health [7], or HIV/AIDS [4]. A review investigating postgraduate infectious disease training programs identified 26 countries that had established programs; however, most programs required a medical degree and the program lengths varied from 2 to 6 years [8]. A program more similar in nature to the CEIDR/COH was the Field Epidemiology Training Program (FETP). This program, supported by the US CDC, works with the foreign ministries of health to assist in building epidemiological training programs modeled after the US CDC’s Epidemic Intelligence Service [9]. In addition to training the individuals in field epidemiology, the program aims to have trainees use the new knowledge to shape policy and thus, strengthen international public health systems [9]. However, unlike the COH, the actual training occurs within the foreign country, often requiring the long-term commitment of an accomplished epidemiologist to provide training.
2. Methods
2.1. Survey goals
The intent of the certificate survey was to determine if the CEIDR and COH programs met their immediate goals to impart knowledge and skills needed to better address emerging infectious disease threats. This survey aimed at determining whether or not alumni furthered their education, enhanced their collaborations with other alumni, garnered project funding, or made scientific contributions via publications. The survey also aimed to determine the strengths and weaknesses of the program and skills gained by the program participants.
2.2. Survey construction
The certificate survey was created from a parent survey used to assess all Environmental and Global Health graduate programs: as individuals answered the survey questions, the use of skip logic or contingency questions tailored the survey for the evaluation of a particular program. To assess the CEIDR/COH programs, members of the OHCE constructed survey questions to assess the following: program type and year completed, employer and position information, contributions to the field of public health in terms of publications or funding/grants received, collaborations acquired due to the program, strengths and weaknesses of the program, and pursuit of higher education post-program (Fig. 1). The survey was created via Word Press software and was embedded on the UF Department of Environmental and Global Health website.
Survey questions for CEIDR and COH alumni.
2.3. Survey data
Survey questions followed either a free-response or multiple-choice format (Fig. 1). Questions 1–6 asked about basic identifying information and program classifications. Thereafter, survey questions extracted a mixture of both qualitative and quantitative data. Questions 9, 10, 12, and 13 yielded descriptive, qualitative data that was then categorized into discreet values for analysis. For the highly diverse and lengthy responses obtained from question 13 (What skills did the program provide to help you in your current position?), a word cloud was created with Wordle™ software (http://www.wordle.net/). The software arranged the words following a proprietary algorithm that increased the font size of each word according to the frequency with which it appeared in the concatenated responses. Since the responses were free response, pronouns, articles, and conjunctions were manually removed and the software option in Wordle™ was selected that removed common English words. With this method of “visualizing” the responses, adjectives, nouns and verbs were highlighted that imparted descriptive information regarding the program.
2.4. Subject selection and survey dissemination
Contact information for program participants from 2008 to 2013 was compiled from enrollment spreadsheets; 176 past participants were sent emails on 9/13/2013 inviting them to complete the survey on the UF website. Past participants were asked to complete the survey by 9/23/2013. A follow-up email was sent to all members on 9/23/2013 and 9/24/2013. A message was posted on the 2013 cohort’s Facebook page to encourage additional responses. A number of participants in the short-course were known (in advance of the survey) to have received additional (formal) education and training, but no distinction was made between those who did and did not pursue additional degrees.
3. Results
3.1. Program statistics
As of 10/7/2013, 85 (48%) CEIDR/COH participants completed the survey. Nine survey respondents (11%) reported that they were unable to complete the certificate program (question 7). Of those nine, six reported time constraints, four reported an inability to access online course materials and one person reported losing their password as factors inhibiting program completion (participants were able to choose all that applied).
Starting in 2010, data were collected on program completion rates. The completion rate for program participants for individual years is as follows: 2010 (72%), 2011 (77%), 2012 (47%), and 2013 (64%). The overall program completion rate for the 2010–2013 programs is 68% (82 out of 121 participants); 89% of those completing the survey had successfully received either the CEIDR or COH.
3.2. Current career status
Thirty-two percent of trainees had a master’s degree, 19% a bachelor’s degree, 15% a medical degree (MD, DO, DVM), 9% a PhD, and 7% reported having ‘other’ degrees. Out of the six respondents who reported ‘other,’ five noted that the certificate was their highest degree. Because this question was added after the survey was open for responses, 15 respondents (18%) had already submitted their responses and their educational status remains unknown.
In terms of professional placement (question 12), these responses were qualitative responses that were classified into various categories. Survey results revealed that 21% of alumni have administrative roles, 20% work in a research capacity, 12% engage in management/consultation positions, 8% in academia and 8% as public health officers or specialists. Positions cited in less than 8% of respondents included the following: graduate student (7%), program coordinator (7%), epidemiologist (6%), and clinical work (5%).
3.3. Program strengths, weaknesses, and skills gained
Strengths and weaknesses of the program (questions 9 and 10) were captured using free-response questions that were later classified into discreet values (Supplementary Tables S1 and S2, respectively). For strengths and weaknesses, program alumni were asked to list three strengths and three weaknesses of the program. Several participants did not report weaknesses.
The most frequently reported strengths of the CEIDR/COH program included: curriculum/content (21.4%), instructors/teaching methods (20.4%), international diversity (17.3%), field work (9.7%), and online learning (8.7%) (Table 1). The most frequently reported program weaknesses included a compressed schedule (46.9%), online courses (14.3%), lack of follow-up (13.3%), content difficulty (8.2%), and logistics/travel (6.1%) (Table 2).
| Coded response | Total times listed (%) |
|---|---|
| Curriculum content | 42 (21.4) |
| Instructors/methods | 40 (20.4) |
| International diversity and networking | 34 (17.3) |
| Field work | 19 (9.7) |
| Online learning | 17 (8.7) |
| Laboratory practicums | 15 (7.7) |
| University system/facilities | 14 (7.1) |
| Logistics/coordination/hospitality | 9 (4.6) |
| Library/references | 6 (3.1) |
CEIDR/COH program strengths as listed by past program participants (survey question 9). Total listed responses (n = 196).
| Coded response | Total times listed (%) |
|---|---|
| Compressed schedule | 46 (46.9) |
| Online courses | 14 (14.3) |
| No follow-up | 13 (13.3) |
| Content difficulty level | 8 (8.2) |
| Logistics/travel | 6 (6.1) |
| Content limited | 3 (3.1) |
| Certificate not valued | 2 (2) |
| Lodging/food | 2 (2) |
| Weak instructors | 2 (2) |
| Language barriers | 1 (1) |
| Auto-visual quality | 1 (1) |
CEIDR/COH program weaknesses as listed by past program participants (survey question 10). Total listed responses (n = 98).
The survey also aimed to assess the skills gained by program participants (question 13). This free response question garnered a great diversity of data. The responses were coded into 10 categories (Supplementary Table S3). The word cloud (Fig. 2) complemented these data and identified frequent descriptive words such as epidemiology, diseases, knowledge, health, laboratory, and biostatistics.
Word cloud created using Wordle™ software (http://www.wordle.net/) and text from survey question 13: ‘What skills did the program provide to help you in your current position?’.
Out of the 151 skills listed, the top five skills gained from the program fell under the following categories: quantitative skills (30%), One Health and emerging infectious disease knowledge (21%), research and critical-thinking skills (15%), laboratory techniques (10%), and environmental health knowledge (9%) (Table 3).
| Coded response | Total times listed (%) |
|---|---|
| Quantitative | 45 (30) |
| One health/emerging infectious diseases | 31 (21) |
| Science/research/critical-thinking skills | 23 (15) |
| Laboratory techniques | 15 (10) |
| Environmental health | 13 (9) |
| Other | 7 (5) |
| Communication skills | 5 (3) |
| Public health | 4 (3) |
| Field work techniques | 4 (3) |
| Academic skills | 4 (3) |
CEIDR/COH program skills gained as listed by past program participants (survey question 13). Total listed responses (n = 151).
3.4. Program connections/networking
For question 14, in which respondents could choose more than one reply, 75% indicated that an email list serve was a good way for them to remain connected with fellow alumni. Other selected options included an alumni website (71%) and Facebook (62%). In the free-text field, respondents also suggested an online discussion or chat group, an alumni association, or using Linked-In or Twitter.
Question 15 assessed whether program alumni kept in-touch and if so, via which method(s). Question 16 determined if these connections led to opportunities or collaborations. The majority of survey respondents (93%) reported still staying in contact with others. Respondents reported using multiple methods of communication including Facebook (49%), E-mail (30%), Skype or Viber (6%), and 4% reported staying in contact through attending conferences or meetings. Three percent of respondents reported also using the following methods: telecommunications, in-person meetings, messenger applications such as iMessage or WhatsApp, and LinkedIn. Fifteen percent of those staying in-touch reported that these connections led to new opportunities or collaborations (question 16).
3.5. Alumni career advancements
Questions 17, 18, and 19 inquired if program alumni had received funding, published any manuscripts, or advanced their education after completing the certificate program, respectively. Twenty-one percent of respondents reported receiving funding, 11% reported that they applied for funding and 68% reported not having applied for funding. Twenty-eight percent of respondents reported 1 or more publications, 7% reported having manuscripts under review at the time of survey administration, and 65% reported having no publications. In regard to pursuit of higher education since program completion, 28 (33%) alumni reported enrollment or graduation from an advanced degree program, 13 (15%) reported they were planning on pursuing higher education, while 30 (35%) reported no plans to pursue higher education and 14 (16%) had no plans to pursue advanced education because they had a terminal degree. Of note however, 7 individuals not in pursuit of higher education were unable to answer whether or not they had a terminal degree because that question was added to the survey after they responded. Seven percent of program participants in a degree program reported that the credits from the CEIDR/COH transferred toward their advanced degree.
4. Discussion
To sustain the short-term effects of the COH abroad, overarching goals of the program are to create the seeds for self-sustaining academic programs among the partners and develop well-trained and broad-thinking professionals who become public health and veterinary health leaders. The program also aims to help these individuals build an academic track record so that COH alumni may compete and win scholarships to further advance their education (MPH, MS, PhD). The COH also offers a unique opportunity for students to foster relationships between other international professionals and in doing so, construct a network of international professional colleagues with whom to partner in future scientific collaborations.
After six years, alumni were surveyed to assess several aspects of the CEIDR/COH program, including its strengths, weaknesses, skills gained by participants, levels of collaboration and networking between alumni, and the program’s impact level on advancing the careers of program alumni. This information was sought to strengthen the program by addressing weaknesses while also creating a baseline level of information from which to compare future performance metrics.
A core target area of the survey was to determine strengths and ways to assess and improve the program process. As curriculum, instructors/teaching methods and international diversity served as the top three strengths, it was interpreted that this was a strong group of strengths since these attributes underlie all academic activities and interactions from the start to the finish of the program.
When analyzing program weaknesses, 47% of program alumni identified that the compressed time schedule (∼18 days) served as the number one weakness of the program. By falling under the coded term, ‘compressed schedule,’ several students referred to either an insufficient amount of time allocated to each topic or the very condensed schedule in general, or too short of a period at UF. However, the program aims to train international professionals without demanding their long-term absence, while providing many online and text resources, in an effort to return alumni home where they can immediately apply their skills and knowledge towards disease-fighting efforts in their countries. In a future survey, an additional question could be added asking respondents for suggestions or changes they would make to improve the time scheduling.
Program participants identified that online courses served as the second greatest weakness. Further investigation revealed a diversity of responses ranging from the report of too many assignments, tight deadlines, issues with the discussion due to international time zone variation, and report of losing access to online course information from loss of password. Like the online course weaknesses, the reported weakness of ‘no follow-up’ included varying responses citing no institutional links with candidate countries, issues with administrative follow-up, losing interaction with UF post-program, and limited sponsorship in terms of pursuing further study at UF. However, the OHCE at UF is in the process of forming international collaborations with many research institutions where some of the trainees study and/or perform research; additionally, the Department of Environmental and Global Health does offer tuition scholarships for successful applicants for the PhD in Public Health with a One Health concentration. The OHCE is looking into ways to enhance follow-up and connections with certificate alumni through a website with a message board, and links to One Health-related events at UF and on the international level.
Program participants reported that quantitative skills, laboratory skills, One Health and emerging infectious disease knowledge, coupled with research and critical thinking abilities as the most frequently acquired skills from the program. As the program was transformed into the COH in 2013, both One Health and emerging infectious disease knowledge was categorized into one variable. Quantitative, laboratory, and research skills underlie the fundamentals of One Health research; thus, reported strengths in these areas supports the COH’s goal of facilitation of One Health-related research in international countries. To quantitatively evaluate program impact in future classes, a pre- and post-training knowledge assessment would serve as a more accurate indicator of program process.
To measure outcomes, such as academic and career success post-program, the survey assessed whether participants had advanced their education, received funding, or published any manuscripts. It appears that the majority of respondents pursued an advanced degree, or was planning on pursuing an advanced degree, or already had a terminal degree at the time of the survey (64%). In terms of publications, about 28% had subsequently published scientific manuscripts and 21% had received funding after training. A follow-up survey to assess publication status and receipt of funding in comparison to baseline data would indicate if program participants are advancing in their careers over the years.
In terms of networking and collaboration, most program participants reported staying in contact with each other and 15% of those in contact reported that these connections led to new collaborations. Barriers to collaboration may be due to distances or regulations between their countries. In future surveys, to elucidate the nature of the collaborations, emphasis could be placed on the question inquiring about the type of research collaborations that transpired and add questions inquiring if the collaborations led to new funding or publications as well as query about ways to facilitate these types of collaborations. Also, if respondents reported ‘no’ for collaborations, another question may be added inquiring about the types of barriers participants encountered if they attempted to collaborate.
To continue to evaluate and measure program success, it is planned to administer an improved version of this survey for each new COH class. Additional questions are planned to be added to assess and improve post-program collaboration, gain information on desired skills, and determine if any future program changes will influence program strengths or weaknesses. Additionally, to better assess program impact, a pre- and post-training knowledge test will be considered. As for program participants that have taken the initial survey, a curtailed survey is planned to be constructed to measure their professional development over time in regard to education, publication status, receipt of funding, and specifics regarding collaborations with other program alumni.
5. Conclusion
As emerging pathogens continue to surface and ignore international borders, the need to train and educate these global partners in One Health principles and their application to emerging infectious diseases is essential for public health and veterinary health. To contribute to international collaboration and the control and prevention of emerging pathogens, the COH/CEIDR aims to support this ideal by continuing in its seventh consecutive year with 175 alumni from over 30 countries. The survey demonstrated that a majority of graduates have advanced their careers educationally and about a third have contributed publications to the field, received funding or applied for funding for ongoing research. The COH program intends to improve its program with the survey results in order to continually support the need for international collaboration in the response efforts against emerging pathogens.
Acknowledgements
This research was supported by multiple grants from the US Department of Defense Armed Forces Health Surveillance Center’s Global Emerging Infections Surveillance and Response Program (Dr. Gray, principal investigator). This publication was also funded in part by a grant from the United States Department of State and facilitated by CRDF Global. The opinions, findings and conclusions stated herein are those of the author(s) and do not necessarily reflect those of the United States Department of State or CRDF.
Appendix A. Supplementary data
Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.jegh.2014.10.004.
References
Cite this article
TY - JOUR AU - Marissa A. Valentine AU - Christopher L. Perdue AU - James F. Cummings AU - Jacqueline C. Smith AU - Gregory C. Gray PY - 2014 DA - 2014/11/24 TI - Evaluation of the certificate in emerging infectious disease research and the certificate in one health training programs, University of Florida JO - Journal of Epidemiology and Global Health SP - 23 EP - 31 VL - 5 IS - 1 SN - 2210-6014 UR - https://doi.org/10.1016/j.jegh.2014.10.004 DO - 10.1016/j.jegh.2014.10.004 ID - Valentine2014 ER -