2.1. Inclusion criteria

The following types of studies were included in the review:

• •

  Randomized controlled trial

• •

  Quasi-experimental study

  • –

    Non-randomized controlled study

  • –

    Before-and-after study

• •

  Observational study

  • –

    Cohort study

  • –

    Case-control study

• •

  Outbreak reports

  • –

    Outbreak/cluster reports

  • –

    Historical archival outbreak analyses

• •

  Surveillance reports

  • –

    Major event infection surveillance reports

The list above is largely in hierarchical order of study design quality [9], but with the addition of outbreak and surveillance reports, not traditionally included in such lists, important to the topic of this review. To cover the other “non-standard” study design encountered, the term “quasi-experimental” is employed to describe controlled but non-randomized studies [10,11].

Only studies published in English were included. Relevant systematic and narrative reviews and operational description papers were utilized for useful background information. The reference lists of the systematic reviews were scanned to identify potentially relevant primary studies that could be considered for inclusion. Case reports, mathematical modelling and human/non-human experimental laboratory studies were excluded from the review.

Outcome measures included laboratory-confirmed influenza infection (i.e., documented by virus isolation, molecular testing such as polymerase chain reaction, and serological studies) and clinical influenza-like illness as defined by the investigators, whilst recognizing that this is less specific.

2.2. Search strategy

The search strategy focused on primary studies taking into account the issues detailed above. The term ‘influenza’ rather than ‘pandemic influenza’ was used to reflect the entire spectrum of influenza for which guidance would be relevant. The transmission of other respiratory viruses was also included because of the similarity of transmission and therefore the potential applicability of any results to influenza.

An initial “scoping” search was followed by further discussions and consultation on the work plan and search strategy. The term mass gathering on its own proved inadequate as a search term as there is no clearly accepted definition of what constitutes a mass gathering. Therefore, a range of additional terms were used, such as public gatherings, social gatherings, large crowds, mass events, festivals, Olympics, Hajj, championship, and others listed in Table 1. Transit settings such as long-distance flights and cruise ships were not included, as it was decided that these settings would fit better in a separate review of evidence for influenza transmission through transport. The final search with revised terms and refined using the “advanced search” tool on PubMed was carried out as described in Table 1, with no time period restrictions. The initial searches included the literature up till July 2010. A further search was conducted on 31 May 2011 to bring the literature coverage up to date as part of an editorial process.

In addition, secondary searches were also initially carried out in the following databases: Scopus, Excerpta Medica Database (EMBASE), and the Cumulative Index to Nursing and Allied Health Literature (CINAHL). These are described in Table 2.

2.3. Other sources

In addition to the search results, the reference lists of papers identified from the search were scanned for other potentially relevant studies. Efforts were also made to identify studies other than those published in the peer-reviewed literature; the authors’ hardcopy literature files were hand-searched to identify additional published articles or other documents of relevance.

2.4. Study selection and data extraction

Study selection was conducted in stages. At all stages, scanning and reviews were done independently by the authors, who then harmonized their selections. Papers identified by the searches were scanned and some excluded on the basis of the ‘title’ for relevance to the review. The abstracts of the remaining papers were then reviewed to identify studies that appeared to meet the inclusion criteria. The full text versions of all these articles were then sought and read in full. For papers that had relevant titles but no abstracts, the full text articles were also obtained, scanned for relevance, and if relevant they were read in full. In addition, some papers that did not meet the inclusion criteria (e.g. operational descriptions of interventions, commentaries, or editorial reviews of influenza transmission or pandemic influenza) were selected to provide relevant background or supplemental information.

Data from the selected full text papers were extracted using a pre-designed form. Data elements included publication information, study characteristics, participant characteristics, the intervention (if any) and setting, outcome and results. The data extracted were used to determine the eligibility of each paper for inclusion in the review. This process resulted in a final selection of eligible articles that were then included in the review.

2.5. Temporal and situational context of the review

The main body of this work was undertaken in 2010 by the UK Health Protection Agency as part of a comprehensive influenza scientific evidence review informing the UK National Pandemic Influenza Strategy. Within this context, the work was carried out within a very tight three-month timeframe and with significantly limited resources. As a result, some of the most comprehensive steps required of a full systematic review could not be realistically accommodated. Therefore, a modified process was adopted in conducting this “rapid evidence assessment” rather than a full, formal systematic review. Rapid evidence assessments are well recognized as a pragmatic approach to include all the major elements of a full systematic review, ensuring an outcome that is as comprehensive as possible while accepting limited compromises in order to meet stipulated deadlines for feeding into the policy process [9].

Due to the time constraints, it was not practical to pursue time-intensive contacts with authors or to seek translations for non-English articles, which were therefore excluded. Articles lacking abstracts were largely excluded, and there was a limited search for unpublished studies. However, to try to mitigate these limitations, full draft versions of the review were submitted to national experts at the UK Health Protection Agency and the Scientific Pandemic Influenza committee of the UK Department of Health, seeking their comments and advice on any papers or documents that may have been omitted. Some further work was carried out in May to June 2011 to update the literature.

2.6. Quality assessment

At the outset, a strategy was developed to assess the quality of eligible studies using the Critical Appraisal Skills Programme (CASP) tools as appropriate for each type of study [12]. However, after completion of the literature search, it was clear that the types of studies and the study design types identified were not suitable for assessment by the CASP tools. Similarly for the outbreak reports included in the review, no suitable existing framework could be readily identified for quality assessment. The ORION tool was not strictly applicable as it is specific to outbreaks of nosocomial infection [13,14]. Therefore, a modified approach was adopted, whereby each paper was categorized as having low risk, some risk, or high risk for bias with respect to the relevant review question addressed by that study. The grading was assessed based on the presence of significant methodological limitations. A high risk of bias was attributed to papers with at least three significant methodological issues as identified by the review authors, while papers with at least two issues were classified as having some risk. This approach was adapted from a system previously used by Jefferson et al. [7].

The quality of the review itself was tested against checklists based on the MOOSE [15] and PRISMA [16,17] standards. Within the restrictions of the time constraints already described, key gaps and issues identified by the checklists were addressed as appropriate and/or acknowledged in Sections 2.5 (explaining the review context including time constraints) and 4.4 (recognizing the study limitations).

2.7. Data synthesis

The data synthesis was restricted to a narrative approach that included an analysis of the relationships within and between studies and an overall assessment of the robustness of the evidence and limitations of both the studies and the evidence review [9]. In addition, the synthesis considered the implications for policy and guidance development as well as future research.

3.1. Quasi-experimental study

One quasi-experimental study by Qureshi et al. [18] attempted to investigate the incidence of vaccine preventable influenza-like illness among Pakistani pilgrims to the Hajj religious gathering in 1999. The Hajj is an annual religious event that takes place over a number of days in a very small geographic area of Saudi Arabia usually involving 2 to 3 million pilgrims from all over the world. Accommodation is at a premium during this event and many pilgrims stay in tents specifically erected for the event and that are often over-crowded. Although primarily a vaccine efficacy study, the rates of influenza-like illness reported in vaccinated pilgrims were 36% compared with 62% in non-vaccinated pilgrims. However, these results were based on clinical endpoints without microbiologic confirmation; a non-randomized design was used, and the study was not designed to address the primary question of this review.

3.2. Observational studies

Nine observational studies estimated the risk of acute respiratory illness and/or influenza-like illness associated with the Hajj pilgrimage by attempting to measure its occurrence in pilgrims. Four [23–25,27] of these studies confirmed the cause of illness by laboratory testing, while the other five relied on specified symptom complexes as surrogate indicators.

Among the nine observational studies were four cross-sectional studies. Balkhy and colleagues [24] tested 500 Hajj pilgrims in 2003 who presented with symptoms of upper respiratory tract infection in the second week of the event; 30 (6%) of the 500 pilgrims tested positive for influenza. Rashid et al. [23] assessed the burden of laboratory-confirmed influenza and respiratory syncytial virus (RSV) infections in symptomatic British Hajj pilgrims in the 2005 event. Of 202 symptomatic pilgrims who underwent nasal swab testing, 28 (about 14%) had confirmed influenza (mostly A type), while only 9 (4%) had RSV infection. In the 2006 Hajj, the same investigators [27] found comparable levels of laboratory confirmed influenza (10–11%) in both UK and Saudi pilgrims. These relatively low levels of infection contrasted with the findings in the cross-sectional study by Deris et al. [20] who based their assessment on syndromic influenza-like illness rather than laboratory-confirmed infection. They found an influenza-like illness prevalence of 40% in Malaysian pilgrims who had just completed the Hajj.

The other five observational studies were similarly designed, involving groups of intending Hajj pilgrims who were recruited in their home regions or countries prior to the event, and then re-assessed after the pilgrimage. Even though three reports were described by their respective authors as “prospective cohort” studies [19,21,22], none of these five studies included an “unexposed” (non-Hajj attending) control group for comparison, indicating that they should be more accurately regarded as “before and after” studies. Choudhry et al. [19] assessed Saudi residents (attending the Hajj from a different part of the country) and found an incidence of influenza-like illness of about 40%. Three studies of French pilgrims by Gautret et al. [21,22,26] found rates of cough of between 48% and 61%; while in El-Bashir and colleagues’ serological study of UK pilgrims [25], respiratory symptoms occurred in more than 80%, but the seroconversion rate was 38% of 115 participants.

These variable data underline the difficulties of comparing different studies that employ contrasting measures of influenza. Interpretation needs to take account of the fact that studies using clinical outcomes suggest much higher levels of influenza-like conditions than the ones that involve laboratory confirmation. Even among the laboratory-based studies, differences in laboratory techniques may also be significant.

3.3. Outbreak reports

There were four types of outbreaks in the reports that were reviewed. The first of those was a paper by Pang et al. [32] that described the experience of dealing with the SARS (severe acute respiratory syndrome) outbreak of 2003 in Beijing and attempted to evaluate the control measures that were deployed. However, there was no reliable indication of the impact of mass gathering restrictions on controlling the outbreak as mass gatherings were grouped together with a number of measures loosely described as social distancing.

The next group consists of four reports of influenza outbreaks occurring at a religious event in Australia and at three large, open-air music festivals in Europe. All of the events lasted several days and involved crowds ranging from 100,000 to 400,000 people. It is particularly relevant that the three music festivals occurred during the 2009 influenza pandemic—two at the beginning and one later during the pandemic.

The report by Blyth et al. [28] described an outbreak of influenza at a large, 5-day religious event in Australia during July 2008 attended by over 400,000 participants from 170 countries. Over 100,000 of the pilgrims were accommodated in a variety of make-shift, overcrowded venues such as sports halls, community centers and schools. One hundred laboratory-confirmed cases of influenza were identified among attendees. Seven different strains of influenza were identified (four influenza A and three influenza B), highlighting the potential for the introduction of novel influenza strains.

Loncarevic et al. [31] described an outbreak of influenza at a 4-day music festival in Serbia during July 2009 involving over 190,000 participants, with a number of them coming from other European countries. Many of the participants stayed at a large campsite where overcrowding was an issue. Sixty-two laboratory-confirmed cases of H1N1 (2009) were identified; some of which were secondary cases. Although the virus was already present in Serbia at the time of the festival, the sudden increase in cases, in particular the secondary cases associated with the festival, suggests possible local spreading. The authors also reported on a small outbreak of influenza at an international sporting event held in Serbia in July over a period of 12 days and attended by over 500,000 spectators. Seven confirmed cases of H1N1 (2009) were identified in six athletes and a volunteer helping at the games. Although a much larger event, the numbers affected are considerably smaller and seem to be restricted to participants rather than those attending.

Gutierrez et al. [30] described a 4-day music festival in Belgium during early July 2009 attended by an estimated 120,000 people from all over Europe. Twelve laboratory-confirmed cases of H1N1 (2009) were identified. Although sporadic cases of H1N1 (2009) had been detected in Belgium prior to the festival, an increase in cases was observed after the event and the decision to shift to mitigation was taken almost a week after the first festival-associated cases were identified. The authors suggest that this festival highlights the potential seeding role for these events in the early stages of a pandemic.

Botelho-Nevers et al. [29] depict an outbreak of influenza at a 7-day music festival in Hungary during August 2009 attended by 390,000 people from all over Europe. Many of the participants were located on a campsite set up for the festival. Eight laboratory-confirmed cases of H1N1 (2009) were identified. Cases of pandemic influenza had already been identified in Hungary and at the time the community influenza-like illness rate was 7.8 per 100,000 against an estimated 3.6 per 100,000 at the music festival.

The third type of outbreak report described by Saenz et al. [33] involved a large, international medical conference held in Iran during September 1968 at the early stages of the 1968/1969 pandemic. The 7-day conference was attended by over 1000 participants from all over the world. It was estimated that about a third of the participants developed an influenza-like illness with an overall attack rate of 36%. The virus was isolated from throat and nasal washings; this was found to be the pandemic A/Hong Kong/68 virus. There was evidence that close contacts of returning attendees were also affected, but none of the episodes led to a rapidly expanding focus of infection. The high attack rate raises the question of whether the indoor setting may represent a particularly high risk.

Lastly, there was an innovative study examining the 2009 A(H1N1) outbreak in Mexico [34]. The authors described the details of 202 cases and then used a series of epidemiological tools to map the spread of the virus throughout the country. They identified two major contributory factors to the rapid spread of influenza: one was the very high population density in parts of Mexico City; the other was the massive religious festival involving more than 2 million people, which took place in the city’s Iztapalapa neighbourhood at the very early stage of the outbreak, shortly before the new virus was identified. It is suggested by the authors that this temporal association between the mass gathering event and the subsequent increase in numbers of cases may reflect participants returning to their own neighbourhoods across the city, and from there to other parts of the country.

3.4. Historical outbreak analyses

Inevitably for a subject of research such as mass gathering restrictions, where prospective studies present serious practical challenges, researchers have sought to utilize historical data to try to draw out major lessons for current impact. This review included three historical analyses of the non-pharmaceutical responses to the 1918–1919 influenza pandemic.

Markel et al. [37] examined the variety of non-pharmaceutical interventions that were deployed in 43 American cities during the 1918–1919 influenza pandemic. The combination of school closures and concurrent public gathering bans was implemented in 34 (79%) of the 43 cities and was the commonest combination of measures deployed. Applied early in the pandemic, this combination was significantly associated with reductions in the weekly excess death rates. Hatchett et al. [36] undertook a similar analysis on a smaller number of American cities where the timing of 19 different types of non-pharmaceutical interventions was available. They found that the early application of multiple interventions showed a trend towards lower cumulative excess mortality, but that no single intervention showed an association with improved aggregate outcomes for the pandemic. Both studies suggested that for non-pharmaceutical interventions to be beneficial, they should be applied early and in a sustained manner.

A further review of the U.S. public health response to the 1918 pandemic by Aimone [35] gave conflicting results. This review examined the public health response in New York City. In New York during the 1918 pandemic, mass gatherings were not prohibited nor were schools closed; instead, the city opted for a policy of staggered business hours to avoid rush-hour crowding, enhanced surveillance so that cases were quickly identified and isolated, and an intensive program of health education. The reported outcome measures for New York City were comparable with those seen in other American cities, and New York City experienced one of the lowest excess death rates on the eastern seaboard of the United States.

3.5. Event surveillance reports

Surveillance reports from four major sporting events within the last decade were considered suitable for review [38–41]. The study by Gundlapalli et al. [38] reported the experience of influenza surveillance during the Winter Olympics at Salt Lake City in 2002. No indication of the numbers attending was given, but it is assumed that the numbers were large. Twenty-eight cases of confirmed influenza from three clusters of influenza-like illness (ILI) were identified and these were restricted to either participants in the Games or support staff for the Games. The clusters consisted of 12 members of a national team who trained and lived together, 8 participants of a sport and 13 law enforcement officers who worked and lived in close proximity.

Lim et al. [39] reported on the experience of managing the Asian Youth Games at Singapore in June 2009. These games involved over 2000 athletes and officials from 43 countries. Although numbers of spectators are not given, it is assumed that the crowds were large. At the start of the Games, Singapore had already reported 600 confirmed cases of H1N1 (2009). Six laboratory-confirmed cases of H1N1 (2009) were identified during the 8 days of the event—four on one football team. No information on the numbers of confirmed H1N1 (2009) in the population after the Games was available.

There was no increase in ILI activity during the Winter Olympics in Torino in 2006 [41]. The final study by Schenkel et al. [40] reported on the experience of syndromic surveillance during the FIFA World Cup in Germany during June/July 2006. At the time, Germany was experiencing a very large outbreak of measles. Measles is essentially spread by the respiratory route and is highly infectious. However, despite enhanced daily surveillance, no outbreaks of respiratory disease or measles associated with the World Cup were detected.

3.6. Systematic review

As stated in Section 2.6, the approach to quality assessment of the reviewed papers was based on a pragmatic framework for categorizing studies into low- or high-risk for bias, depending on the presence or absence of significant methodological limitations. A high risk of bias was attributed to papers with at least two significant methodological problems as identified by the review authors (Table 6).

4.1. What is a mass gathering?

In attempting to understand and describe a situation or intervention, a common understanding or definition is essential. There is currently no generally accepted definition of what constitutes a mass gathering. However, the literature on mass gathering medical care highlights an emerging consensus amongst those providing emergency medical care at organized events. In this setting, mass gatherings are considered to be organized events with more than 1000 people in attendance [4–6]. A recently published guidance document from the World Health Organisation (WHO) expanded the term to cover any organized or unplanned event involving enough people to “strain the planning and response resources of the (host) community, State or nation” [42].

The major limitation in trying to define mass gatherings is that any single definition would inevitably be too simplistic as it would need to incorporate events as diverse as the Hajj (lasting about 1 month and involving between 2 and 3 million people), and a football match (involving several thousand spectators over a period of about 2 h).

A system for classifying mass gatherings on the basis of size and duration is lacking and may be required.

4.2. What does the evidence say about mass gatherings and the risk of influenza transmission?

In recognition of the difficulties of conducting hypothesis-based studies that directly implement and assess the effects of restrictions of mass gatherings in real life, an indirect approach was taken to address the review questions as follows:

4.3. Practical implications of the findings of the review

There are two further, critical domains of uncertainty that need to be considered in the development of evidence-based guidance and policies regarding mass gatherings. The first domain relates to issues around the current understanding about how influenza is spread, and factors that can affect transmissibility (e.g. host factors, pathogen factors, environmental factors and particle size) [9]. Key questions remain in these areas, which may be important in making specific recommendations regarding particular types and scope of mass gathering restrictions.

The second domain impinging on the potential effectiveness of any public policy on mass gatherings includes the whole range of factors affecting adherence and compliance. For instance, the experience of the 2009 pandemic has raised significant questions around how willing people might be to comply with bans imposed on mass gathering restrictions [48–51]. Other challenging issues include the problematic ethical and legal frameworks for implementing restrictions for public health purposes [52,53], as well as considerable logistical and economic implications. If long-planned events were to be cancelled, who would be liable for the huge personal, corporate and national costs that such cancellations might incur? In considering policy recommendations within a pandemic context, the most practical approach for all but the more severe pandemics may be a strategy of encouraging voluntary restrictions. This would involve giving the public the best available information and advising rather than legislating that organizations and individuals avoid non-essential events where there is at least some evidence of transmission risk. For such a strategy to have a chance of succeeding, it would be most important to have in place a carefully and sensitively prepared communication strategy, since recommendations to avoid public gatherings are likely to run against powerful social pressures to do otherwise [54]. The success of any public health strategy—and this must be particularly true in relation to a potential or imminent pandemic situation—requires the building of trust [55] and an intelligent and purposeful engagement of the public even prior to the event [56].

There are other important issues to note. Although this review has focused on mass gatherings, limiting transmission of influenza clearly requires a multifaceted approach. Some studies in this review reflected such an approach; for example, in the historical outbreak investigations where restrictions on mass gatherings were combined with other non-pharmaceutical measures [32,36,37]. It would be prudent to apply the best evidence relating to other social distancing interventions in conjunction with any specific policies on mass gatherings.

An additional question that should be considered is whether non-pharmaceutical health interventions such as mass gathering restrictions, actually reduce the health burden of influenza, or only modify the epidemiology by temporarily delaying the eventual impact of an event such as a pandemic. It is tempting to speculate on the possibility of a reduction in disease burden; historically, some of the American cities that implemented mass gathering restrictions during the 1918–1919 pandemic seemed to achieve significant reductions in peak morbidity and mortality that may well have reduced the overall burden. However, pragmatically, countries like the United Kingdom aim to achieve a slower spread, to prolong the troughs and to flatten the peaks of the epidemic curves to buy time for countermeasures to take effect and to enable services to cope with a lower volume of activity.

In this context, the most important point to emphasize is that mass gathering restrictions must only be a part of a range of interventions. The synergistic effects of multiple interventions is what would allow the best chance to achieve significant reductions in overall burden.

4.4. The need for further evidence

More evidence is needed on this subject. There is a need for well-designed studies to more accurately quantify the nature of the infections causing flu-like symptoms at mass gatherings, confirm the key parameters that influence the transmission of influenza in these settings, and to directly assess the impact of mass gathering restrictions. But as “gold standard” randomized comparison designs are extremely difficult to apply to this kind of intervention, attention needs to focus on optimal observational study options. From a UK perspective, British people travelling abroad for mass gatherings such as the Hajj or other large events could be approached for inclusion in prospective studies for comparison with appropriately matched, non-travelling controls, with care being taken to avoid drawbacks observed in existing studies. Within the UK, mass gatherings ranging from indoor events in theatres and cinemas, to outdoor events such as football matches and major musical events like the Glastonbury festival or Travellers’ horse fairs represent potential opportunities for carefully designed prospective cohort studies to be undertaken. The biggest issue is persuading funders that given the current paucity of good studies, such further research is still needed.

4.5. Strengths and weaknesses of the review

As previously noted, this review has examined an intervention area in which there are a limited number of relevant studies and there is limited common understanding of what constitutes a mass gathering. Despite these challenges, it has been possible to extract limited strands of evidence that may be useful towards policy development. Also, within the boundaries of the inclusion criteria, and despite the limited work timescale, this review was able to capture all the relevant studies identified by recent systematic reviews investigating non-pharmaceutical interventions designed to limit transmission of influenza and/or other respiratory viral infections [7,8,57].

However, due to time constraints in carrying out this work, as described in Section 2.5, while every effort has been made to be as comprehensive as possible, it is acknowledged that the rapid evidence assessment process necessarily involved some limitations. The literature search is not necessarily exhaustive and there is therefore a possibility of incomplete retrieval of all potentially relevant studies.

While a range of study designs were reviewed, there was not a single randomized controlled trial that was suitable for inclusion. This is of course not surprising, given the formidable logistical, cost and ethical hurdles that make large-scale experimental epidemiological studies of the restriction of mass gatherings impractical and probably impossible. Some of the included studies had significant design and quality issues as duly reflected in the individual paper summaries and the discussion, and highlighted in Tables 4 and 6.

A common issue with a number of the studies included in the review is that they depended on clinical symptoms like cough or syndromes such as influenza-like illness, rather than laboratory diagnosis of influenza. In such studies it is not possible to isolate the impact of influenza as opposed to other respiratory viruses, and the risk of influenza could potentially be over-interpreted. In the studies where laboratory testing was undertaken, influenza was only confirmed in minority proportions of people with respiratory symptoms, ranging between 6% and 14% [23,24,27], but possibly up to a roughly estimated 40% in one study [25].

This review did not include specific “specialized” settings that may arguably be regarded as mass gatherings, such as groups of people travelling for periods of time on cruise ships. The authors took the view that any transit-related settings would be best considered under a separate evidence review of influenza transmission through transportation. Modelling studies were also excluded, and it is acknowledged that their inclusion may offer an additional dimension in order to build a fuller picture.

The application of quantitative techniques could have potentially enhanced the simple narrative approach that was adopted for the analysis. However, there are insufficient studies presenting quantitative data on this subject, and moreover, there is a high level of heterogeneity amongst the identified studies.