2.1. Multicriteria Group Decision-Making

MCGDM methods can be considered to be an extension of traditional group decision-making methods [2]. In a traditional group decision-making method, experts need to rank a set of alternatives based on their preferences. On the other hand, in MCGDM methods experts are asked to rank the alternatives considering a pre-specified set of criteria. Introducing criteria to group decision-making processes helps experts carry out the decision in a less subjective way. Moreover, the criteria will not have the same importance. Therefore, it is possible to associate a weighting schema that indicates the weight that should be assigned to each criteria when calculating the final results.

The choice and weighting of the criteria derive from a dynamic procedure where it is necessary to manage situations in which alternatives are added or removed throughout the process [10]. Some approaches propose the use of a multi-granular fuzzy linguistic modeling process in order to allow the modification of experts, criteria, and alternatives at any time during the process [11].

One of the major issues restraining the ability to determine the right method for choosing the best player in a match is the absence of the ability to handle uncertain and inadequate information, which mostly happens in match conditions. In the practical problems of player performance evaluation, several criteria are affected by uncertainty. Experts' judgments are usually uncertain and difficult to measure by exact numerical values, so the application of fuzzy logic has become the best alternative for solving the shortcomings characterized by vagueness and imprecision. Recently, several studies have applied the typical MCGDM methods to a range of fuzzy environments [12].

At the end of the process, an aggregation method has to be applied to evaluate the player performance. For this purpose, different aggregation operators can be found in the literature [13]. Fuzzy logic aggregations operators are useful for managing uncertainty, vagueness, and subjectivity. These operators go beyond combining numerical values on [0,1] and provide mechanisms for combining fuzzy information components [14]. These approaches have been applied with success in a wide range of applications [15–17].

2.2. Player Evaluation Methods

The evaluation of a player's performance is one of the most critical aspects of the application of advanced analytics and statistics in sport.

It seeks to achieve a more objective view of productivity, efficiency, effectiveness, and value of the components of the game at the individual level. In addition, this aspect can play an important role in making better decisions both within a club and at the level of the team.

In many sports, there is a great deal of literature related to the evaluation of player performance [18]. Various approaches have been proposed to rate players for specific tasks, for example, a Bayesian regression model to obtain an estimation of the number of goals scored of any soccer player [19]; and a soccer player ranking system based completely on the value of the passes completed [20]. The final value is computed based on the relationship between pass locations and shot opportunities generated. Data-driven frameworks that offers a multi-dimensional and role-aware evaluation of the performance of soccer players [21] are useful tools to address some limitations of the valuation approaches, for example, the mono-dimensional focus, the specificity of each player's role on the field, and the lack of a gold standard.

The player ratings can be labeled as bottom-up and top-down. In bottom-up ratings, a player's scoring increases if he performs an action that is associated with a positive result for the team. Player's scoring is not affected by players who did not participate in the action. On the other hand, in top-down ratings, the main idea is to reward the team as a whole for each positive action. The credit for this performance is distributed among individual players involved in the match, independently of the actions they performed.

One of the most popular top-down productivity metrics is the PM rating model [22]. The PM rating and its variations have been used extensively in ice-hockey [23], basketball [24], and soccer [25]. However, there are some doubts about its usefulness when it comes to comparing individual players, as the rating is established by many factors beyond the control of the player being evaluated. Moreover, in sports where player-specific functions (NFL or handball) stand out, the effectiveness of this metric is limited and requires attention beyond this study. For this reason, we apply a bottom-up rating method instead of a top-down rating like PM.

Handball experts such as coaches and sports journalists have long estimated player performance. Although the academic and professional attention to quantitative analysis of data has exponentially grown over the past years, there is no much work concerning evaluating a handball player in a game-by-game scenario [26]. According to several studies [27], variables like shots saved by the opposing goalkeeper, technical fouls, steals, and the goalkeeper's saves are the key indicators of team performance.

In this context, we aim to provide a comprehensive framework for assessing handball player performance on a match-by-match basis.

3.1. Initial Settings

Before starting the process, the initial set of parameters for carrying out the MCGDM

• Initial set of experts: The initial set of experts who will participate in the discussion is established.

  E={e1,e2,⋯,en}

  It is possible to assign weights to the experts. Usually, experts that have more experience are assigned higher weighting values.

  We={we1,⋯,Wen}

• Initial set of criteria: The initial set of criteria values is defined.

  C={c1,c2,⋯,cl}
  that refer to the performance of the set of players X based on the preferences they provided.
  X={x1,x2,⋯,xm}

  Criteria can make a positive or negative contribution to performance. Thus, a sign vector is associated with the set of criteria:

  σc={σc1,σc2,⋯,σcl}
  where σci∈{+,−}

• Initial linguistic label set: Experts can express their views on the criteria using one of the pre-defined sets of labels, each with a different discriminatory power.

  S={S3,S5,S7}
  where the superindex represents the number of available labels. For instance, the experts can use the following linguistic label set for providing preferences:
  S5={Very Low,Low,Medium,High,Very High}

Once all these parameters have been established, experts can start the selection process.

In traditional group decision-making problems, experts are required to rank a set of alternatives. However, in this study, the issue to solve consists of the difference in importance of each criterion. Then, experts must evaluate the set of criteria. Therefore, it is necessary to obtain a weighting vector,

which indicates the weight that should be assigned to each criterion while calculating the evaluation of a player xi in one match.

3.2. Expert Opinion

Every expert, depending on their own experience, must express an opinion regarding the selected criteria. In this process, experts have the following degrees of flexibility:

• Each expert chooses the linguistic label set (Sn) that they want to use.

• Each expert selects the criteria that they want to provide evaluations on.

• Each expert can add new criteria that were not considered at the beginning of the process.

Once all the experts have provided the required information, if the expert applied different sets of linguistic labels, the information is standardized (represented by the same scale) to be used later in the aggregation processes. For this purpose, multi-granular fuzzy linguistic modeling methods [28] can be applied. Our study adopts a model based on the concept of 2-tuple fuzzy linguistic representation [29]. This approach has produced excellent results in practical applications, especially in terms of efficiency [30].

A high level of consensus concerning the expert opinions is mandatory to assure the viability of the aggregation process, that is, we require that the expert estimates have a common intersection at some α-level cut. For this purpose, we compute a global consensus degree that is compared against a consensus threshold provided by the set of experts prior to the application of the consensus. If this degree is greater than or equal to this consensus threshold, then the consensus-reaching process is considered successful and hence it should end. Otherwise, the experts need to discuss it further.

Once the expert opinions are validated, a fuzzy aggregation process is applied to summarize the results as explained in the following subsection.

3.3. Aggregation

We use fuzzy techniques for finding a compact and synthesized representation of expert opinions. The aggregation process is carried out in the following steps:

1. Each expert defines the linguistic vector Ve={v1e,v2e,⋯,vle} indicating the importance that they give to each of the chosen criteria. The union of all these linguistic vectors generates the global evaluation matrix containing all the valuations from all the experts.

2. Individual Aggregation: The linguistic vector Ve={v1e,v2e,⋯,vle} representing the expert evaluations of all indicators Se is aggregated. The weights assigned to the experts We are used. This is done through the following steps:

   1. Baseline: A fuzzy definition of each label is used (balanced or unbalanced)

   2. Aggregated Valuation Matrix: We obtain the weight of each label in each criterion according to the occurrence frequency and the expert weighting wen. The weighted mean operator is used with this aim.

   3. Fuzzification: We use a fuzzy operator in order to obtain a fuzzy set for each criterion, for example, the minimum t-norm, which truncates each baseline membership function according to the previous computed weighting of each label.

   Then, the aggregated output will be the fuzzy set representing how relevant each indicator would be in measuring player performance.

3. Defuzzification: The center of area (COA) defuzzification method computes the center of mass of the membership function of the fuzzy set (the centroid). The COA method maintains the underlying semantic ranking relation within the set of linguistic labels, that is, given two linguistic labels si,sj∈S such that si<sj then uCOA(si)<uCOA(sj). Thus, the centroid of a type-1 fuzzy set A in a continuous domain X is calculated as follows:

   UCOA(A)=∫xx∗μA(x)dx∫xμA(x)
   and it will be the method adopted here to obtain a numeric output.

4. Stratification: In order to reduce the sparsity of the weighting distribution, we carry out a stratification process. The purpose is to group criteria with similar evaluations using clustering. As a result, we obtain the criteria segmented into groups according to this importance.

5. Collective Aggregation: The aggregation process obtains the final solution according to the opinions given by the experts. These results allow us to define a weighting scheme to evaluate the performance in accordance with this set of criteria. For this purpose, we aggregate this value for each group of indicators and normalize the final result.

3.4. Information Gathering

For each player (xi) within each match, the values that each player obtains, associated with each rating criterion ([xi,ci]), are collected. Subsequently, using the weighting scheme previously obtained, the valuation of each player and the rankings of the players are calculated.

5.1. Initial Setting

A collection of basic criteria is selected as a baseline of the decision process to determine the indicators that must be considered to model the player performance. Table 6 shows those indicators that can be obtained from online statistics of handball games.

This basic set is extended using the criteria of a national handball coach, and the indicators used for the evaluation of junior players by the Spanish Handball Federation. This extension achieves a higher level of granularity than the Basic Indicator Set, for example, missed shots are broken down into several subcategories to better capture the cause of the error.

Table 7 shows the criteria collection for this extended baseline.

5.2. Expert Opinion

The obtained set of criteria is given to the 15 experts (Spanish handball coaches) who are weighted according to their coach experience. Fuzzy linguistic terms, such as most important, important, and normal, were used to determine the experts' influence weightings, which were represented by fuzzy membership functions. For example, a National Handball coach was assigned the most important influence, while a Level 1 coach was assigned an important influence weighting [31].

Every expert, depending on his own experiences, must express an opinion regarding the chosen criterion. In this case, each criterion is evaluated by linguistic labels belonging to one of the following predefined label sets.

These labels sets are represented by using Trapezoidal and Triangular Fuzzy Numbers as can be seen in Figure 5.

Figure 5

Linguistic labels definitions as fuzzy sets.

Once the expert opinions are known, the fuzzy techniques explained below are applied to aggregate the set of linguistic labels representing the evaluations of all criteria.

5.3. Aggregation

Then, the fuzzy aggregation process is applied. Table 8 shows the results of the importance of each chosen indicator.

These results allow us to define a weighting scheme to evaluate the performance of a handball player in a specific match (see Eq. 1).

where the sign of each group is chosen according to the characteristics of the criteria.

6.1. Basic Plus-Minus Comparison

In this section, the evaluation of the handball match is carried out with a basic PM rating, and the results obtained are compared with the proposed approach rating.

The main idea is to apply the basic PM rating method [22] in handball: (i) Consider a player from a handball team and a given match. (ii) Count the number of goals scored by the player's team while the player is in the game, and then subtract the number of points scored by the opposing team during the same time intervals. (iii) The resulting number will here after be referred to as the basic PM rating of the player.

Table 10 shows the results using both rating processes in a match from the second division of handball in 2019, between Handball Málaga and the Handball Pozuelo de Calatrava.³ The table is simplified by showing only the two best players and the two worst of each team according to both methodologies.

The best player according to the basic PM rating was player No. 21 of Handball Pozuelo de Calatrava and according to our approach, it was player No. 4 of Handball Málaga, who was chosen as Most Valuable Player. The worst player according to basic PM rating was player No. 14 of Handball Málaga, and according to our approach, it was player No. 16 of Handball Pozuelo de Calatrava.

These results highlight the main problem of PM in handball: the evaluation of players who are defensive specialists (players who only defend) or offensive specialists (players who only attack). Defensive players have great difficulty performing positive actions, and will lose points as they only play when their team will probably concede goals, except goals in fast breaks. On the other hand, the player who only attacks will have positive ratings unless the opponent team scores a goal with a fast break.

The results obtained show that the basic PM rating generates false MVP's. This is the case with player No. 21 offensive specialist of Handball Pozuelo de Calatrava, who only participated in the attacking actions of her team but was not decisive. The chosen MVP in this match was player No. 4 of Handball Málaga, who was found to be the most valuable player with our approach.

In the same way, the defensive players get low PM ratings, for example, players Nos. 20 and 22 of Pozuelo de Calatrava and players Nos. 14 and 65 of Handball Málaga. They obtain the worst values according to the PM rating, being player No. 14 of Handball Málaga, who only defended and accumulated fewer errors and negative actions than player No. 16 of Handball Pozuelo de Calatrava, who had the minimum score with the proposed approach rating in this study.

In the case of handball, where changes are normally made without stopping the clock, the basic PM may offer false high and low values that do not reflect the outcome of the match. For example, specialized players who only play when it is more likely they will score goals (positive actions) or concede them (negative actions). Moreover, the basic PM rating system, as opposed to the approach proposed here, does not value many positive actions in defense and attack, nor the actions and errors of each player.