QUANTIFYING TRADE-OFFS BETWEEN MULTIPLE CRITERIA IN THE FORMATION OF PREFERENCES

 

A. P. SANT'ANNA

 Universidade Federal Fluminense

L. F. A. M. GOMES

Faculdades IBMEC

           

ABSTRACT            

We develop here a strategy to model the formation of preferences starting from ranking according to simple criteria. Rules to transform the original ordinal variables are discussed. The weights attributable to each criterion are finally obtained by fitting a linear regression.

 

Keywords: multicriteria decision analysis - regression analysis - probability of choice

 

 

1.  INTRODUCTION

We model here the composition of preferences derived from ranking according to multiple criteria. The basic model starts with a linear equation explaining the final probabilities of choice in terms of the preferences elicited when judgement is based isolatedly on each criterion.

Alternatives are developed to represent, through transformations of the explanatory variables, the influence of factors that systematically distort the relation between the objective partial ranks and the final preferences. Among such factors, prominence of one criterion over another and magnitude effects conducting to concentration in the alternatives most advantageous and overestimation of alternatives of large rewards are considered. 

These transformations involve deriving a truncated geometric probability distribution from the ranks. Normalization and equalization of low probabilities are also used. To take into account the effect of prominence when a hierarchy among the different criteria is provided, another transformation approaches to the average of the values given by the more important criteria those values in larger variation.

The application of these transformations in a case studied results in improved fit. The case considered was that of the formation of preferences of bettors in a horse race based on two criteria.

In the following section, we discuss different forms of handling the data in order to take into account distortions due to the effect of systematic factors.  In Section 3, we describe the model building process. In Section 4, we present specific models and alternatives to perform inference on the parameters of these models. Finally, in Section 5, we present an example.

 

 

2. TRANSFORMATIONS OF VARIABLES

            Our goal is to find the precise weights given to evaluations resulting from the application of multiple criteria when ranking choices between different possible alternatives. We try to estimate these weights by adjusting a linear model relating global preferences to evaluations according to the different criteria.

The preference must be based on objective knowledge. But its construction passes through psychological channels. The modeling of the process of formation of preferences has to relate to final preferences the measurements of objective knowledge of advantages and disadvantages of each option. This must be done through a function that takes into account the effect of the affective aspects involved.

            Modeling preferences derivation is thus twofold. By one side, we have to determine which criteria affect the final preference. And by the other side, we have to determine how to evaluate the options according to each criterion in order to make our measurement reflect the influence of perception distorting factors.

We consider here one source of distortion: the need to simplify things. Different effects of the human desire to deal on simpler representations can be systematically incorporated into the modeling strategy:

a) prominence effects, that lead to disregard criteria judged less important; we consider this aspect especially in the case of great divergence between the criteria, risk aversion inducing a reduction of the chance of being chosen options for which different criteria present contradictory indications;

b) certainty effects, leading to amplify the probability of choice of the most probable options;

c) magnitude effects, leading to equalize the probability of choice of options of very low probability.

We make these effects appear in the model through changes in the values of each option initially derived from the isolated application of each criterion. But there are other modeling alternatives, besides transforming variables, to take into account these aspects. We may introduce additional explanatory variables in the model to represent the affective factors. For instance, a measure of dispersion in the vector of partial preferences can be added to convey the effect of risk aversion.

The coefficients of the preference building equation may also be modeled, in second level equations, as function of certainty or magnitude measuring variables. If we try to model also the evolution of these parameters through time, we would finally have a dynamic hierarchical model (Gamerman and Migon, 1993).

By treating the initial coefficients as second level dependent random variables, what we do, essentially, is to add an explained component to the variance of the first level equation disturbance. If we are not able to collect data on variables that might explain the coefficients evolution through time or among the options, we may just model them as normal random variables, with unknown expected value and dispersion. This is equivalent to model the variance disturbance of the first level equation as a sum of different components, some of them proportional to the same equation explanatory variables. When fitting such a model, we are accessing the influence of each criterion on the variability of the final preferences.

We do not follow this approach here because it involves adding new parameters to the model. Since we are not able to deal with a large number of alternatives jointly when building preferences, we would in practice run into an excessively large number of parameters relatively to the number of observations and it would be difficult to produce statistically significant estimates.

            Another approach would be to add explanatory variables representing iterations between the criteria. Once more, the problem would be the lack of statistical significance that we ought to expect due to the increase in the number of parameters to estimate and the correlation between the explanatory variables.

 

 

3. MODEL BUILDING

            We present, in the following, step by step, the transformations of variables that we apply in order to derive a final linear representation of the preferences in terms of partial evaluations according to each particular criterion.

First, we have to choose a starting point, an initial manner of presenting the results of the application of each isolated criteria. Variables transformation consists in modifying these initial measurements to reflect the affective influences through which the final preferences are built.

We take as initial information a full set of coherent preference indications according to well-defined criteria. Indifference is admitted, in the sense that ties are allowed: two or more options may receive the same rank. Empty spaces between successively ranked options, possibly corresponding to alternatives that are not among those compared, may also be left. Ties and empty spaces allow for a compromise between purely ordinal and cardinal classifications. Setting our evaluations always in terms of ranks, instead of measurements in varying scales, allows for easier numerical determination of breakpoints for the transformations. This makes them easier to be followed and results easier to interpret. 

We may obtain the needed coherent preference indications by attributing to each option a value in a previously defined scale where each point corresponds to a position that is easy to determine according to the criterion that is being applied.  We may also start by comparing pairs of alternatives, according to such criterion, as in Saaty (1980) or Lootsma (1993), carefully preserving transitivity.

Transformations designed to model the effect of risk aversion must result in attributing greater importance in the model fit to the observations presenting vectors of classifications according to the various criteria presenting greater agreement between the coordinates. In the analysis of investments, traditionally (see, for example, Sharpe, 1964 or Engle, 1993), the risk is measured by the dispersion in the portfolio. We may incorporate risk aversion into our model, through the inclusion of an explanatory variable measuring, for each alternative, the dispersion in the vector of original evaluations of the alternative. For the same parsimony considerations above explicited, we prefer to work in the transformation of the initial measurements. This is done here, taking simultaneously into account the prominence effect. We start by ordering the criteria and modify the values corresponding to the less important criteria whenever they present values that we identify as very far from the values derived from those that we have classified as  more important.

           A concrete formulation for this transformation consists of:

1) calculate, for each option and each criterion, the distance from the rank of the option according to the criterion to the average rank of the same option according to the criteria classified as more important than it,

2) after that, calculate, for each criterion, a measure of the dispersion, for instance, the standard deviation, of this vector of distances, and, finally,

3) establish a limit in terms of this dispersion measurement, for instance the 3 sigma or 0.001 probability limit of Shewhart control charts, and shift each evaluation that is found out of bounds by an amount enough to reduce to such limit its distance to the average rank referred in step 1.

A simpler alternative consists on only shifting values that are away, from the nearest among the values supplied by the criteria relatively more important, more than a previously fixed bound.

Some kind of prominence principle can also be applied within the range of application of an isolated criterion, to amplify distances between the preferred options and to eliminate differences between low probabilities. If we replace ranks by probabilities of choice as best, the options ranked first will be much far away from those following them than the others. Another way to transform the variables in the direction of expanding distances between the preferred options is to change from the arithmetic to the geometric scale. This can be done by treating the ranks as exponents on a given basis. We do that, by fixing the basis in such a way as to replace ranks by probabilities belonging to a truncated geometric distribution. The result is very close to the idea of measuring preferences in powers of two.

This form of measurement of preferences as probabilities of choice that add to 1 may bring difficulties to the inference on the parameters of the linear model. The hypothesis of independence is violated, as much as the hypothesis of normality. To circumvent this problem, we change a second time the distances, by applying the inverse standard normal transformation. Since our highest value, according to the truncated geometric probability distribution was around one half, this second transformation has a smoothing effect on the largest and smallest distances.

            Our last change consists in slightly raising, to a uniform threshold, all small probabilities. This final transformation pays attention to the principle of loss aversion, present in the Prospect Theory (see Kahnemann and Tversky (1979) and Gomes and Lima (1992)). We associate it to the idea of simplifying reasoning, that may answer for the attitude of disregarding numerical values when dealing with small probabilities and huge gains. The transformation we use here to take into account this aspect consists in equalizing all probabilities in the set of the smallest probabilities responsible, altogether, for 1/1000 of the total after all other changes have been made, to the larger among them.

A simpler alternative to this last transformation would consist in just raising to a threshold, possibly given by the highest among them, the probabilities of all options in the last quarter or any other fraction previously chosen of worst ranked options.

 

 

4. MODEL ADJUSTMENT

               The linear regression model for the formation of the preferences starting from ranks derived from different criteria and subsequently modified can be formally represented by:

E (Y | R) = a + Sb_iX_i(R),                                                                       (3.1)

with the following specifications.

1) Dependent variable: Y.

Y_j denotes the final preference for the j-th alternative.

2) Initial explanatory variables: R_i, for i varying from 1 to k, k denoting the number of criteria.

R_ji denotes the rank of the j-th alternative, being these ordered from the most advantageous to the less advantageous according to the i-th criterion.

 3) Explanatory variables transformed due to the effect of prominence among criteria: P_i(R), i varying from 1 to k.

P₁(R) = R₁                                                                                              (3.2)

and, for i >1, P_ji(R) = P_ji                                                                       (3.3)

if D_ji < 3*sigma(i),                                                                                  (3.4)

where D_ji = R_ji - Average(R_jq: q<i),                                                       (3.5)

the difference between the rank of the j-th alternative according to the i-th criterion and the arithmetic mean of its ranks according to the criteria that precede the i-th criterion, and sigma(i) is the standard deviation of the vector = (D_1i..., D_ni), n denoting the number of alternatives.

Otherwise P_ji(R) = Average(R_jl: l<i) + 3*sigma(i).                                 (3.6)

4) Explanatory variables transformed by the effect of prominence among alternatives: V_i, i varying from 1 to k.

V_ji denotes the value of the power of basis B_i and exponent P_ji, where B_i is determined by the constraint that be 1 the sum of the B_i^s, for s varying from 1 the n.

5) Explanatory variables transformed due to the reward magnitude equalization effect: X_i, i varying from 1 to k.

X_ji = f(V_ji),                                                                                        (3.7)

if the sum of the V_mi for V_mi<V_ji is greater than 0.001 that means, if the value given by the i-th criterion to the j-th alternative is above the percentile p=0.001. Here f denotes the standard normal inverse accumulated distribution function.

                           Otherwise, X_ji = f (V_pi), p denoting the .001percentile.                            (3.8)

Specified the model, we must decide about the procedures for inference on its parameters. The transformations that we apply to the initial ranks involve changes, basically, on extreme values. For this reason, it is important to detect outliers and eventually correct mistakes in the evaluation of options with discrepant values. An automatic mechanism to generate information about extreme values is to simultaneously fit the linear model by least squares and by minimization of the sum of absolute residuals. As the change of the optimization rule does not involve variables transformations, the comparison of the results of the adjustments produced by the two algorithms can help to identify the options where contradictory evaluations, as well as extreme evaluations according to some isolated criterion, have greater power of affecting the final ranking.

 

 

5. AN EXAMPLE

Here we apply the modeling strategy above developed to the formation of preferences of bettors in horse races. In this case, we know precisely the final preferences distribution, given by the amount of money bet on each animal. Thaler and Ziemba (1988), extending results of Ali (1977), have demonstrated that, under general conditions, longshots are overbet, so that favorites are underbet. But we do not know which factors combine to produce such preferences. We consider here two criteria: previous ranking of the animals, provided by the Track Official Program, and ranking of the jockeys, derived from the number of riding compromises signed throughout the season.

We have chosen to analyze the last Rio de Janeiro's Selective Proof to the Latin American Association of Jockey Clubs, one of the few proofs in Rio de Janeiro races midyear calendar counting with an official previous rank of the competitors.

            We applied to the ranks all the transformations above described and fitted the model by classical least squares and by minimization of the sum of absolute residuals. 

                Table 5.1 presents, for each animal and jockey, the final preferences, the initial ranks according to both criteria and the transformed ranks.

We assumed the first criterion (animal previous rank) prominent with respect to the second (previous preferences for the jockey). With a standard deviation of 3.15 for the vector of deviations of ranks, the shift in the second criterion classification for highly discrepant evaluations produced a slight improvement in the positions of the options ranked 14^th and 15^th according to this second criterion. This was enough to produce a final reduction of the number of options in the inferior 1/1000 percentile from 8 to 7.  Table 5.2 shows the results of the least squares adjustment, first for the model without variables transformations and, in the second line, for the model with transformed variables. We can see that, in this example, all adjustment indicators are larger for the second model. The effect of the second criterion, that was not statistically significant before the transformations of the data, becomes statistically significant after the transformations.  

 

TABLE 5.1. INITIAL DATA AND TRANSFORMATIONS

 

TABLE 2. MODELS COMPARISON

 

 6. FINAL COMMENTS

We sketch in this work some alternatives for the handling of different factors affecting the formation of preferences. We demonstrate that statistical models based on the transformation of explanatory variable are able to incorporate the effect of behaviors suggested by different theories about sources of departures from rational behavior. The present ideas can be used to advantage in other contexts. The good adjustment of the statistical model always depends on the existence of a large enough volume of data. When this is available, however, it provides a path to understand effects so far little investigated quantitatively.

 

REFERENCES

            Ali, M. 1977. Probability and Utility Estimates for Racetrack Bettors. Journal of Political Economy 85, 803-815.

Engle, R 1993. Statistical Models for Financial Volatility, Financial Analysis Journal, 49, 23-28.

Gamerman, D. e Migon, H. S. Dynamic Hierarchical Models. JRSS, B 55, 629-642.

Gomes, L. F. A. M. and Lima, M. M. P. M. 1992. From Modeling Individual Prefer­ences to Multicriteria ranking of Discrete Alternatives: a Look at Prospect Theory and the Ad­ditive Difference Model. Foundations of Computing and Decision Sciences 17, 172-184.

Kahnemann, D. and Tversky, A. 1979. Prospect Theory: an Analysis of Decision under Risk, Econometrica 47, 263-291.

Lootsma, F. A. 1993. Scale Sensitivity in the Multiplicative AHP and SMART. Journal of Multicriteria Decision Analysis 2, 87-110.

Saaty, T. L. The Analytic Hierarchy Process, Planning, Priority Setting and Resource Allocation. Mc. Graw Hill: New York, 1980.

Sharpe, W. F. 1964. Capital Asset Prices, a Theory of Market Equilibrium under Con­ditions of Risk. Journal of Finance 19: 425-442.

Thaler, R. and Ziemba, W. T. 1997. Parimutual Betting Markets: Racetracks and Lotteries. Journal of Economic Perspectives. 2, 161-174.