Qualitative Decision Methods for Multi-Attribute Decision Making

08/04/2015 ∙ by Ankit Agrawal, et al. ∙ Northwestern University 0

The fundamental problem underlying all multi-criteria decision analysis (MCDA) problems is that of dominance between any two alternatives: "Given two alternatives A and B, each described by a set criteria, is A preferred to B with respect to a set of decision maker (DM) preferences over the criteria?". Depending on the application in which MCDA is performed, the alternatives may represent strategies and policies for business, potential locations for setting up new facilities, designs of buildings, etc. The general objective of MCDA is to enable the DM to order all alternatives in order of the stated preferences, and choose the ones that are best, i.e., optimal with respect to the preferences over the criteria. This article presents and summarizes a recently developed MCDA framework that orders the set of alternatives when the relative importance preferences are incomplete, imprecise, or qualitative in nature.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The fundamental problem underlying all multi-criteria decision analysis (MCDA) problems is that of dominance between any two alternatives: ‘Given two alternatives A and B, each described by a set criteria, is A preferred to B with respect to a set of decision maker (DM) preferences over the criteria?’. Depending on the application in which MCDA is performed, the alternatives may represent strategies and policies for business, potential locations for setting up new facilities, designs of buildings, etc. The general objective of MCDA is to enable the DM to order all alternatives in order of the stated preferences, and choose the ones that are best, i.e., optimal with respect to the preferences over the criteria.

The preferences over the criteria typically represent the relative importance of the various criteria to the DM, which is typically expressed either in the form of an ordering (e.g., a ranking) over the criteria, or if available, a set of numeric weights on each criterion. Most of the existing quantitative multi-criteria decision methods (QnMCDMs) [70, 71] assume that it is possible to (a) completely rank the criteria in order of their importance, and/or (b) precisely quantify the degree of importance of criteria (e.g., as weights). Some formalisms such as Analytic Hierarchy Process (AHP) [75, 4], SMART [7], ELECTRE [6, 16, 43], TOPSIS [32, 27], VIKOR [62] and PROMETHEE [15, 16, 5] elicit relative importance of criteria using a matrix that specifies the pairwise ordering of the criteria, which is then used to generate weights for each criteria by a mathematical transformation [71]. In formal terms, this amounts to assuming that the preferences will always be a weak order [73]. The weak ordering is then typically transformed into a quantitative (numeric) scale, and the transformed importance values are then used to evaluate and compare candidates.

However, it is not surprising that generation of weights based on qualitative assessments of pairwise relative importance of criteria often leads to instability in rankings of alternatives and their high sensitivity to changing weights or addition of alternatives [14]

. In addition, DMs may not understand the assumptions involved in generating the weights. Moreover, in many applications, the DM’s preferences may be naturally incomplete and/or imprecise, especially when the stakes are really high (e.g., trade-offs in facility location for setting up new nuclear reactors). For example, a building designer as a DM may state that increased energy efficiency and pollution efficiency (lesser pollution) are more important than cost efficiency when evaluating candidate designs, without quantifying the relative importance between cost efficiency and either of the other criteria (imprecision), or without even specifying any relative importance between energy efficiency and pollution (incompleteness). A reason for such imprecision may be that precise weights for each criteria may not be available; another reason may be that the DM does not want to commit to precise weights a priori to avoid sensitivity of rankings to initial weight estimation errors.

In this article, we present and summarize a recently developed MCDA framework that orders the set of alternatives when the relative importance preferences are incomplete, imprecise, or qualitative in nature. To this end, we allow the DM to specify the preference for a decision problem with a set of criteria as a strict partial order over , i.e., the relative importance preference is represented by a binary relation . It is easy to see that such a criteria set can be readily represented as a directed acyclic graph. We discuss a dominance relation

recently developed in artificial intelligence literature

[29], which has the desirable property that it is a partial order whenever the relative importance preference is an interval order relation111For to be an interval order, it must be irreflexive, transitive, and for all and must imply that either or [1, 3]. The ordering of alternatives based on is provably correct, i.e., if , then it can be argued using principles of rational choice that A is indeed preferred to B. In that sense, the partial ordering produced by can be considered as a reference ordering that all other rankings must be consistent with. We discuss methods of comparing the orderings produced by and other MCDA ranking methods, drawing from recent advances in order theory.

2 Background

The following is the typical decision process for multi-criteria decision problems such as the above.

  1. First, the stakeholder defines a set of attributes or criteria on which the alternatives can be evaluated and compared. These criteria can be either quantitatively or qualitatively evaluated, e.g., the noise level during construction of a pavement type could be rated on a relative, ordinal scale (Low/Medium/High), or in terms of the maximum decibel level anticipated during construction.

  2. Next, the stakeholder uses his knowledge and experience to evaluate each alternative with respect to each of the attributes on a qualitative scale (e.g., ranking, partial order) or quantitative scale (e.g., utility function), such that the evaluation of two designs with respect to a criterion indicates their relative desirability. Further, the stakeholder may specify relative importance preferences over the criteria quantitatively in terms of numeric compensation (e.g., weights) for the criteria, or qualitatively (e.g., energy consumption is more important than noise levels).

  3. Finally, an appropriate multi-criteria decision method [72] is chosen, which takes into account the evaluations of the alternatives with respect to the criteria, as well as relative importance preference over the criteria and produces an ordering of the alternatives. Each decision method differs primarily in the set of rules of rational choice that it uses to compare alternatives, and hence the orderings they produce may accordingly differ.

Note that three factors are crucial to the validity and accuracy of the decision: a) the choice of whether each of the criteria will be evaluated on a qualitative or quantitative scale; b) the choice of whether the relative importance preference will be qualitatively or quantitatively specified; and c) the choice of the decision method which determines how the alternatives will be compared with respect to each other based on the stated preferences.

2.1 Need for a Qualitative Approach to MCDM

Quantitative MCDMs, i.e., QnMCDMs such as multi-attribute utility theory (MAUT)222The notion of utility automatically includes a notion of uncertainty

which is quantified in the form of probabilities. In this article, we do not consider settings where there is uncertainty over the outcomes of the alternative choices.

 [74] require the criteria evaluation scale and the relative importance preferences to be all quantitative. Some other MCDMs such as AHP [75], SMART, ELECTRE, TOPSIS, and other outranking methods (see [71]) that focus on certainty allow the stakeholder to specify qualitative input, but eventually convert them into quantitative scores according to some rules of assignment. All the above QnMCDMs apply quantitative decision rules to evaluate dominance, i.e., process (quantitatively elicited or qualitatively elicited and quantified) criteria evaluations and preferences in order to produce a ranking of the alternatives. However, in many design environments there is incompleteness and imprecision in the stakeholder’s evaluations and preferences over alternatives.

2.2 Limitations of QnMCDMs

The challenge in using QnMCDMs for decision making in design problems is that when three or more alternatives with multiple attributes are involved, the commonly employed techniques for normalizing evaluations, weighting the criteria and multi-attribute ranking cannot guarantee the selection of the best alternative in an unambiguous, rational, and consistent manner. While QnMCDMs have been largely successful when alternatives can be quantitatively evaluated with respect to criteria and weights can be precisely estimated, they are inappropriate for problems where some preferences are naturally qualitative as shown below.

  1. Incomplete Preferences: Stakeholders may assert that two alternatives are incomparable on the same criterion. Similarly, stakeholders may not always be able to totally rank the criteria [53, 8]. However QnMCDMs assign numeric weights to criteria, assuming that there is a total or weak ordering of the criteria. To deal with incompatibilities, different QnMCDMs extrapolate the stakeholder’s preferences and ‘fill in the blanks’ based on certain assumptions to obtain a complete ranking. This questions the validity of use and the accuracy of the results produced by QnMCDMs for the problem.

  2. Semantic Discrepancies: Numeric weights often have no meaning more than the relative ordering of criteria and the precise meaning of criteria weights is itself often not well understood by stakeholders [2, 18]. In particular, there is a disconnect between the intuitive meaning of the values in the scale as understood by the stakeholder versus the algorithmic meaning of the values in the same scale that is assumed by the QnMCDM while comparing alternatives [70].

  3. Imprecise Preferences: In some settings, stakeholder may be able to rank alternatives for each criterion and the criteria themselves on their importance, but may not want to commit on the strength of such preferences. Thus, evaluation of alternatives on a quantitative scale and weight estimation for criteria in QnMCDMs may involve assumptions extraneous to user input [53] regarding the extent or strength of preference, questioning the validity of the results.

  4. Complex Tradeoffs: Stakeholders may tradeoff one set of criteria against another (specify a partial order over sets of criteria). For instance, ‘Improve on both efficiency and cost, if possible, even if it increases pollution,’ i.e., tradeoff Z against both X and Y (but not just one). QnMCDMs are clearly inapplicable in such scenarios, as any weight assignment will violate such a preference.

  5. Possibility of manipulation: Rankings produced by QnMCDMs are sensitive to changes in the attribute weights, e.g., rank reversals are known to occur even when the attribute weights are kept constant [14]. This raises the question as to which set of weights is faithful to the preferences of the stakeholder, and makes the results of QnMCDMs vulnerable to manipulation.

  6. Lack of Transparency: Stakeholders are often unaware of the mathematical assumptions used in QnMCDMs and their implications on the decisions [18]. The aggregation of criteria based on numeric weights makes it difficult for stakeholders to understand the rationale behind a decision directly in terms of the stated preferences and rules based on principles of rational choice.

  7. Handling Component Dependencies: When each alternative is composed of multiple components, the preference over alternatives is a function of the preference over the components that make up the alternatives. Two components may make the alternative highly rated with respect to two different criteria, however when part of the same design, they may cancel out their contributions. For example, a certain choice of wall and paint for a building may be independently preferred, but the particular combination may release toxic gases raising safety concerns. QnMCDMs do not provide a way of accounting for component dependencies in such problems.

3 Qualitative Preference Reasoning based Decision Method

We now describe a recently developed qualitative decision making procedure that relies purely on the qualitative relative importance preferences specified by the decision maker and uses mathematical logic to compute dominance and obtain an optimal ordering of candidates. This procedure was first introduced and studied in [29, 19], and has also been applied to several application domains such as pavement design in civil engineering [36], requirements engineering [25], sustainable design [28], and materials science and engineering [60]. We now describe how the qualitative decision method computes dominance by formulating and solving logical equations in contrast to existing decision methods, and hence produces an ordering of alternatives based on principles of rational choice.

3.1 Qualitative Dominance

Let and be two candidates described by a set of properties or attributes. Suppose that is the intra-variable preference relation on the domain of . For example, if has its domain as the set of all real numbers then is often the natural total order on the set of real numbers. Further, let the binary relation represent the partial trade offs over attributes (relative importance among properties), e.g., Cost Performance means that Cost is more important than Performance. We say that dominates (equivalently, is preferred to, or is better than ), denoted whenever the following conditions hold:

  1. There exists at least one variable, called the witness , with respect to which is preferred to , i.e.,

  2. is better than or equal to with respect to all attributes except those that are less important than

If both the above conditions hold, then A is said to be preferred to B with respect to the relative importance tradeoffs of the stakeholder, i.e., . The above conditions can be succinctly expressed as an equation in mathematical logic notation:

The semantics or meaning of the above logical equation is as follows: states that the left hand side () holds true if and only if the logical formula in the right hand side holds true. The right hand side of the above equation states the existence of a witness property () such that is not worse than for all attributes that are not less important than .

The advantage of the above method is that it can handle imprecision in attribute values for alternatives. Hence, it is possible to order alternatives that specify attributes in terms of interval range (as opposed to precise values), or where they are estimated on a qualitative scale (very expensive, expensive, inexpensive, etc.).

3.2 Properties of the Dominance Relation

In order to apply any qualitative decision method to compare and choose among alternatives, it must be checked for conformance to well established principles of rational choice. It is expected that a good dominance relation would satisfy the properties of irreflexivity, transitivity and asymmetry, which makes it a strict partial order. We formally state the definitions of these properties.

Definition 1.

Properties of Strict Partial Order Relation [73]. A dominance relation is:

  1. irreflexive if for every alternative , never holds

  2. transitive if for all triples , and of alternatives, if and both hold, then

  3. asymmetric if then

Definition 2.

Properties of Interval Order Relation [1, 3]. A relative importance relation is an interval order if:

  1. is a partial order, and

  2. for all and implies that either or

It can easily be shown that an irreflexive and transitive relation is always asymmetric, and hence it suffices to show only that is irreflexive and transitive to show that is a strict partial order. We recall the propositions from [29], relating to the properties of and state below the main result that qualifies as a rational dominance relation.

Theorem 1.

When the input intra-variable preferences for each is strictly partially ordered and the relative importance (trade-offs) over the properties is interval ordered, the dominance relation defined in Section 3.1 is a strict partial order.

3.3 Computational Complexity of Qualitative Decision Making

The above described qualitative decision procedure involves more computational complexity to compute dominance and to rank the alternatives, in comparison with the 2-objective optimization methods or the MCDA methods like MAUT or AHP. In particular, the computational complexity for evaluating the logical equation in Section 3.1 for each pair of alternatives is of the order of where is the number of properties or attributes. Hence, if there are alternatives and properties from which the best alternative has to be selected with respect to tradeoffs, then it would require the order of computations to order alternatives, which is expensive (although possible with increased computational power). Despite the complexity, it may be worth using this qualitative method for several reasons. For example, in strategic decisions, one may need precision when incorrect ordering of two alternatives can have a large adverse impact to the decision maker. Another scenario that may need the rigor and precision of qualitative decision method is safety critical decisions such as site selection for harmful chemicals disposal.

4 Conclusion

There are several decision situations where the decision maker needs to order a set of alternatives from the best to the worst, purely based on possibly incomplete, qualitative attributes describing the alternatives, and possibly incomplete qualitative trade-offs among the attributes. We have described a case for using a qualitative decision method in such settings that provides a strict partial order of the alternatives as opposed to linear rankings provided by traditional quantitative decision methods. In the future we will examine the appropriateness of other advances in qualitative decision methods [49, 66, 29, 19, 20, 12] for different applications in engineering [11, 36, 65, 55, 59, 42, 44], optimization-based methods [65, 58, 45, 63, 64], cybersecurity [50, 51, 30], decision-support systems in healthcare [38, 40, 46, 47, 37, 39, 31, 22, 21, 24], ranking and recommendation systems [52, 34, 54, 56, 41, 35], service and cloud computing [9, 33, 17, 25, 10, 13, 26, 48], financial decision making [57], and other areas of computing [67, 68, 23, 69, 61].

References

  • [1] Peter C Fishburn. Interval graphs and interval orders. Discrete mathematics, 55(2):135–149, 1985.
  • [2] B. Schoner E.U. Choo and W.C. Wedley. Interpretation of criterion weights in multiple criteria calibration. Computers and Industrial Engineering Journal, 37:527–541, 1999.
  • [3] Amy Myers. Basic interval orders. Order, 16(3):261–275, 1999.
  • [4] Thomas L Saaty. Decision making—the analytic hierarchy and network processes (ahp/anp). Journal of systems science and systems engineering, 13(1):1–35, 2004.
  • [5] Jean-Pierre Brans and Bertrand Mareschal. Promethee methods. In Multiple criteria decision analysis: state of the art surveys, pages 163–186. Springer, 2005.
  • [6] José Figueira, Vincent Mousseau, and Bernard Roy. Electre methods. In Multiple criteria decision analysis: State of the art surveys, pages 133–153. Springer, 2005.
  • [7] Jyri Mustajoki, Raimo P Hämäläinen, and Ahti Salo. Decision support by interval smart/swing—incorporating imprecision in the smart and swing methods. Decision Sciences, 36(2):317–339, 2005.
  • [8] Ellen Peters, Daniel Västfjäll, Paul Slovic, C.K. Mertz, Ketti Mazzocco, and Stephan Dickert. Numeracy and decision making. Psychological Science, 17(5):407–413, 2006.
  • [9] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. On utilizing qualitative preferences in web service composition: A cp-net based approach. In Proceedings of the IEEE Congress on Services-Part I, pages 538–544. IEEE, 2008.
  • [10] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. TCP-Compose – A TCP-Net Based Algorithm for Efficient Composition of Web Services Using Qualitative Preferences. In Proceedings of the International Conference on Service-Oriented Computing–ICSOC 2008, pages 453–467. Springer Berlin Heidelberg, 2008.
  • [11] Hieu Q Pham, Ganesh Ram Santhanam, James D McCalley, and Vasant G Honavar. Bensoa: a flexible service-oriented architecture for power system asset management. In North American Power Symposium (NAPS), 2009, pages 1–6. IEEE, 2009.
  • [12] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. A dominance relation for unconditional multi-attribute preferences. In Computer Science Technical Reports. Paper 235. 2009. http://lib.dr.iastate.edu/cs_techreports/235.
  • [13] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Web service substitution based on preferences over non-functional attributes. In IEEE International Services Computing Conference, pages 210–217. IEEE, 2009.
  • [14] Katie Steele, Yohay Carmel, Jean Cross, and Chris Wilcox. Uses and misuses of multicriteria decision analysis (mcda) in environmental decision making. Risk Analysis, 29(1):26–33, 2009.
  • [15] Majid Behzadian, Reza B Kazemzadeh, Amir Albadvi, and Mohammad Aghdasi. Promethee: A comprehensive literature review on methodologies and applications. European journal of Operational research, 200(1):198–215, 2010.
  • [16] Olivier Cailloux. Electre and promethee mcda methods as reusable software components. In 25th Mini-EURO Conference on Uncertainty and Robustness in Planning and Decision Making (URPDM 2010), 2010.
  • [17] Zachary J Oster, Ganesh Ram Santhanam, and Samik Basu. Decomposing the service composition problem. In IEEE 8th European Conference on Web Services (ECOWS), pages 163–170. IEEE, 2010.
  • [18] Adam M Ross, M Gregory O’neill, Daniel E Hastings, and Donna H Rhodes. Aligning perspectives and methods for value-driven design. In AIAA Space, volume 2010, 2010.
  • [19] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Dominance testing via model checking. In Twenty-Fourth AAAI Conference on Artificial Intelligence, pages 357–362. AAAI Press, 2010.
  • [20] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Efficient dominance testing for unconditional preferences. In Twelfth International Conference on the Principles of Knowledge Representation and Reasoning, pages 590–592. AAAI Press, 2010.
  • [21] Ankit Agrawal and Alok Choudhary. Association rule mining based hotspot analysis on seer lung cancer data. International Journal of Knowledge Discovery in Bioinformatics (IJKDB), 2(2):34–54, 2011.
  • [22] Ankit Agrawal and Alok Choudhary. Identifying hotspots in lung cancer data using association rule mining. In 2nd IEEE ICDM Workshop on Biological Data Mining and its Applications in Healthcare (BioDM), pages 995–1002, 2011.
  • [23] Ankit Agrawal, Sanchit Misra, Daniel Honbo, and Alok Choudhary. Parallel pairwise statistical significance estimation of local sequence alignment using message passing interface library. Concurrency and Computation: Practice and Experience, 23(17):2269–2279, 2011.
  • [24] Ankit Agrawal, Sanchit Misra, Ramanathan Narayanan, Lalith Polepeddi, and Alok Choudhary. A lung cancer outcome calculator using ensemble data mining on seer data. In Proceedings of the Tenth International Workshop on Data Mining in Bioinformatics (BIOKDD), pages 1–9, New York, NY, USA, 2011. ACM.
  • [25] Zachary J Oster, Ganesh Ram Santhanam, and Samik Basu. Automating analysis of qualitative preferences in goal-oriented requirements engineering. In Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering, pages 448–451. IEEE Computer Society, 2011.
  • [26] Zachary J Oster, Ganesh Ram Santhanam, and Samik Basu. Identifying optimal composite services by decomposing the service composition problem. In 2011 IEEE International Conference on Web Services (ICWS), pages 267–274. IEEE, 2011.
  • [27] D Ozturk and F Batuk. Technique for order preference by similarity to ideal solution (topsis) for spatial decision problems. International Society for Photogrammetry and Remote Sensing, 2011.
  • [28] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Identifying sustainable designs using preferences over sustainability attributes. In 2011 AAAI Spring Symposium Series, 2011.
  • [29] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Representing and reasoning with qualitative preferences for compositional systems. Journal of Artificial Intelligence Research (JAIR), 42:211–274, 2011.
  • [30] Ganesh Ram Santhanam, Yuly Suvorov, Samik Basu, and Vasant Honavar. Verifying intervention policies to counter infection propagation over networks: A model checking approach. In Twenty-Fifth AAAI Conference on Artificial Intelligence, pages 1408–1414. AAAI Press, 2011.
  • [31] Ankit Agrawal, Sanchit Misra, Ramanathan Narayanan, Lalith Polepeddi, and Alok Choudhary. Lung cancer survival prediction using ensemble data mining on seer data. Scientific Programming, 20(1):29–42, 2012.
  • [32] Majid Behzadian, S Khanmohammadi Otaghsara, Morteza Yazdani, and Joshua Ignatius. A state-of the-art survey of topsis applications. Expert Systems with Applications, 39(17):13051–13069, 2012.
  • [33] Zachary J Oster, Syed Adeel Ali, Ganesh Ram Santhanam, Samik Basu, and Partha S Roop. A service composition framework based on goal-oriented requirements engineering, model checking, and qualitative preference analysis. In International Conference on Service-Oriented Computing, pages 283–297. Springer Berlin Heidelberg, 2012.
  • [34] Yusheng Xie, Yu Cheng, Daniel Honbo, Kunpeng Zhang, Ankit Agrawal, and Alok Choudhary. Crowdsourcing recommendations from social sentiment. In Proceedings of KDD Workshop on Issues of Sentiment Discovery and Opinion Mining (WISDOM), pages 9:1–9:8. ACM, 2012.
  • [35] Yusheng Xie, Yu Cheng, Daniel Honbo, Kunpeng Zhang, Ankit Agrawal, Alok Choudhary, Yi Gao, and Jiangtao Gou. Probabilistic macro behavioral targeting. In Proceedings of the CIKM workshop on Data-driven user behavioral modelling and mining from social media (DUBMMSM), pages 7–10. ACM, 2012.
  • [36] Pavement life-cycle sustainability assessment and interpretation using a novel qualitative decision procedure. Journal of Computing in Civil Engineering, 27(5):544–554, 2013.
  • [37] Ankit Agrawal, Reda Al-Bahrani, Ryan Merkow, Karl Bilimoria, and Alok Choudhary. Colon surgery outcome prediction using acs nsqip data. In Proceedings of the KDD Workshop on Data Mining for Healthcare (DMH), pages 1–6, 2013.
  • [38] Ankit Agrawal, Reda Al-Bahrani, Jaishankar Raman, Mark J. Russo, and Alok Choudhary. Lung transplant outcome prediction using unos data. In Proceedings of the IEEE Big Data Workshop on Bioinformatics and Health Informatics (BHI), pages 1–8, 2013.
  • [39] Ankit Agrawal, Jaishankar Raman, Mark J. Russo, and Alok Choudhary. Heart transplant outcome prediction using unos data. In Proceedings of the KDD Workshop on Data Mining for Healthcare (DMH), pages 1–6, 2013.
  • [40] Reda Al-Bahrani, Ankit Agrawal, and Alok Choudhary. Colon cancer survival prediction using ensemble data mining on seer data. In Proceedings of the IEEE Big Data Workshop on Bioinformatics and Health Informatics (BHI), pages 9–16, 2013.
  • [41] Yu Cheng, Yusheng Xie, Zhengzhang Chen, Ankit Agrawal, Alok Choudhary, and Songtao Guo. Jobminer: A real-time system for mining job-related patterns from social media. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD), pages 1450–1453, 2013.
  • [42] PD Deshpande, BP Gautham, A Cecen, S Kalidindi, Ankit Agrawal, and A Choudhary.

    Application of statistical and machine learning techniques for correlating properties to composition and manufacturing processes of steels.

    In 2nd World Congress on Integrated Computational Materials Engineering, July 7-11, 2013, Salt Lake City, Utah, pages 155–160, 2013.
  • [43] José Rui Figueira, Salvatore Greco, Bernard Roy, and Roman Słowiński. An overview of electre methods and their recent extensions. Journal of Multi-Criteria Decision Analysis, 20(1-2):61–85, 2013.
  • [44] Kasthurirangan Gopalakrishnan, Ankit Agrawal, Halil Ceylan, Sunghwan Kim, and Alok Choudhary. Knowledge discovery and data mining in pavement inverse analysis. Transport, 28(1):1–10, 2013.
  • [45] Chen Jin, Qiang Fu, Huahua Wang, Ankit Agrawal, William Hendrix, Wei-keng Liao, Md. Mostofa Ali Patwary, Arindam Banerjee, and Alok Choudhary.

    Solving combinatorial optimization problems using relaxed linear programming: A high performance computing perspective.

    In Proceedings of the KDD Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications (BigMine), pages 39–46, 2013.
  • [46] Lu Liu, Jie Tang, Yu Cheng, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. Mining diabetes complication and treatment patterns for clinical decision support. In Proceedings of 22th ACM International Conference on Information and Knowledge Management (CIKM 2013), San Francisco, USA, Oct. 2013, pages 279–288, 2013.
  • [47] Jason Scott Mathias, Ankit Agrawal, Joe Feinglass, Andrew J Cooper, David William Baker, and Alok Choudhary. Development of a 5 year life expectancy index in older adults using predictive mining of electronic health record data. Journal of the American Medical Informatics Association, 20:e118–e124, 2013.
  • [48] Zachary J Oster, Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Model checking of qualitative sensitivity preferences to minimize credential disclosure. In Formal Aspects of Component Software, pages 205–223. Springer Berlin Heidelberg, 2013.
  • [49] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Preference based service adaptation using service substitution. In IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), volume 1, pages 487–493. IEEE, 2013.
  • [50] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. Verifying preferential equivalence and subsumption via model checking. In International Conference on Algorithmic Decision Theory, pages 324–335. Springer Berlin Heidelberg, 2013.
  • [51] Ganesh Ram Santhanam, Zachary J. Oster, and Samik Basu. Identifying a preferred countermeasure strategy for attack graphs. In Proceedings of the Eighth Annual Cyber Security and Information Intelligence Research Workshop, CSIIRW ’13, pages 11:1–11:4. ACM, 2013.
  • [52] Yusheng Xie, Zhengzhang Chen, Kunpeng Zhang, Yu Cheng, Chen Jin, Ankit Agrawal, and Alok Choudhary. Elver: Recommending facebook pages in cold start situation without content features. In Proceedings of IEEE International Conference on Big Data (IEEE BigData 2013), pages 475–479, 2013.
  • [53] Denys Yemshanov, Frank H. Koch, Yakov Ben-Haim, Marla Downing, Frank Sapio, and Marty Siltanen. A new multicriteria risk mapping approach based on a multiattribute frontier concept. Risk Analysis, 2013.
  • [54] Kunpeng Zhang, Doug Downey, Zhengzhang Chen, Yusheng Xie, Yu Cheng, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. A probabilistic graphical model for brand reputation assessment in social networks. In Proceedings of IEEE/ACM International Conference on Social Networks Analysis and Mining (ASONAM), Niagara Falls, Canada, Aug. 2013, pages 223–230, 2013.
  • [55] Ankit Agrawal, Parijat D Deshpande, Ahmet Cecen, Gautham P Basavarsu, Alok N Choudhary, and Surya R Kalidindi.

    Exploration of data science techniques to predict fatigue strength of steel from composition and processing parameters.

    Integrating Materials and Manufacturing Innovation, 3(8):1–19, 2014.
  • [56] Yu Cheng, Ankit Agrawal, Huan Liu, and Alok Choudhary. Social role identification via dual uncertainty minimization regularization. In Proceedings of International Conference on Data Mining (ICDM), pages 767–772, 2014.
  • [57] Ruoqian Liu, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. Enhancing financial decision-making using social behavior modeling. In Proceedings of 8th KDD Workshop on Social Network Mining and Analysis for Business, Consumer and Social Insights (SNAKDD), pages 13:1–13:5, 2014. Article No. 13.
  • [58] Ruoqian Liu, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. Search space preprocessing in solving complex optimization problems. In Proceedings of IEEE BigData Workshop on Complexity for Big Data (C4BD), pages 1–5, 2014.
  • [59] Bryce Meredig, Ankit Agrawal, S Kirklin, J E Saal, J W Doak, A Thompson, Kunpeng Zhang, Alok Choudhary, and Christopher Wolverton. Combinatorial screening for new materials in unconstrained composition space with machine learning. Physical Review B, 89(094104):1–7, 2014.
  • [60] Ganesh Ram Santhanam, Pallavi Dubey, Srikant Srinivasan, Scott Broderick, and Krishna Rajan. A decision theoretic framework for materials selection in the presence of uncertainty. In Materials Research Society Fall Meeting. https://mrsfall14.zerista.com/event/member/146946, 2014.
  • [61] Seung Woo Son, Zhengzhang Chen, William Hendrix, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. Data compression for the exascale computing era - survey. Supercomputing Frontiers and Innovations, 1(2):76–88, 2014.
  • [62] Morteza Yazdani and Felipe R Graeml. Vikor and its applications: A state-of-the-art survey. International Journal of Strategic Decision Sciences (IJSDS), 5(2):56–83, 2014.
  • [63] Chen Jin, Qiang Fu, Huahua Wang, William Hendrix, Zhengzhang Chen, Ankit Agrawal, Arindam Banerjee, and Alok Choudhary. Running map inference on million node graphical models: A high performance computing perspective. In Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pages 565–575, 2015.
  • [64] Ruoqian Liu, Ankit Agrawal, Zhengzhang Chen, Wei-keng Liao, and Alok Choudhary.

    Pruned search: A machine learning based meta-heuristic approach for constrained continuous optimization.

    In Proceedings of 8th IEEE International Conference on Contemporary Computing (IC3), 2015. To appear.
  • [65] Ruoqian Liu, Abhishek Kumar, Zhengzhang Chen, Ankit Agrawal, Veera Sundararaghavan, and Alok Choudhary. A predictive machine learning approach for microstructure optimization and materials design. Nature Scientific Reports, 5(11551), 2015.
  • [66] Ganesh Ram Santhanam, Samik Basu, and Vasant Honavar. CRISNER: A Practically Efficient Reasoner for Qualitative Preferences. 2015. eprint arXiv:1507.08559 [cs.AI] URL: http://arxiv.org/abs/1507.08559.
  • [67] Ganesh Ram Santhanam, Gopalakrishnan Sivaprakasam, Giora Slutzki, and Samik Basu. A knowledge based framework for case-specific diagnosis. In International Conference on Agents and Artificial Intelligence, 2015.
  • [68] Ankit Agrawal, Alok Choudhary, and Xiaoqiu Huang. Sequence-Specific Sequence Comparison Using Pairwise Statistical Significance, volume 696 of Advances in Experimental Medicine and Biology, AEMB, pages 297–306. Springer, 2011.
  • [69] Ankit Agrawal, Mostofa Patwary, William Hendrix, Wei-keng Liao, and Alok Choudhary. High performance big data clustering, pages 192–211. IOS Press, 2013.
  • [70] Valerie Belton and Theodor Stewart. Multiple criteria decision analysis: an integrated approach. Springer Science & Business Media, 2002.
  • [71] J. Figueira, S. Greco, and M. Ehrgott. Multiple criteria decision analysis: state of the art surveys, volume 78. Springer Verlag, 2005.
  • [72] J. Figueira, S. Greco, and M. Ehrgott. Multiple Criteria Decision Analysis: State of the Art Surveys. Springer Verlag, Boston, Dordrecht, London, 2005.
  • [73] P.C. Fishburn. Utility Theory for Decision Making. Wiley, New York, 1970.
  • [74] R. L. Keeney and H. Raiffa. Decisions with Multiple Objectives: Preferences and Value Trade-Offs. Cambridge University Press, 1993.
  • [75] T.L. Saaty. Fundamentals of the Analytic Hierarchy Process. RWS Publications, 4922 Ellsworth Avenue, Pittsburgh, PA 15413, 2000.