skip to main content
research-article

Sorting and Selection with Imprecise Comparisons

Published:17 November 2015Publication History
Skip Abstract Section

Abstract

We consider a simple model of imprecise comparisons: there exists some δ > 0 such that when a subject is given two elements to compare, if the values of those elements (as perceived by the subject) differ by at least δ, then the comparison will be made correctly; when the two elements have values that are within δ, the outcome of the comparison is unpredictable. This model is inspired by both imprecision in human judgment of values and also by bounded but potentially adversarial errors in the outcomes of sporting tournaments. Our model is closely related to a number of models commonly considered in the psychophysics literature where δ corresponds to the Just Noticeable Difference (JND) unit or difference threshold. In experimental psychology, the method of paired comparisons was proposed as a means for ranking preferences among n elements of a human subject. The method requires performing all (n2) comparisons, then sorting elements according to the number of wins. The large number of comparisons is performed to counter the potentially faulty decision-making of the human subject, who acts as an imprecise comparator. We show that in our model the method of paired comparisons has optimal accuracy, minimizing the errors introduced by the imprecise comparisons. However, it is also wasteful because it requires all (n2). We show that the same optimal guarantees can be achieved using 4n3/2 comparisons, and we prove the optimality of our method. We then explore the general tradeoff between the guarantees on the error that can be made and number of comparisons for the problems of sorting, max-finding, and selection. Our results provide strong lower bounds and close-to-optimal solutions for each of these problems.

References

  1. Gagan Aggarwal, Nir Ailon, Florin Constantin, Eyal Even-Dar, Jon Feldman, Gereon Frahling, Monika R. Henzinger, S. Muthukrishnan, Noam Nisan, Martin Pál, Mark Sandler, and Anastasios Sidiropoulos. 2008. Theory research at Google. SIGACT News 39, 2 (2008), 10--28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Noga Alon and Yossi Azar. 1988. Sorting, approximate sorting, and searching in rounds. SIAM Journal on Discrete Math 1, 3 (1988), 269--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Shay Assaf and Eli Upfal. 1991. Fault tolerant sorting networks. SIAM Journal on Discrete Math 4, 4 (1991), 472--480. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Michael Ben-Or and Avinatan Hassidim. 2008. The bayesian learner is optimal for noisy binary search (and pretty good for quantum as well). In Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS). 221--230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Manuel Blum, Robert W. Floyd, Vaughan R. Pratt, Ronald L. Rivest, and Robert E. Tarjan. 1973. Time bounds for selection. Journal of Computer and Systems Science 7, 4 (1973), 448--461. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Béla Bollobás and Andrew Thomason. 1983. Parallel sorting. Discrete Applied Mathematics 6 (1983), 1--11.Google ScholarGoogle ScholarCross RefCross Ref
  7. Ryan S. Borgstrom and S. Rao Kosaraju. 1993. Comparison-based search in the presence of errors. In Proceedings of the 25th Annual ACM Symposium on Theory of Computing (STOC). 130--136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ferdinando Cicalese. 2013. Fault-Tolerant Search Algorithms - Reliable Computation with Unreliable Information. Springer. 1--198 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Herbert Aron David. 1988. The Method of Paired Comparisons (2nd ed.). Charles Griffin & Company Limited.Google ScholarGoogle Scholar
  10. Uriel Feige, Prabhakar Raghavan, David Peleg, and Eli Upfal. 1994. Computing with noisy information. SIAM Journal on Computing 23, 5 (1994). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Vitaly Feldman. 2009. Robustness of evolvability. In Proceedings of COLT. 277--292.Google ScholarGoogle Scholar
  12. Irene Finocchi, Fabrizio Grandoni, and Giuseppe F. Italiano. 2009. Optimal resilient sorting and searching in the presence of memory faults. Theoretical Computer Science 410 (2009), 4457--4470. Issue 44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Irene Finocchi and Giuseppe F. Italiano. 2004. Sorting and searching in the presence of memory faults (without redundancy). In Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC). 101--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. William I. Gasarch, Evan Golub, and Clyde P. Kruskal. 2003. Constant time parallel sorting: An empirical view. Journal of Computer and System Science 67, 1 (2003), 63--91. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Roland Häggkvist and Pavol Hell. 1981. Parallel sorting with constant time for comparisons. SIAM Journal on Computing 10, 3 (1981), 465--472.Google ScholarGoogle ScholarCross RefCross Ref
  16. Roland Häggkvist and Pavol Hell. 1982. Sorting and merging in rounds. SIAM Journal on Algebraic Discrete Methods 3, 4 (1982), 465--473.Google ScholarGoogle ScholarCross RefCross Ref
  17. Richard M. Karp and Robert Kleinberg. 2007. Noisy binary search and its applications. In SODA. 881--890. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Michael Kearns. 1998. Efficient noise-tolerant Learning from statistical queries. Journal of the ACM 45, 6 (1998), 983--1006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. H. G. Landau. 1953. On dominance relations and the structure of animal societies: III. The condition for a score structure. The Bulletin of Mathematical Biophysics 15, 2 (1953), 143--148. DOI:http://dx.doi.org/10.1007/BF02476378Google ScholarGoogle ScholarCross RefCross Ref
  20. László Lovász. 1986. An Algorithmic Theory of Numbers, Graphs, and Convexity. CBMS-NSF Regional Conference Series in Applied Mathematics 50, SIAM.Google ScholarGoogle ScholarCross RefCross Ref
  21. Stephen B. Maurer. 1980. The king chicken theorems. Mathematics Magazine 53, 2 (March 1980), 67--80.Google ScholarGoogle ScholarCross RefCross Ref
  22. Andrzej Pelc. 2002. Searching games with errors—fifty years of coping with liars. Theoretical Computer Science 270, 1--2 (2002), 71--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bala Ravikumar, K. Ganesan, and K. B. Lakshmanan. 1987. On selecting the largest element in spite of erroneous information. In Proceedings of the 4th Annual Symposium on Theoretical Aspects of Computer Science (STACS). 88--99. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Alfréd Rényi. 1962. On a problem in information theory. Magyar Tud. Akad. Mat. Kutató Int. Közl 6 (1962), 505--516.Google ScholarGoogle Scholar
  25. Ronald L. Rivest, Albert R. Meyer, Daniel J. Kleitman, Karl Winklmann, and Joel Spencer. 1980. Coping with errors in binary search procedures. Journal of Computer and System Science 20, 3 (1980), 396--405.Google ScholarGoogle ScholarCross RefCross Ref
  26. Jian Shen, Li Sheng, and Jie Wu. 2003. Searching for sorted sequences of kings in tournaments. SIAM Journal on Computing 32, 5 (2003), 1201--1209. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Scott M. Smith and Gerald S. Albaum. 2005. Fundamentals of Marketing Research (1st ed.). Sage Publications, Inc.Google ScholarGoogle Scholar
  28. Louis Leon Thurstone. 1927. A law of comparative judgment. Psychological Review 34 (1927), 273--286.Google ScholarGoogle ScholarCross RefCross Ref
  29. Stanisław Marcin Ulam. 1976. Adventures of a Mathematician. Scribner’s, New York.Google ScholarGoogle Scholar
  30. Leslie G. Valiant. 1975. Parallelism in comparison problems. SIAM Journal on Computing 4, 3 (1975), 348--355.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Andrew Chi-Chih Yao and Frances Foong Yao. 1985. On fault-tolerant networks for sorting. SIAM Journal on Computing 14, 1 (1985), 120--128.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Sorting and Selection with Imprecise Comparisons

      Recommendations

      Reviews

      M. Rahman

      Sorting and selection with imprecise comparisons has long been the focus of extensive research attention among theoreticians. There have been a number of models and frameworks of imprecision considered in the literature. In this paper, the authors study the following simple model: when comparing two elements, the comparison is made correctly if the values of those elements differ by at least some constant δ>0; otherwise, the outcome of the comparison is unpredictable. The authors here present a number of interesting algorithmic results, as well as some useful lower bounds. They first present a deterministic max-finding algorithm with error 2 using 2 n 3/2 comparisons. Using this algorithm recursively, they present deterministic algorithms with error k that require comparisons. They further prove a lower bound of comparisons for the problem. They also present a linear-time randomized algorithm that achieves error 3 with probability at least 1 - 1/ n 2, establishing that randomization greatly changes the complexity of the problem. Subsequently, the authors prove that the selection of an element of any order i can be achieved using comparisons, and sorting with error k can be done using comparisons. They also prove a lower bound of comparisons for sorting with error k . All of the algorithms have the same time order as the number of comparisons they make. Notably, to obtain the algorithms for larger error k , the authors use several different ways to partition elements, recursively apply algorithms for smaller error, and then finally combine results. As noted by the authors, a similar approach to finding a small set of "good" elements was used by Borgstrom and Kosaraju [1] in the context of noisy binary search. The model studied by the authors is inspired by both imprecision in human judgment of values and also by bounded but potentially adversarial errors in sporting tournaments, which make it all the more interesting and useful. In summary, the results achieved by the authors basically establish that there exist algorithms that are robust to imprecision in comparisons while using substantially fewer comparisons than the naive methods. Additionally, the deterministic algorithms presented here can be used in another interesting well-studied problem of finding a k -king in any tournament graph while minimizing the number of edges checked. Finally, the authors list a number of interesting and natural open problems. Online Computing Reviews Service

      Access critical reviews of Computing literature here

      Become a reviewer for Computing Reviews.

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Algorithms
        ACM Transactions on Algorithms  Volume 12, Issue 2
        February 2016
        385 pages
        ISSN:1549-6325
        EISSN:1549-6333
        DOI:10.1145/2846106
        Issue’s Table of Contents

        Copyright © 2015 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 November 2015
        • Revised: 1 December 2014
        • Accepted: 1 December 2014
        • Received: 1 July 2013
        Published in talg Volume 12, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader