Selection Procedures for Simulations with Multiple Constraints under Independent and Correlated Sampling

Abstract

We consider the problem of selecting the best feasible system with constraints on multiple secondary performance measures. We develop fully sequential indifference-zone procedures to solve this problem that guarantee a nominal probability of correct selection. In addition, we address two issues critical to the efficiency of these procedures: namely, the allocation of error between feasibility determination and selection of the best system, and the use of Common Random Numbers. We provide a recommended error allocation as a function of the number of constraints, supported by an experimental study and an approximate asymptotic analysis. The validity and efficiency of the new procedures with independent and CRN are demonstrated through both analytical and experimental results.

References

1. Sigrún Andradóttir and Seong-Hee Kim. 2010. Fully sequential procedures for comparing constrained systems via simulation. Naval Research Logistics 57, 5, 403--421.
2. Demet Batur and Seong-Hee Kim. 2010. Finding feasible systems in the presence of constraints on multiple performance measures. ACM Transactions on Modeling and Computer Simulation 20, 3, Article 13.
3. Chun-Hung Chen. 1996. A lower bound for the correct subset-selection probability and its application to discrete-event system simulations. IEEE Transactions on Automatic Control 41, 8, 1227--1231.
4. Hsiao-Chang Chen, Chun-Hung Chen, and Enver Yücesan. 2000. Computing efforts allocation for ordinal optimization and discrete event simulation. IEEE Transactions on Automatic Control 45, 5, 960--964.
5. Stephen E. Chick. 2006. Subjective probability and Bayesian methodology. In Shane G. Henderson and Barry L. Nelson (Eds.), Handbooks in Operations Research and Management Science (Vol. 13). Elsevier, 225--257.
6. Stephen E. Chick and Koichiro Inoue. 2001. New procedures to select the best simulated system using common random numbers. Management Science 47, 8, 1133--1149.
7. Edward J. Dudewicz and Siddhartha R. Dalal. 1975. Allocation of observations in ranking and selection with unequal variances. Sankhya 37, 1, 28--78.
8. Peter I. Frazier and Aleksandr M. Kazachkov. 2011. Guessing preferences: A new approach to multi-attribute ranking and selection. In Proceedings of the Winter Simulation Conference (WSC'11). 4324--4336.
9. Peter I. Frazier, Jing Xie, and Stephen E. Chick. 2011. Value of information methods for pairwise sampling with correlations. In Proceedings of the Winter Simulation Conference (WSC'11). 3979--3991.
10. Mark Hartmann. 1991. An improvement on Paulson's procedure for selecting the population with the largest mean from k normal populations with a common unknown variance. Sequential Analysis 10, 1--2 (1991), 1--16.
11. Christopher M. Healey, Sigrún Andradóttir, and Seong-Hee Kim. 2013. Efficient comparison of constrained systems using dormancy. European Journal of Operational Research 224, 2, 340--352.
12. Christopher M. Healey, Sigrún Andradóttir, and Seong-Hee Kim. 2014. A Minimal Switching Procedure for Constrained Ranking and Selection under Independent or Common Random Numbers. Technical Report.
13. Susan R. Hunter, Chun-Hung Chen, Raghu Pasupathy, Nugroho Artadi Pujowidianto, Loo Hay Lee, and Chee Meng Yap. 2011. Optimal sampling laws for constrained simulation optimization on finite sets: The bivariate normal case. In Proceedings of the Winter Simulation Conference (WSC'11). 4294--4302.
14. Susan R. Hunter and Raghu Pasupathy. 2013. Optimal sampling laws for stochastically constrained simulation optimization on finite sets. INFORMS Journal on Computing 25, 3, 527--542.
15. Alireza Kabirian and Sigurdur Olafsson. 2009. Selection of the best with stochastic constraints. In Proceedings of the Winter Simulation Conference (WSC'09). 574--583.
16. Seong-Hee Kim and Barry L. Nelson. 2001. A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation 11, 3 (July 2001), 251--273.
17. Seong-Hee Kim and Barry L. Nelson. 2006. Selecting the best system. In Shane G. Henderson and Barry L. Nelson (Eds.), Handbooks in Operations Research and Management Science (Vol. 13). Elsevier, 501--534.
18. Averill M. Law and David M. Kelton. 2000. Simulation Modeling and Analysis (3rd ed.). McGraw-Hill Higher Education.
19. Loo Hay Lee, Ek Peng Chew, Suyan Teng, and David Goldsman. 2010. Finding the non-dominated Pareto set for multi-objective simulation models. IIE Transactions 42, 9, 656--674.
20. Vadim Lesnevski, Barry L. Nelson, and Jeremy Staum. 2007. Simulation of coherent risk measures based on generalized scenarios. Management Science 53, 11, 1756--1769.
21. Douglas J. Morrice and John C. Butler. 2006. Ranking and selection with multiple “targets.” In Proceedings of the Winter Simulation Conference (WSC'06). 222--230.
22. Barry L. Nelson. 2010. Optimization via simulation over discrete decision variables. TutORials in Operations Research (Vol. 7). INFORMS, 965, 193--207.
23. Barry L. Nelson and Frank J. Matejcik. 1995. Using common random numbers for indifference-zone selection and multiple comparisons in simulation. Management Science 41, 12, 1935--1945.
24. Edward Paulson. 1964. A sequential procedure for selecting the population with the largest mean from k normal populations. Annals of Mathematical Statistics 35, 1, 174--180.
25. S. K. Perng. 1969. A comparison of the asymptotic expected sample sizes of two sequential procedures for ranking problem. Annals of Mathematical Statistics 40, 6, 2198--2202.
26. Juta Pichitlamken, Barry L. Nelson, and L. Jeff Hong. 2006. A sequential procedure for neighborhood selection-of-the-best in optimization via simulation. European Journal of Operational Research 173, 1, 283--298.
27. Nugroho Artadi Pujowidianto, Loo Hay Lee, Chun-Hung Chen, and Chee Meng Yap. 2009. Optimal computing budget allocation for constrained optimization. In Proceedings of the Winter Simulation Conference (WSC'09). 584--589.
28. Yosef Rinott. 1978. On two-stage selection procedures and related probability-inequalities. Communications in Statistics: Theory and Methods 7, 8, 799--811.
29. Thomas J. Santner and Ajit C. Tamhane. 1984. Designing experiments for selecting a normal population with a large mean and a small variance. In Thomas J. Santner and Ajit C. Tamhane (Eds.), Design of Experiments—Ranking and Selection: Essays in Honor of Robert E. Bechhofer. Marcel-Dekker, 178--198.
30. Roberto Szechtman and Enver Yücesan. 2008. A new perspective on feasibility determination. In Proceedings of the Winter Simulation Conference (WSC'08). 273--280.

Index Terms

• Computing methodologies

  • Modeling and simulation

    • Simulation evaluation

    • Simulation types and techniques