Tobias Helms
Roland Ewald
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The state and structure of a model may vary during a simulation and, thus, also its computational demands. Adapting simulation algorithms to these demands at runtime can therefore improve their performance. While this is a general and cross-cutting concern, only few simulation systems offer reusable support for this kind of runtime adaptation. We present a flexible and generic mechanism for the runtime adaptation of component-based simulation algorithms. It encapsulates simulation algorithms applicable to a given problem and employs reinforcement learning to explore the algorithms’ performance during a simulation run. We evaluate our approach on a modeling formalism from computational biology and on a benchmark model defined in PDEVS, thereby investigating a broad range of options for improving its learning capabilities.
Supplemental Material
Available for Download
pdf
a7-helms-apndx.pdf (199.7 KB)
Online Appendix to: Automatic Runtime Adaptation for Component-based Simulation Algorithms
- Ryan P. Adams and David J. C. MacKay. 2007. Bayesian Online Changepoint Detection. Technical Report. University of Cambridge. arXiv:0710.3742{stat.ML}Google Scholar
- Warren Armstrong, Peter Christen, Eric McCreath, and Alistair P. Rendell. 2006. Dynamic algorithm selection using reinforcement learning. In Proceedings of the International Workshop on Integrating AI and Data Mining (AIDM’06). 18--25. DOI:http://dx.doi.org/10.1109/AIDM.2006.4 Google ScholarDigital Library
- Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Machine Learning 47, 2--3 (May 2002), 235--256. DOI:http://dx.doi.org/10.1023/A:1013689704352 Google ScholarDigital Library
- Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. 1995. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science. 322--331. DOI:http://dx.doi.org/10.1109/SFCS.1995.492488 Google ScholarDigital Library
- Diana M. Ball and Stephen M. Hoyt. 1990. The adaptive Time-Warp concurrency control algorithm. In Proceedings of the SCS Multiconference on Distributed Simulation. 174--177.Google Scholar
- Andrea Bonarini, Alessandro Lazaric, and Marcello Restelli. 2007. Reinforcement learning in complex environments through multiple adaptive partitions. In AI*IA 2007: Artificial Intelligence and Human-Oriented Computing. Vol. 4733. Springer Berlin Heidelberg, 531--542. DOI:http://dx.doi.org/10.1007/978-3-540-74782-6_46 Google ScholarDigital Library
- Azzedine Boukerche and Sajal K. Das. 1997. Dynamic load balancing strategies for conservative parallel simulations. In Proceedings of the 11th Workshop on Parallel and Distributed Simulation (PADS’97). 20--28. DOI:http://dx.doi.org/10.1145/268826.268897 Google ScholarDigital Library
- François E. Cellier and Ernesto Kofman. 2010. Continuous System Simulation. Springer.Google Scholar
- Samir R. Das. 1996. Adaptive protocols for parallel discrete-event simulation. In Proceedings of the Winter Simulation Conference (WSC’96). 186--193. DOI:http://dx.doi.org/10.1109/WSC.1996.873277 Google ScholarDigital Library
- Olivier Decroly and Albert Goldbeter. 1982. Birhythmicity, chaos, and other patterns of temporal self-organization in a multiply regulated biochemical system. In Proceedings of the National Academy of Sciences USA. 6917--6921.Google ScholarCross Ref
- Roland Ewald. 2010. Automatic Algorithm Selection for Complex Simulation Problems. Ph.D. Dissertation. University of Rostock.Google Scholar
- Roland Ewald, Stefan Leye, and Adelinde M. Uhrmacher. 2009. An efficient and adaptive mechanism for parallel simulation replication. In Proceedings of the 23rd Workshop on Principles of Advanced and Distributed Simulation (PADS’09). 104--113. DOI:http://dx.doi.org/10.1109/PADS.2009.11 Google ScholarDigital Library
- Richard M. Fujimoto and Maria Hybinette. 1997. Computing global virtial time in shared-memory multiprocessors. ACM Transactions on Modeling and Computer Simulation (TOMACS) 7, 4, 425--446. DOI:http://dx.doi.org/10.1145/268403.268404 Google ScholarDigital Library
- Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-oriented Software. Addison-Wesley Professional. Google ScholarDigital Library
- Daniel T. Gillespie. 1977. Exact stochastic simulation of coupled chemical reactions. Journal of Physical Chemistry 81, 25, 2340--2361. DOI:http://dx.doi.org/10.1021/j100540a008Google ScholarCross Ref
- Carla P. Gomes and Bart Selman. 2001. Algorithm portfolios. Artificial Intelligence 126, 1--2, 43--62. Google ScholarDigital Library
- Fiete Haack, Heiko Lemcke, Roland Ewald, Tareck Rharass, and Adelinde M. Uhrmacher. 2015. Spatio-temporal model of endogenous ROS and Raft-Dependent WNT/Beta-catenin signaling driving cell fate commitment in human neural progenitor cells. PLoS Computational Biology 11, 3, e1004106.Google ScholarCross Ref
- Tobias Helms. 2012. Adaptive Laufzeit-Konfiguration von ML-Rules Simulationen. Master’s thesis. University of Rostock.Google Scholar
- Tobias Helms, Roland Ewald, Stefan Rybacki, and Adelinde M. Uhrmacher. 2013. A generic adaptive simulation algorithm for component-based simulation systems. In Proceedings of the 27th Workshop on Principles of Advanced and Distributed Simulation (PADS’13). 11--22. DOI:http://dx.doi.org/10.1145/2486092.2486095 Google ScholarDigital Library
- Tobias Helms, Jan Himmelspach, Carsten Maus, Olivier Röwer, Johannes Schützel, and Adelinde M. Uhrmacher. 2012. Toward a language for the flexible observation of simulations. In Proceedings of the Winter Simulation Conference (WSC’12). 418:1--418:12. Google ScholarDigital Library
- Tobias Helms, Martin Luboschik, Heidrun Schumann, and Adelinde M. Uhrmacher. 2013. An approximate execution of rule-based multi-level models. In Proceedings of the 11th International Conference on Computational Methods in Systems Biology (CMSB’13). 19--32.Google Scholar
- Jan Himmelspach and Adelinde M. Uhrmacher. 2007a. Plug’n simulate. In Proceedings of the 40th Annual Simulation Symposium (ANSS’07). 137--143. DOI:http://dx.doi.org/10.1109/ANSS.2007.34 Google ScholarDigital Library
- Jan Himmelspach and Adelinde M. Uhrmacher. 2007b. The event queue problem and PDEVS. In Proceedings of the SpringSim’07, DEVS Integrative M&S Symposium. 257--264. Google ScholarDigital Library
- Holger H. Hoos. 2012. Programming by optimization. Communications of the ACM 55, 2, 70--80. Google ScholarDigital Library
- Bernardo A. Huberman, Rajan M. Lukose, and Tad Hogg. 1997. An economics approach to hard computational problems. Science 275, 51--54. DOI:http://dx.doi.org/10.1126/science.275.5296.51Google ScholarCross Ref
- Matthias Jeschke and Roland Ewald. 2008. Large-scale design space exploration of SSA. In Proceedings of the 6th International Conference on Computational Methods in Systems Biology (CMSB’08). 211--230. DOI:http://dx.doi.org/10.1007/978-3-540-88562-7_17 Google ScholarDigital Library
- Tackseung Jun. 2004. A survey on the bandit problem with switching costs. De Economist 152, 513--541. Issue 4. DOI:http://dx.doi.org/10.1007/s10645-004-2477-zGoogle ScholarCross Ref
- Kenji Kira and Larry A. Rendell. 1992. The feature selection problem: Traditional methods and a new algorithm. In Proceedings of the 10th National Conference on Artificial Intelligence (AAAI’92). 129--134. Google ScholarDigital Library
- Michail G. Lagoudakis, Michael L. Littman, and Ronald E. Parr. 2001. Selecting the right algorithm. In Proceedings of the 2001 AAAI Fall Symposium Series (AAAI’01). 74--75.Google Scholar
- Ivan S. K. Lee and Henry Y. K. Lau. 2004. Adaptive state space partitioning for reinforcement learning. Engineering Applications of Artificial Intelligence 17, 6, 577--588. Google ScholarDigital Library
- Michael Lees, Brian Logan, Chen Dan, Ton Oguara, and Georgios K. Theodoropoulos. 2005. Decision-theoretic throttling for optimistic simulations of multi-agent systems. In Proceedings of the 9th International Symposium on Distributed Simulation and Real Time Applications (DS-RT’05). 171--178. Google ScholarDigital Library
- Carsten Maus, Stefan Rybacki, and Adelinde M. Uhrmacher. 2011. Rule-based multi-level modeling of cell biological systems. BMC Systems Biology 5, 166.Google ScholarCross Ref
- Orianne Mazemondet, Mathias John, Stefan Leye, Arndt Rolfs, and Adelinde M. Uhrmacher. 2012. Elucidating the sources of β-catenin dynamics in human neural progenitor cells. PLoS ONE 7, 8, e42792. DOI:http://dx.doi.org/10.1371/journal.pone.0042792Google ScholarCross Ref
- Philip K. McKinley, Seyed M. Sadjadi, Eric P. Kasten, and Betty H. C. Cheng. 2004. Composing adaptive software. Computer 37, 7, 56--64. DOI:http://dx.doi.org/10.1109/MC.2004.48 Google ScholarDigital Library
- Sina Meraji, Carl Tropper, and Wei Zang. 2010. A multi-state Q-learning approach for the dynamic load balancing of time warp. In Proceedings of the 24th Workshop on Principles of Advanced and Distributed Simulation (PADS’10). 1--8. DOI:http://dx.doi.org/10.1109/PADS.2010.5471661 Google ScholarDigital Library
- Andrew W. Moore and Christopher G. Atkeson. 1995. The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Machine Learning 21, 3, 199--233. Google ScholarDigital Library
- David M. Nicol and Paul F. Reynolds Jr. 1990. Optimal dynamic remapping of data parallel computations. IEEE Transactions on Computing. 39, 2, 206--219. DOI:http://dx.doi.org/10.1109/12.45206 Google ScholarDigital Library
- Jörg Nievergelt, Hans Hinterberger, and Kenneth C. Sevcik. 1984. The grid file: An adaptable, symmetric multikey file structure. ACM Transactions on Database Systems 9, 1, 38--71. Google ScholarDigital Library
- Avinash C. Palaniswamy and Philip A. Wilsey. 1993. Adaptive bounded time windows in an optimistically synchronized simulator. In Proceedings of the 3rd Great Lakes Symposium on Design Automation of High Performance VLSI Systems. 114--118. DOI:http://dx.doi.org/10.1109/GLSV.1993.224467Google Scholar
- Krzysztof Pawlikowski, Hae-Duck J. Jeong, and Jong-Suk R. Lee. 2002. On credibility of simulation studies of telecommunication networks. IEEE Communications Magazine 40, 1, 132--139. Google ScholarDigital Library
- Stuart I. Reynolds. 2000. Decision boundary partitioning: Variable resolution model-free reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning (ICML-2000). 783--790. Google ScholarDigital Library
- Pannagadatta Shivaswamy and Thorsten Joachims. 2012. Multi-armed bandit problems with history. In International Conference on Artificial Intelligence and Statistics, Vol. 22. 1046--1054.Google Scholar
- Lisa M. Sokol and Brian K. Stucky. 1990. MTW: Eperimental results for a constrained optimistic scheduling paradigm. In Distributed Simulation. Vol. 22. SCS, 169--173.Google Scholar
- Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press. Google ScholarDigital Library
- Adelinde M. Uhrmacher, Jan Himmelspach, and Roland Ewald. 2011. Effective and efficient modeling and simulation with DEVS variants. In Discrete-Event Modeling and Simulation: Theory and Applications. CRC Press, Taylor and Francis, 139--176.Google Scholar
- Joannès Vermorel and Mehryar Mohri. 2005. Multi-armed bandit algorithms and empirical evaluation. In Proceedings of the 16th European Conference on Machine Learning (ECML’05). 437--448. Google ScholarDigital Library
- Roberto Vitali, Alessandro Pellegrini, and Francesco Quaglia. 2010. Autonomic log/restore for advanced optimistic simulation systems. In Proceedings of the International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS). 319--327. Google ScholarDigital Library
- Bernard P. Zeigler, Tag Gon Kim, and Herbert Praehofer. 2000. Theory of Modeling and Simulation (2nd ed.). Academic Press, Inc. Google ScholarDigital Library
Index Terms
- Automatic Runtime Adaptation for Component-Based Simulation Algorithms
Recommendations
A generic adaptive simulation algorithm for component-based simulation systems
SIGSIM PADS '13: Proceedings of the 1st ACM SIGSIM Conference on Principles of Advanced Discrete SimulationThe state of a model may strongly vary during simulation, and with it also the simulation's computational demands. Adapting the simulation algorithm to these demands at runtime can therefore improve the overall performance. Although this is a general ...
Dynamic State Space Partitioning for Adaptive Simulation Algorithms
VALUETOOLS'15: Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and ToolsAdaptive simulation algorithms can automatically change their configuration during runtime to adapt to changing computational demands of a simulation, e.g., triggered by a changing number of model entities or the execution of a rare event. These ...
Bayesian changepoint detection for generic adaptive simulation algorithms
ANSS '15: Proceedings of the 48th Annual Simulation SymposiumAdaptive simulation algorithms are used to deal with changing computational demands of simulations due to state changes of the model and the environment. If such an algorithm is developed in a generic manner, i.e., it is not equipped by the developer ...
Comments