Agentbased Simulation Platform Evaluation in the Context of
Agent-based Simulation Platform Evaluation in the Context of Human Behavior Modeling Michal Laclavík, Štefan Dlugolinský, Martin Šeleng, Marcel Kvassay, Bernhard Schneider, Holger Bracker, Michał Wrzeszcz, Jacek Kitowski, Ladislav Hluchý
IKT Group - Institute of Informatics SAS Dept. of Parallel and Distributed Computing Research and Development Areas: – Large-scale HPCN and Grid applications – Intelligent and Knowledge oriented Technologies URL: http: //ikt. ui. sav. sk Director & leader of PDC: Dr. Dipl. Ing. Ladislav Hluchý Experience from European projects: – 6 project in FP 6: EGEE II, K-Wf Grid, DEGREE (coordinator), EGEE, int. eu. grid, MEDIGRID – 4 projects in FP 7: Commius, Admire, EGEE III, Secricom – 1 EDA project: EUSAS Several National Projects (SPVV, VEGA, APVT) IKT Group Focus: – – – Multi-Agent Systems Information Processing Semantic Web Knowledge oriented Technologies Parallel and Distributed Information Processing Solutions: – – 2 May 2011 Agent. OWL: semantic web and FIPA agents Ontea: Pattern-based Semantic Annotation ACo. MA: KM tool in Email EMBET: Recommendation System ITMAS 2011 2
EDA R&T JIP FP project (for R&T Joint Investment Programme on Force Protection) European Urban Simulation for Asymmetric Scenarios 3 ITMAS 2011 2 May 2011
EUSAS Objectives results qualitative evaluation Serious Game updates interactions Training Rules Of Engagement Modelling updates models Learning results Data Farming interactions quantitative evaluation Analysis results 4 ITMAS 2011 2 May 2011
ABS Evaluation Approach • Survey Literature – List of available simulation platforms is in Deliverable Appendix – Existing ABS evaluations were considered • Evaluation Criteria/Features – 12 features selected – list on the next slide • Principle: Evaluation through implementation – Exemplary Human Behavior scenario defined • Civilians getting angry (throwing stones) or afraid (running to safety). • Soldiers arresting Civilians if hit by stone – Implemented in MASON, Net. Logo and VBS 2 2 May 2011 ITMAS 2011 5
Survey Literature • Stupid Agent Model – 16 features – S. F. Railsback, S. L. Lytinen and S. K. Jackson Agent Based Simulation Platforms: Review and Development Recommendations Simulation 8: 9 (2005) – Net. Logo, MASON, Repast, Swarm and Java Swarm – Later also others, like Eco. Lab • Human Behavior Modeling – We have chose 12 features – Generic, but evaluated on Human Behavior Scenario 2 May 2011 ITMAS 2011 6
12 Features selected as evaluation criteria 1. Loading and Representing the Environment and the Scenario 2. Creating and Representing Agents 3. Behavior Implementation 4. Movement Implementation 5. Visualization 6. Parameterization 7. Model check-pointing 8. Analytical Tools 9. Logging 10. Performance 11. Standards 12. Development Environment 2 May 2011 ITMAS 2011 7
Exemplary Human Behavior Scenario • Soldiers – Catching civilians if hit twice by stone – If civilian is caught (arrested), civilian will disappear – Soldiers are robotic (no emotions) • Civilians – Driven by 2 emotions (fear and anger) – When angry, trying to find stone, going to soldier and throws the stone – When afraid, flying to safety area (yellow) 2 May 2011 ITMAS 2011 8
NETLOGO 4. 1 • Advantages – Net. Logo can be invoked and controlled by another program running on the JVM by Controlling facility API (e. g. app which automates series of model runs, embed Net. Logo models in a larger app) – Simulation state (i. e. world) can be saved to a CSV file and later loaded – Java API for creating custom extensions to Net. Logo (commands, reporters) – Models can be run without visualization – Ability to load vector GIS data (points, lines, and polygons - ESRI shapefiles), and raster GIS data (grids) into Net. Logo by GIS extension – Easy to draw graphs, create simulation parameter controllers (sliders, buttons, etc. ) – Many useful tools like Behavior. Space, System Dynamics Modeler, Hub. Net, Logging 2 May 2011 ITMAS 2011 9
NETLOGO 4. 1 • Disadvantages – Slower than Mason, some parts of user code are interpreted at runtime – The Controlling facility API is considered experimental. It is likely to continue to change and grow. – Support for creating 3 D worlds is still in an experimental state. Only 2 D world is fully supported. 2 May 2011 ITMAS 2011 10
MASON • Advantages – Fast, overhead of simulation environment is minimal – Java – Models are completely independent from visualization – Models may be checkpointed and recovered – Agents are not forced to have a physical location, which is good if we want to create agents representing groups (meso and macro levels) – Physical environment – any number of 2 D or 3 D layers – Multiple Displays of simulation – Time series Graphs, variable inspectors – GIS data can be loaded • Disadvantages – Net. Logo has better support for movement and analysis of distances, objects etc. in physical environment – this impacts development speed, but gives flexibility 2 May 2011 ITMAS 2011 11
VBS 2 • Integration Challenges – VBS is thread- and event-based. Our candidate ABS systems (MASON and Net. Logo) are step-based. Integration is not straightforward but feasible. – Changing the action in the middle of its execution may cause a jerking animation. For example: while throwing a stone – the agent decides to run to the safety area – Movement in VBS may be executed a bit differently from what was planned and simulated in ABS: we need to use waypoints to minimize the discrepancy. 2 May 2011 ITMAS 2011 12
Evaluated Features • Loading and Representing the Environment and the Scenario – MASON • 2 D, 3 D, layered: Int. Grid 2 D, Continuos 2 D • GIS support: tested – Net. Logo • 2 D: two-dimensional grid of “patches”, 3 D experimental • easy import from bitmap • GIS support: tested • Creating and Representing Agents – Soldier, Civilian, Stone – MASON • Represented by Java class (Steppable interface), step(Sim. State state) method • access to environment : Sim. State state – Net. Logo • Turtles (dynamic), patches, links and the observer 2 May 2011 ITMAS 2011 13
Evaluated Features • Behavior Implementation – Net. Logo • turtle variable and the RUN command – MASON • step(Sim. State state) • Movement Implementation – Net. Logo • Direction and step – MASON • Go to X, Y • Flocking, steering: implemented in demo • Visualization – MASON: strong separation of Model and Visualization – Net. Logo: possibility to switch off visualization, speed does not change. 2 May 2011 ITMAS 2011 14
Evaluated Features • Parameterization – supported • Model check-pointing – Supported in both – MASON: platform independent • Analytical Tools – MASON: improvement over the years • Property inspectors • Video, snapshot, streaming, charts – Net. Logo: • Property inspectors • snapshot, streaming, charts • Logging – MASON: log 4 j can be used – Net. Logo: using log 4 j integration 2 May 2011 ITMAS 2011 15
Evaluated Features: Performance Net. Logo 2 May 2011 Mason ITMAS 2011 16
Evaluated Features • Standards – Agent Standards: FIPA – not relevant for simulation agents – DIS and HLA standards • Relevant but we did not test • VBS 2 will be integrated for training – we plan to use the plug-in functionality in VBS 2 and CORBA technology • need to create a FOM - Federation Object Model • Java based HLA: – po. RTIco – Java port of CERTI • Development Environment – Both step based, easy debug, better then thread based MAS – Net. Logo: • Net. Logo IDE, debugging using variable inspectors – MASON: • Any Java IDE, standard Java debbuging • We have used Eclipse 2 May 2011 ITMAS 2011 17
Conclusion • Both are almost equal in many features • Net. Logo: better in physical movement support and some analytical tools. • MASON: much faster, supports strong separation of visualization and behavior models, better support for 3 D environment, Java based - easier to integrate with other systems. 2 May 2011 ITMAS 2011 18
- Slides: 18