2 nd Georgia ADL conference Tbilisi Georgia November

2 nd Georgia ADL conference Tbilisi, Georgia November 20 th, 2018 ADL in international exercises: VIKING 18 and beyond Biljana Presnall www. jeffersoninst. org

Today’s agenda ü NGDLE ü Viking 18 ü Next steps

NGDLE What the heck it that? NEXT GENERATION DISTRIBUTED LEARNING ECOSYSTEM

Military Learning Ecosystem Simulations Classroom E-learning Exercises Social learning ? ? ?

Next Generation Learning Ecosystem Classroom Simulations E-learning Exercises Micro- learning Social learning ROI = competent soldiers

Total Learning Architecture It is a work in progress… But we can keep trying and we have HELP

Biggest help – data standard • The Experience Application Programming Interface (also known as x. API, or by its original working name of Tin Can API) is a data standard for reporting learning activities.

Learning Analytics with x. API in a Multinational Exercise Goal: Operational integration of ADL into Viking 18 – an international exercise.

Objectives • Co-production of courses with V 18 countries; • Implementation of x. API across V 18 platforms; • Setup & secure V 18 LRS; • Implement a multiplatform x. API dashboard; • Capture lessons-learned and build a multi-year forward plan on integrating ADL in exercises.

Stakeholders • A Swedish / U. S. Initiative, Viking 18 is the most recent in a series of exercises initiated at NATO’s 50 th Anniversary Summit in 1999. • A comprehensive and unique collective training opportunity for military, civilian and police.

Stakeholders It is the most extensive multifunctional exercise of its kind and includes more than 50 countries, 35 organization, and 2, 500 participants for 10 days. Viking 18 is arranged by the Swedish Armed Forces and the Folke Bernadotte Academy, with remote sites in Brazil, Bulgaria, Finland, Ireland, Serbia and Sweden.

Solution space Look at an exercise as a total learning experience, capturing all elements, comparing learning and exercise objectives.

Outcome • Demonstration of ADL cutting edge capabilities. • Improved collective training outcomes, partner defense readiness and joint knowledge.

Timeline § Start date: November 2017 § February 28, 2018 LMS open, LRS ready to track live results § Exercise dates: April 16 -26, 2018 § AAR and Planning Workshop: May 29 -31, 2018

Results: Technology The technology is ready: • DATA can be extracted from a hard case non-x. API-compliant LMS; • Visualizations are capable of managing asynchronous data streams from diverse sources in multiple formats and of various scales; • Stakeholders are able to deliver data in a secure and timely manner.

Results: Operational Processes ADL is approaching something close to interoperability in course repositories. • 29 courses drawn from 6 sources: NATO ACT, JKO, Sweden, Slovenia, Bi. H, and Macedonia. • A "course plan" was built for each V 18 unit with a combination of the highly recommended Intro to Viking and 2 -5 recommended courses with content relevant for their role in the exercise.

Results: Cyber security The secure cloud-based LRS and Dashboard remained live, with no intrusions throughout the pre-training and exercise

Results: User demand 773 exercises participants successfully created e. Learning accounts for the V 18 LMS, with over 700 course completions and 1, 000 additional courses initiated, but not completed, spending an average time on site of 5 hrs 30 min.

Top performers Finland Sweden were top performers. Top courses completed, in rank order, were: Intro to Viking, Gender Awareness, Sitaware, Exonaut, UNPKO, and Humanitarian Law. Of these top six, 4 were authored with RADLI countries, and one by JKO.

Results: Learning Analytics Dashboard: we offered the only current V 18 data on exercise performance in the Innovation tent (both e-learning and experiential via Exonaut observations data).

Results: Learning Analytics Nation pages: offered a useful summary for participant delegations: something clear, immediate and relevant to bring home.

Supporting Talent Management Matching learning and performance data to inform decision making on talent alignment: right person, right job

Lessons Learned • x. API is ready for timely secure implementation at scale in a complex multinational environment of legacy systems. • Collective training with partners is fertile ground for transformative innovation. • E-learning course objectives and exercise objectives should better align. • Learning analytics should be driven by stakeholder demand signals: learners, operators, planners, and strategic actors. • User access to both learning content and learning analytics must be improved.

NEXT STEPS Beyond Viking 18 • To develop recommendations or some kind of instruction document with the outlined process for successful ADL integration into multinational exercises. • Evaluation Methods to enhance learner assessment

NEXT STEPS Develop new pedagogical approaches. Online learning is NOT NEW any more, we need approaches that include modern learning paths – we think micro-learning or learning on demand. This includes a study on approaches for motivating learners (Initial field test Micro Assessments tracking reaction and motivation)

ROI • ROI analytics dashboard prototype(s) • Develop evidence-based ROI report

Stay tuned

Thank you! bpresnall@jeffersoninst. org @Gradina The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of any U. S. or foreign defense agency. The authors are grateful for the support and assistance of U. S. Defense agencies including the Joint Staff J 7’s Joint Knowledge Online, the Center for Civil-Military Relations at the Naval Postgraduate School, and the ADL Initiative; the Swedish Armed Forces; NATO Allied Command Transformation, and the Partnership for Peace Consortium ADL Working Group; and the Regional ADL Initiative (RADLI) member nations of Slovenia, Macedonia and Bosnia; and Nordic Defense Cooperation ADL (NORDEFCO). Our participation in Viking 18, and this presentation, are direct products of our joint effort. This work was supported, in part, under a contract with the ADL Initiative (W 900 KK-17 -D-0004).