AIRS Certification Program ComputerBased Testing CBT AIRS Webinar

AIRS Certification Program Computer-Based Testing (CBT) AIRS Webinar: October 2013

The Original Way • AIRS Certification has been around for over 15 years • Eligible candidates take a 100 question exam and acquire either CIRS, CIRS-A or CRS Certification if they pass the exam • The exams have always been proctored with testing taking place in classroom style settings

The Original Way • And the exam answers have always been entered and marked using Scantron forms

Problems with the Original Way • Scantron forms are not the friendliest pieces of paper. It is a strain on the eyes and requires a reasonably intense focus • Although machine readable, AIRS Certification staff often hand-scored exams if the score was within 5 of a passing mark or it looked like a candidate had made an honest mistake in alignment • The exam questions themselves were always at risk as boxes of question papers were sent around the US and Canada, then returned

Problems with the Original Way • Providing timely results to candidates was also a constant challenge – we had to wait for the boxes to be returned, retrieve the forms, feed them through the machine, verify the results – and during busy times, there would be boxes going in and out every day, and if one person was off for a couple of days, then the backlog would build up, etc. … • Candidates wanted their results within a few days. Our target was 30 days and sometimes, we were not even able to meet that …

First Look at Computer-Based Testing • CBT has been on the AIRS radar for about 6 years • Original concerns were: – Expense to members (costs of up to $100 person) – Lack of location choices (most operated through commercial testing centers – less options, even more expense) – Concern of impact on Affiliate conferences – Sense that not everyone was ready for the switch – AIRS Certification had other priority (from 2008 to 2012, all AIRS Certifications underwent stringent psychometric review with new questions and new exams)

Second Look at Computer-Based Testing • More software options with lower pricing and more flexible options • More unease about Scantron as suitable for persons with visual impairments • Survey of AIRS members in 2012 showed that 40% of members had a training room with computers/Internet

Second Look at Computer-Based Testing • Do you know of a 'training-type room' within your community (for example, an employment center, a library, community college) that has a number of Internet-capable computers that could be easily accessed for a CBT Certification exam? • Yes, and I am aware that either my own or other agencies have used this space for various reasons = 39% • Yes, I am aware of such a place but I have never had cause to contact them = 33% • No, but I imagine such a space exists and obtaining it should not be an obstacle = 15% • No, this may be a problem = 13%

Second Look at Computer-Based Testing • Which of the following best describes your feelings towards Computer-based testing? • I fully welcome it = 69% • I welcome it but have some concerns= 27% • I am concerned about it but appreciate there are some benefits= 10% • I prefer the existing system = 4%

Conclusion: Let’s Make the Move to Computer-Based Testing

Software Selection • After testing several CBT softwares, AIRS chose an application called Questionmark • What we liked about it: – Admin end of it was relatively easy to use – Existing clients included US military, major technology companies, financial services, government – Price … honestly, it was the only one that would not have driven up the cost of Certification for AIRS members

Software Selection • Over past 12 months: – Loading of 800+ questions each with 4 option choices – Testing of software by AIRS Certification Commission members, volunteer agencies, Affiliates – Framing and refining of documentation – More testing – Bit more testing

Operating for past few months on Stealth Mode

But now CBT is the only option for AIRS Certification exams … BUT THERE ARE OPTIONS WITHIN THAT OPTION …

CBT: What stays the same • This is not a low stakes Certification exam. There is still a requirement for a proctor • The application process to be approved as a candidate to take an AIRS Certification exam remains the same • AIRS Affiliates still need to approve tests taking place in their state/regions • The ‘atmosphere’ and ‘general expectations’ of an exam environment remain the same

CBT: What is different • An organization wanting to host an exam must fulfill more detailed criteria (basically, a set number of seats with reliable computers and Internet access with technical backup on hand for start of exam) • The proctor’s environment changes and proctor must enter a password before each candidate can start their exam

CBT: What is different • The candidate experience is much improved: – There is one question per page. It is much easier to read the question and answer options – Easy to leave questions unanswered and go back to them later or to answer them and ‘flag’ them for later review – You get the results within 5 business days

Three Exam Options – An exam hosted by an AIRS member (either at their own facility or at a community-based one) – An exam proctored by a college or university – An exam conducted by an online proctor via a webcam at home or at work

Hosted by AIRS member • Host must complete an application form (found via www. airs. org/certification) • Requirements include: – Facility with computers/laptops and Internet – In a quiet setting – Securing a proctor – Having some technical expertise on hand on the day of the exam just in case • No additional cost unless required by the host site to cover specific costs

Hosted by College/University • Exam can be taken at certain colleges in the United States (along with a few in Canada) that are members of the National College Testing Association (NCTA at www. nctatesting. org • Find an institution near you and go through the required steps • Additional cost usually of around $25 to $30 person that is paid directly to the college

Hosted by Webcam-based Proctor • AIRS working with company called Proctor. U • Allows exam to be taken at home or at work (but in quiet/private setting) as long as you have webcam and secure Internet • Cost is $25 but provides flexibility and no travel costs

So What Does The Exam Look Like? ? • https: //ondemand. questionmark. com/deliver y/perception. php? customerid=399354

AIRS CBT: First Reports • The work involved in hosting an exam is a bit more involved for the AIRS member organization but (as with all other things), it will seem a lot simpler the second and third times around • The feedback from candidate has been very positive. It makes the exam process so much easier • However, we are working with technology and although it has not happened yet, something somewhere will go wrong – be patient and we will try our best to fix it

Questions • How come … ? • What about … ? • So what if … ? And when you are ready, go to www. airs. org/certification If stuck or confused, email clivemjones@gmail. com