Breakout/Snapshot Sessions & Panel Discussions

Monday, September 14, 2020

  • Assessing the ‘delivery divide’. How can we ensure fairness in this time of transitioning from center-delivered to online proctored tests? Monday, September 14, 2020 | 08:00:AM - 08:40:AM ET We’re all familiar with ensuring test form equivalence, or even equivalence when transitioning a qualification from paper to on-screen, but what about ensuring equivalence between center-delivered and online proctored tests? Despite delivering exactly the same test content, in the same format, can you be ...

  • Online Proctoring in a Post COVID-19 Era Monday, September 14, 2020 | 08:00:AM - 08:40:AM ET "The COVID-19 crisis brought about the most sudden and dramatic shift in test delivery that we have ever witnessed. Forced with a binary choice of moving testing online or stop testing, many organisations chose the former and moved with creditable agility to adopt online proctoring ...

  • SPONSOR SESSION (BTL Surpass): Advanced Item Banking in Surpass: SME Management, Standard Setting, and What the Future Holds Monday, September 14, 2020 | 09:30:AM - 10:10:AM ET Surpass is one of the most sophisticated solutions on the market for innovative and streamlined test development. In this presentation, members of the Surpass team will be joined by guests from the community to uncover how advanced item banking capabilities in Surpass are helping organizations ...

  • Stealing Test Content with Hidden Cameras Monday, September 14, 2020 | 09:30:AM - 10:10:AM ET With the move towards online testing, it is easier now, more than ever, to steal test content. We present three inexpensive and effective methods for stealing test content with hidden cameras. Testing programs need to be aware of and prepared for these security threats. Hidden ...

  • The importance of soft skills in the post-pandemic era Monday, September 14, 2020 | 09:30:AM - 10:10:AM ET Advances in technology, digitalization and automation have led to changes in the workplace, with increasing importance being placed on inter- and intra-personal skills. Similarly, in education, the movement towards addressing 21st century skills has led to a broadening of focus from basic literacy and numeracy, ...

  • Asia EdTech and Value of Assessment—Use Case Exploration Monday, September 14, 2020 | 01:30:PM - 02:10:PM ET In this session, the panel of experts from Asia ATP will discuss the trend of EdTech in K-12 education, AI technology use in testing and training, value of assessment in the new technology platforms and disruption of existing testing business in China, Japan, Korea and ...

  • Head of the Class: Being Smart about Item Classification Monday, September 14, 2020 | 01:30:PM - 02:10:PM ET Classifying items to incorrect content domains impacts subsequent exam development activities and the integrity of an exam. Item classification is an integral part of multiple stages in the exam development life cycle and can be as simple as creating a crosswalk document for a programmer ...

  • Modeling Methodologies for Automated Item Generation Across Multiple Domains Monday, September 14, 2020 | 01:30:PM - 02:10:PM ET AIG is becoming a viable option for an increasing number of test developers. Besides cognitive and content considerations, specifications around the exam use should be considered when determining the best methodologies for writing AIG models. Use specifications could include formative/summative or fixed form/adaptive. ...

  • Objectifying the subjective: How the cognitive experience of panelists shapes the standard setting process Monday, September 14, 2020 | 01:30:PM - 02:10:PM ET Most standard setting methods rely on subject matter expert (SME) judgments to arrive at a cut score or passing standard. This session will focus on the cognitive experience of standard setting panelists and how their experience informs their judgments and ultimately shapes the standard. Topics ...

  • SPONSOR SESSION (Alpine Testing Systems): What does it Take to Migrate to a New Candidate Management System? Monday, September 14, 2020 | 01:30:PM - 02:10:PM ET In the world of academic and licensure testing in most test markets, on-site test administration security is still the weakest link in the security plan of most exam program. For some developing markets ~ 90% of the public tests are still using paper form administrations which makes ...

  • Beyond Angoff: A Review of Standard Setting Methods for Multiple-Choice and Oral Examinations Monday, September 14, 2020 | 02:50:PM - 03:30:PM ET Standard setting is the process of establishing a performance standard on an examination. While every aspect of exam development and administration contributes to the exam validity argument, special attention should be given to sound standard setting practices. Arguably the most commonly used method is the ...

  • Building on Success: A Case for Using Certifications to Build a Competent Workforce Monday, September 14, 2020 | 02:50:PM - 03:30:PM ET The National Roofing Contractors’ Association (NRCA) is committed to developing a competent, sustainable, and high-performing roofing industry workforce to help address a global shortage of skilled workers. This breakout session describes how NRCA engaged industry partners to create an accessible network to deliver its hands-on ...

  • Employer Perspectives on Shifting Workforce Trends Monday, September 14, 2020 | 02:50:PM - 03:30:PM ET Where will the workforce be in the next 5-10 years, and what does that mean for workforce skills credentialing? The World Economic Forum’s 2018 report on the future of jobs suggests that by 2022, approximately 50% of employers will see a reduction in workforce and 38% will add ...

  • Scaffolding Complex Activities to Strengthen the Validity of Next Generation Assessments Monday, September 14, 2020 | 02:50:PM - 03:30:PM ET Next generation requirements related to college and career preparedness are bringing about new complex and multidimensional assessments, such as Next Generation Science assessments and English Language Proficiency assessments. Substantiating their validity necessitates deliberate, complex methods that extend traditional test development procedures and requires deeper knowledge ...

  • SPONSOR SESSION (Internet Testing Systems): Reinventing Certification Monday, September 14, 2020 | 02:50:PM - 03:30:PM ET Programs do a great job in getting candidates certified. But once certified, candidates are often one and done. Join ITS in exploring Continuous Certification, combining assessment with learning in an at-home, mobile friendly platform. In a Continuous Certification model, candidates keep their certification and program ...

  • Delivering Occupational Assessments under ISO 10667: Gaining an International Advantage Monday, September 14, 2020 | 04:15:PM - 04:55:PM ET ISO 10667 (“Procedures and methods to assess people in work and organizational settings”) is an international standard which sets requirements and guidelines for using tests in the workplace. This session explains how 10667 allows service providers to differentiate themselves in the marketplace. Service providers who reference and ...

  • It’s Not All Fun and Games: Real-World Considerations for Performance Assessments and Gamification Monday, September 14, 2020 | 04:15:PM - 04:55:PM ET This session will weigh both the advantages and considerations of these assessment approaches and will provide a practical set of guidelines that can be used for assessment organizations to determine if performance-based and game-based assessments are right for your program.

  • Measuring Test Security: Determining the Effectiveness of Test Security Innovations Monday, September 14, 2020 | 04:15:PM - 04:55:PM ET Your organization has worked diligently to put into place test security measures, but how do you know if they are working? As we consider advancement of the testing industry with integrity, it is critical to fully understand the extent to which test security programs impact ...

  • Mentoring and Measuring: Implementation and Evaluation of Peer SME Mentoring for Item Writers Monday, September 14, 2020 | 04:15:PM - 04:55:PM ET A perennial issue for credentialing programs is the gap between subject-matter expertise and item-writing ability: subject-matter experts (SMEs) often struggle to write good test questions. When returns from staff training diminish, mobilizing SMEs to mentor other SMEs may improve item-writing outcomes and build critical relationships. ...

  • SPONSOR SESSION (ATA): How Do we Connect to the World Through Our Test Delivery Service and AI-Assisted Technology Monday, September 14, 2020 | 04:15:PM - 04:55:PM ET Back in 1999, when the testing industry was not commercialized and completely owned and operated by the government, ATA’s founder and chairman Mr. Ma seized the opportunity with the vision that a neutral third party services private company could do a better job driven by ...

Tuesday, September 15, 2020

  • The New Normal in Academic Assessments: How Universities in India are Managing Examinations in this Pandemic? Tuesday, September 15, 2020 | 05:50:AM - 06:30:AM ET Need for digitization in universities to cope with global pandemic. Areas that can be digitized with focus on examinations. Necessary preparation required from the university/board to adapt digital examinations. How was the examination for Manipal University a trend setting event?

  • AI-Assisted Remote Test Proctoring (AI-ARTP): Technology, Policy, Applications, Opportunities and Challenges Tuesday, September 15, 2020 | 08:00:AM - 08:40:AM ET To curb the spread of the Covid-19 pandemic, over 190 countries, including England, have canceled or delayed all of their national examinations, especially those in education. With the 2nd wave of potentially worse infections in the fall, it is hard to imagine another round of all-out ...

  • Do all test item types take the same route to travel? Introduction to the diagnostic model of a test travel agent Tuesday, September 15, 2020 | 08:00:AM - 08:40:AM ET Generally speaking, unless the aim is to test foreign language proficiency, a test should be administered in the test takers' native language: this allows them to express their ability. Exceptions? Sure. When a language doesn't have the lexical coverage in the domain that is being ...

  • When your Plan B Should Have Been your Plan A: Contingency Planning for Creating an Effective Assessment Ecosystem Tuesday, September 15, 2020 | 08:00:AM - 08:40:AM ET The world has been in the midst of an unimaginable crisis that has brought uncertainty and a necessity for fluidity in terms of test program decisions and processes. It’s fair to say that many testing programs were caught without a test delivery failsafe when ...

  • A Multi-Faced Approach to Counteract the Inequitable Educational Impact of COVID-19 Tuesday, September 15, 2020 | 09:30:AM - 10:10:AM ET The harmful effects of this global pandemic will not be distributed equally across parents or students. Not surprisingly, the impacts are expected to be most damaging for children in the most disadvantaged countries, the poorest neighborhoods, and for those children already in vulnerable situations around ...

  • Many Out of One: Item Modelling and Automatic Item Generation - Best Practices and Lessons Learned Tuesday, September 15, 2020 | 09:30:AM - 10:10:AM ET Automatic Item Generation holds the promise to quickly expand item banks at a fraction of the cost of traditional item development. There is however no such thing as clicking on a button and one item automatically becomes many different items. This session will share Item ...

  • SPONSOR SESSION (British Council): From Afghanistan to Zimbabwe: Taking Computer Based Testing to the Wider World Tuesday, September 15, 2020 | 09:30:AM - 10:10:AM ET Are you looking to transform and grow your testing business worldwide? The British Council is a highly trusted partner, providing access to the world’s most reputable qualifications for more than 85 years. Underpinned by principles of integrity, security and fairness, we are one of the ...

  • The Detection of Cheating Using Machine Learning and the Discrete-Option Multiple-Choice Item Format Tuesday, September 15, 2020 | 09:30:AM - 10:10:AM ET Cheating on tests, and the theft of test content, continue to be ongoing threats to high-stakes testing programs in diverse fields such as K-12 education, higher education, educational college admissions, certification, licensure, pre-employment and employment. Each of these areas has numerous testing programs that are ...

  • To Automate or Not to Automate? Tuesday, September 15, 2020 | 09:30:AM - 10:10:AM ET Automations are becoming a more integral part of developing and delivering assessments. Generating assessment content automatically increases item bank capacity to support demands for more static forms to help solve security issues, more items to support computer adaptive testing, and more high quality items to ...

  • Great Answers to Tough Questions Tuesday, September 15, 2020 | 01:30:PM - 02:10:PM ET "I had a problem with my administration. Why can’t you just change my score?" "Why haven't you released my score -- do you think I cheated?" "Why can't I see the questions you asked last time, to help me study?" "Can you just help ...

  • Implementing Personnel Assessment Programs: Comparing Best Practices to Reality Tuesday, September 15, 2020 | 01:30:PM - 02:10:PM ET Publishers of personnel assessments work to ensure their assessments operate in accordance with a series of legal guidelines and industry best practices. These best practices specify the evidence needed to show assessments are fair, reliable, and valid for employee selection and the best methods for ...

  • Preparing Students for Success : Blending Learning and Testing in Digital Environments Tuesday, September 15, 2020 | 01:30:PM - 02:10:PM ET Recent advances and the widespread use of education technology have blurred the lines between learning and testing. In this session, a panel of experts representing testing organizations and learning organizations will address the role of EdTech at the intersection of learning & testing. The panelists ...

  • SPONSOR SESSION (ACT): Developing Psychometrically-Sound and Commercially Viable Solutions for Social-Emotional Learning (SEL) and Cross-Cutting Competencies (CCC) Tuesday, September 15, 2020 | 01:30:PM - 02:10:PM ET According to a 2018 Pew Research Center survey of American youth ages 13 to 17, anxiety and depression are on the rise among young people. A majority of the young people who responded to the survey also noted that issues such as anxiety and depression, bullying, and drug ...

  • Show and Tell: Using Innovation to Improve Exam Security Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET Our assessments are perhaps the most vulnerable during the administration period, so how can we most effectively minimize risks during that time period? For decades testing programs have used proctors, testing manuals, and written instructions to testing vendors to help secure the test during test ...

  • So You're Thinking about Implementing Automatic Item Generation at your Organization? Practical Issues you’ll Wish you Considered Beforehand Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET Model-based content development techniques are increasingly common. Many presentations have discussed the technical aspects of building models. These presentations raise the question: How do you go about implementing this initiative? This session will answer this question by sharing lessons learned from two certification testing organizations ...

  • SPONSOR SESSION (Examity): For Online Proctoring and Test Integrity, Hindsight is 2020 Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET No one could have predicted the challenges and opportunities the assessment community has faced in 2020, but one thing is certain, change is here to stay. In this session, we will review how organizations like yours have adopted online proctoring during our new normal. How does ...

  • Strategic and Legal Lessons Learned in a Transformative, Global Shift in Test Delivery Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET This session will share practical, seldom-used approaches for testing organizations shifting between different test delivery modalities. We will discuss lessons learned from a case study of a high-stakes testing organization’s conversion from paper- to computer-based delivery, including procurement strategies and approaches that challenge conventional ...

  • Strategies and Tactics for AIG Design and Application Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET Automated Item Generation (AIG) is a process which uses cognitive and psychometric theories to produce hundreds of items from a single item template utilizing computer technology. AIG may be the answer to the never-ending need for high-quality test items, but a good foundation is needed ...

  • The Forgotten Benefits of a Job Analysis Tuesday, September 15, 2020 | 02:50:PM - 03:30:PM ET "The job analysis is the foundation of any testing program. Its ability to help define the purpose and intention of a credentialing exam is well supported, and the building blocks are well known. The benefits of carrying out a job analysis are many, but some ...

  • Addressing Middle Eastern Testing Market Needs with Latest Technology Tuesday, September 15, 2020 | 04:15:PM - 04:55:PM ET In the world of academic and licensure testing in the developing markets, paper based administration is still the main stream. In China ~ 95% of the public tests are still using paper form administrations. In this session, you will see how video-based behavioral analysis helps in the ...

  • From Research Labs to the Hands of Customers: Alternatives to “Throw it Over the Wall” Tuesday, September 15, 2020 | 04:15:PM - 04:55:PM ET "Companies struggle to turn research investments into innovations and successful products (or product features) delivered at scale. Like many industries, the testing industry is experiencing a period of transformation and trying to establish more efficient and effective innovation processes. This session focuses on several research-based ...

  • Innovative Item Types: Development, Review, and Analysis Tuesday, September 15, 2020 | 04:15:PM - 04:55:PM ET "Innovative item types are increasingly being used in assessments due to their capability of effectively assessing higher-order skills that cannot be measured well using traditional multiple-choice items. The development of the construct and the items are different from traditional tests due to their unique features. ...

  • Maximizing Your Job Analysis: Options and Opportunities Tuesday, September 15, 2020 | 04:15:PM - 04:55:PM ET The goal of this session is to share the opportunities that exist in the area of job analyses so that credentialing programs can best leverage their next analysis to maximize the benefits of this process (and, just maybe, save some time and money).

  • Can Advancements in Virtual Reality/Augmented Reality Improve Assessments? Tuesday, September 15, 2020 | 08:00:PM - 08:40:PM ET Virtual Reality/Augmented Reality are making rapid advancements in technology. We can expect it to begin penetrating more aspects of our lives. I would like to explore how advancements in Virtual Reality/Augmented Reality can potentially enhance how we deliver, take and evaluate assessments. What ...

Wednesday, September 16, 2020

  • Assessments – Why, What and How Wednesday, September 16, 2020 | 05:50:AM - 06:30:AM ET The main purpose of assessment is to provide feedback, and it is, hence, an absolute essential in higher education systems. Different types of assessment seek to achieve distinctive objectives, and each should follow the key principles of reliability, validity, relevance and transferability. However, the ground ...

  • Digital Shift in Pandemic: The Changing Face of Corporate Recruitment and Employee Skill Evaluation Wednesday, September 16, 2020 | 05:50:AM - 06:30:AM ET The conventional recruitment and skill evaluation that we have known. How pandemic forced the digital adoption overnight – shift in recruitment modes and employee roles? What should corporates do to make the best of this digital shift? Levers to enable this shift (platform, content, process and ...

  • ATP Guidance on Video Surveillance. What is Safe? What is Sensible? What is Unwise? Wednesday, September 16, 2020 | 08:00:AM - 08:40:AM ET This session summarizes good practice on using video surveillance in testing in compliance with the GDPR. What do you need to do to legitimately use video at test centers and when using remote proctoring? How long can you retain video data for? Do you need ...

  • How Do Unexpected Environmental Changes Impact Testing? And Is the Market Changing to Support Student Needs? Wednesday, September 16, 2020 | 08:00:AM - 08:40:AM ET "In the wake of a global crisis, schools have had to quickly pivot to new and innovative ways of learning, instruction and assessment. With the widespread closing of schools and educational institutions, learning and instruction has been interrupted midstream. Organizations have reacted by quickly implementing ...

  • Would Content By Any Other Name Deliver Value As Sweet? Combining Technology and Science Through Semantic Web Technologies to Enhance Content Wednesday, September 16, 2020 | 08:00:AM - 08:40:AM ET "Value from assessment often comes from delivering the appropriate content. Test publishers need to deliver content that is contextually relevant to the audience’s desired outcomes (eg personal development) as well as being cognizant of the audience’s context (eg test expertise). When content is ...

  • Considerations for Remote Administration and Interpretation of Standardized Assessments Wednesday, September 16, 2020 | 09:30:AM - 10:10:PM ET This session will highlight the messages many governing organizations have delivered to set the stage for the discussion. After a healthy discussion on the “should we?” debate, the session will then move into the topic of considerations to take when engaging in remote administration of ...

  • Game-Based Assessments: Fad or Sound Psychometric Measures? Wednesday, September 16, 2020 | 09:30:AM - 10:10:AM ET Despite game-based assessments (GBAs) being increasingly employed in selection practice, there is limited discussion of design principles for these innovative assessments. Furthermore, evidence regarding their psychometric properties is lacking. There is a need for more rigorous investigation of what GBAs measure, what they do not ...

  • Upholding Assessment Standards for the Benefit of Greater Society and the Future of our Learners Wednesday, September 16, 2020 | 09:30:AM - 10:10:AM ET Online and distance learning have historically had the reputation of being easier or not as valuable as a traditional in-person learning experience. However, by upholding assessment standards, the value of an online education becomes protected. Institutions have set standards in place to ensure students meet ...

  • Evaluating the Quality of a High Stakes Certification Program Against Other Programs: A Comparative Study Wednesday, September 16, 2020 | 01:30:PM - 02:10:PM ET In this presentation, the presenters will discuss the methodology and results of a comparative study of how credentialing bodies assess the efficiency and efficacy of credentialing practices. Efficiency refers to the ability to accomplish the task with minimum effort and cost, while efficacy refers to ...

  • Gaining and Maintaining Stakeholder Buy-in for Assessments: Beyond Validity Evidence Wednesday, September 16, 2020 | 01:30:PM - 02:10:PM ET Do you struggle gaining and maintaining buy-in from stakeholders for using assessments? We often discuss the best methods for demonstrating the validity of our assessments, but how do we address this problem with stakeholders that do not value validity? For example, recent data show HR ...

  • Linear, LOFT, CAT: The Good, The Bad and the Ugly Wednesday, September 16, 2020 | 01:30:PM - 02:10:PM ET Test delivery models have advanced far beyond the first ever standardized exams enhanced by psychometric modeling capabilities and advancements in technology. However, newer is not always better. Rather, selecting a test delivery model must take into consideration many program-specific factors. In this session, presenters will ...

  • SPONSOR SESSION (Kryterion) -- Getting To Yes: Tools and Processes For Engaging and Educating Program Stakeholders Wednesday, September 16, 2020 | 01:30:PM - 02:10:PM ET The long-term success of credentialing programs depends, in large part, upon the ability of stakeholders to agree upon a number of important programmatic decisions in order to avoid costly mistakes in the future. This session will present a case study of a large, multinational European ...

  • Global Program: Local Version. Challenges and Best Practices When Taking Your Certification Program to Another Country Successfully Wednesday, September 16, 2020 | 02:50:PM - 03:30:PM ET Where do we start? What do we need to take into consideration when looking at taking our program global? Who are the stakeholders? What do they need? What processes and procedures should we follow? Should we translate into other languages and how do we ensure ...

  • Is the Psychometric Quality of Situational Judgment Items Affected by the Type of Task Analysis Performed? Wednesday, September 16, 2020 | 02:50:PM - 03:30:PM ET "Given that SJIs are intended to measure practical intelligence, researchers argue that a hybrid approach combining aspects of the critical incident approach and the construct-driven approach is needed to develop SJIs. This debate raises the question of whether the psychometric quality of SJIs varies depending ...

  • Item Bank Hygiene – Is Your Item Bank Squeaky Clean or is There Dirt in the Crevices? Wednesday, September 16, 2020 | 02:50:PM - 03:30:PM ET What is the hygiene of your item bank? When was the last time your item bank underwent a systematic “cleaning” to confirm your items are accurate, relevant, and important? Full item bank reviews require a significant investment of time and resources, so how do you ...

  • Out With the Old, In With the New: Rethinking How to Design, Administer, and Score Large-Scale Assessment to Improve Teaching and Learning Wednesday, September 16, 2020 | 02:50:PM - 03:30:PM ET We’ve heard the typical “story” of large-scale assessment many times over: a set of standards are identified as assessment targets; items are written to those standards; the test is administered as a summative evaluation of achievement; an overall score of student ability is provided; ...

  • SPONSOR SESSION (Zoomorphix Systems): Using Innovative Tools in an Advanced Online Item Banking Solution to Enhance your Item Writing and Review Processes Wednesday, September 16, 2020 | 02:50:PM - 03:30:PM ET This presentation will describe and demonstrate how innovative, user-friendly online tools can enhance your item writing and review processes. Beginning with real-world examples of effective sequential and concurrent review processes, we will demonstrate how those processes can be enhanced by using tools developed with clients ...

  • Evolving a Proven Paper-Based Oral Testing Approach to a Successful Online Solution Wednesday, September 16, 2020 | 04:15:PM - 04:55:PM ET "“A bend in the road is not the end of the road … unless you fail to make the turn.” -- Helen Keller Navigating to an online solution from a paper-based exam can feel daunting, especially for organizations with a long, distinguished history. The outcome, however, ...

  • Item Parameter Drift and SME Groups: Identifying, Presenting, and Discussing Item Parameter Changes Wednesday, September 16, 2020 | 04:15:PM - 04:55:PM ET When key aspects of item measurement change over time, item parameter drift (IPD) is said to occur. The underlying causes are varied and obvious changes in item functioning can be due to one or more underlying causes. Empirical manifestation of IPD may also be influenced ...

  • Juggling Operational Realities and Organizational Requests: Improving High-Quality Programs Wednesday, September 16, 2020 | 04:15:PM - 04:55:PM ET Two highly regarded IT certification programs discuss their evolution while advancing with integrity (maintaining high-quality and psychometric rigor). Both programs grapple with “What we envision versus what we can realistically achieve”. Different stakeholders have different goals; frequently in direct opposition to one another (even when ...

  • SPONSOR SESSION (Learn on Demand Systems): 5 Things You Can Do to Ensure Success in Your Performance-Based Testing Program Wednesday, September 16, 2020 | 04:15:PM - 04:55:PM ET The IT industry's drive toward performance-based testing is changing how technology companies build and deliver learning and skills validation solutions. Discover the latest trends in skills assessments and learning for software companies and how the trends parallel those in performance-based testing. You'll learn: • Why the ...

  • 5 Trends Impacting Assessments Accelerated by COVID Wednesday, September 16, 2020 | 08:00:PM - 08:40:PM ET Discussion on changes in testing philosophies and methodologies from educational institutions as the industry is adapting through COVID-19. The speaker will identify risks and opportunities presented to educators, students, and testing service providers.

Thursday, September 17, 2020

  • The Future of Examinations & Assessment: Transparency, Innovation and Technology Thursday, September 17, 2020 | 05:50:AM - 06:30:AM ET Over a period of time, the world of assessment has changed remarkably and dramatically, especially in the context of the ongoing pandemic. The very framework of examinations, class assessments, testing, grading and certifications are being discussed and debated today, and even widely contested. The recent ...

  • Preparing for Labs in High-Stakes Exams Thursday, September 17, 2020 | 08:00:AM - 08:40:AM ET High-stakes exams are an already complex system of interconnected software and services: an item bank, test driver, delivery software, the internet, delivery computer, and more. Labs, a term used to define an experience where a candidate is assessed on their ability to perform tasks within ...

  • You Have a Competency Model, Now What? Thursday, September 17, 2020 | 08:00:AM - 08:40:AM ET In this session, the presenters will discuss how a nursing organization developed and is implementing its competency model for a nursing specialty. The process included the necessary phases to develop a competency model. It included the design, inputs, focus groups, surveys, analysis, modeling, and reporting ...

  • Test Security Innovation: Industry Feedback and the Value to Testing Organizations Thursday, September 17, 2020 | 09:30:AM - 10:10:AM ET Download the session materials HERE. The last five years have seen many test security innovations. Technology has fueled creative solutions to prevent, deter, and detect test theft and cheating. Developments such as facial recognition software, kiosk-based exam administration, item format innovation, and computer algorithms, transcend ...

  • SPONSOR SESSION (Netlogon Content Development): What No One Talks About: The Daily Challenges of Deploying and Running a Performance-Based Testing Program Thursday, September 17, 2020 | 01:30:PM - 02:10:PM ET This session covers PBT distributed at scale for what is arguably the world's largest technological certification program. The session will include details about technical issues, programmatic decisions, logistical concerns, global distribution, and coordinating several stake holders. The presenters will discuss the pitfalls, dangers, issues and ...

  • Take Your Organization’s Talent to the (Item) Bank: Ways to Recruit, Retain and Promote Item Writers Thursday, September 17, 2020 | 01:30:PM - 02:10:PM ET Among the most valuable assets of a credentialing and testing program are its item bank and volunteer item writers. Certifying organizations often struggle with how best to attract, retain, and cultivate “talent” so that quality assessments can be developed and maintained. This session will present ...

  • Test Security in the Digital Age: Advances in Design and Analytics Thursday, September 17, 2020 | 01:30:PM - 02:10:PM ET A diverse panel representing two large credential programs and two assessment service providers will draw from their real-world experience and research to present and discuss advances in technology-enabled test deign, performance-based assessment, and data forensics to provide enhanced security.

  • Who's Minding the Cloud? Thursday, September 17, 2020 | 01:30:PM - 02:10:PM ET "It used to be that testing programs relied on in-house information technology staff to manage their sensitive data such as test items, item statistics, candidate records, and credentialing decisions. This data was supported and accessed only trusted personnel, using passwords, located in fire-proof, cardkey-accessed server ...

  • No Internet? No Problem. An Innovative Approach to Offline Testing That Goes Beyond Paper. Thursday, September 17, 2020 | 02:50:PM - 03:30:PM ET The phrase “paper is dead” has been going around for decades, but paper-based testing has continued as a prominent delivery mode because of a lack of consistency around internet availability. And as long as the internet is required for computer-based testing, the move away from ...

  • Remote Standard Setting for Credentialing Programs: Innovative ways to Improve Outcomes and Yield a Defensible Cut Score Thursday, September 17, 2020 | 02:50:PM - 03:30:PM ET Setting a defensible cut score or performance standard is a critical component of any assessment-based credentialing program. Ideally, this type of meeting is conducted in-person with an appropriate number of representative subject matter experts. Unfortunately, circumstances within a credentialing organization (or profession) may exist that ...

  • SPONSOR SESSION (Pearson VUE): A Critical Conversation: Marching toward Social Justice Thursday, September 17, 2020 | 02:50:PM - 03:30:PM ET 2020 is a year none of us could have predicted. Perhaps more than ever in recent decades, it seems we’re living history every single day. It’s been a year fraught with change, hardship, and turmoil. But it’s also presented an incredibly unique opportunity ...

  • The Unification of Philosophies: How Qualitative and Quantitative Evidence Gathering Methods Can Inform Your Organization’s Approach to a Job Task Analysis or Practice Analysis Thursday, September 17, 2020 | 02:50:PM - 03:30:PM ET This session will provide attendees an opportunity to critically think about the planning and design of a job task/practice analysis using both qualitative and quantitative approaches. The session presenters will engage the attendees by providing a case-based example of how a mixed methods approach ...

  • From Boots to Suits – Establishing a “Ground Operation” for Testing Military Candidates Thursday, September 17, 2020 | 04:15:PM - 04:55:PM ET Two million men and women currently serve in the U.S. Armed Forces, and every year approximately 200,000 people exit the military to return to civilian life. More than 75 percent of these individuals are within the working age demographic and are highly experienced in military occupations ...

  • If a Computer Algorithm Detects a Security Anomaly, What Reactions are Appropriate? Thursday, September 17, 2020 | 04:15:PM - 04:55:PM ET "This session looks at the legal, practical and ethical issues in how to react to computer-detected anomalies which could mean a test taker is cheating or stealing content. AI, improved video and increasing automation are making it more practical to detect anomalies. But what threshold ...

  • Novel Solutions to Familiar Problems – Practical AI Applications in Assessment Thursday, September 17, 2020 | 04:15:PM - 04:55:PM ET Over the past several years, the level of interaction between people and artificial intelligence (AI) has dramatically increased. Web-based and mobile application use data to shape the experience of users and the outcomes they seek. Search engines attempt to complete your query, recommender systems suggest ...

  • SPONSOR SESSION (Prometric) Visualizing Program Performance in the 21st Century through Data Dashboards, Data Mining, and Predictive Analytics Thursday, September 17, 2020 | 04:15:PM - 04:55:PM ET One of the major benefits for exam programs that transition from paper-based test administrations to computer-based test administrations is the ability to collect a myriad of data about candidates’ behaviors during testing. Thanks to computer-based testing we can now capture every keystroke made by candidates ...

  • How National Policy like Thai 4.0 Improves the Talent Quality Through Digital Awareness Certification Program? Thursday, September 17, 2020 | 08:00:PM - 08:40:PM ET Despite the rhetoric from the Thai government around Thailand 4.0—the national strategy focused on accelerating innovation and digital transformation—Thailand still faces constraints in realising her future-ready vision. Raising the Digital Literacy of the general workforce has become the priority of the nation to assist ...

  • Modifying Testing Models and Model-Based Approaches to Meet Customer’s Needs in China Thursday, September 17, 2020 | 09:30:PM - 10:10:PM ET Testing models and model-based approaches are very useful in test development. However, most of the current scaling and equating approaches do not appeal to Chinese consumers because most people prefer raw scores to scaled scores. In addition, the typical common item equating design doesn’t ...

Friday, September 18, 2020

  • Promoting Equity, Accessibility, Diversity, and Inclusion in Remote Learning Environments Friday, September 18, 2020 | 08:00:AM - 08:40:AM ET Utilizing remote proctoring solutions for both in-person and distance learning can promote equity, accessibility, diversity, and inclusion within the education system. Remote proctoring protects the integrity of assessments and promotes equity in those assessments, by holding everyone to the same academic standards. One of the ...

  • When your Plan B Should Have Been your Plan A: Contingency Planning for Creating an Effective Assessment Ecosystem Friday, September 18, 2020 | 08:00:AM - 08:40:AM ET The world has been in the midst of an unimaginable crisis that has brought uncertainty and a necessity for fluidity in terms of test program decisions and processes. It’s fair to say that many testing programs were caught without a test delivery failsafe when ...

  • Is Remote Proctoring the New Normal of How Tests are Delivered? Friday, September 18, 2020 | 08:45:AM - 09:15:AM ET Has COVID-19 irreverabily changed the way tests would be taken? Is this the new normal? Where do we go from here?

  • Maintaining Sanity While Working From Home Friday, September 18, 2020 | 08:45:AM - 09:15:AM ET Share the challenges you have faced during this period of working from home. What have you found helps you cope with stress, social isolation and working remotely from your work team?

  • The State of the Assessment Industry in Middle East and Africa Friday, September 18, 2020 | 08:45:AM - 09:15:AM ET

  • Diversity Considerations in Emerging Assessment Technologies Friday, September 18, 2020 | 09:30:AM - 10:10:AM ET This session will focus on the rapid pace of technological innovation in assessment and the extent to which it may increase diversity and fairness, or create additional challenges. The assessment landscape has changed significantly during the past decade in both employment and education. Advances in ...

  • The Past, Present, and Possible Future of Proctoring Friday, September 18, 2020 | 09:30:AM - 10:10:AM ET For obvious reasons, interest in online proctoring solutions is currently surging. Many of us rightfully call on all the assistance we can to ensure that our tests are secure during a time of unbelievable upheaval. Proctoring is the word on everyone’s lips. “How can ...

  • Performance-Based Testing: Trials and Triumphs Friday, September 18, 2020 | 01:30:PM - 02:10:PM ET Performance-based testing represents a shift from assessing whether a candidate knows something to whether they can do something. As it brings high-fidelity experiences into exams, performance-based testing has the potential to also introduce a number of new complexities and unknowns into the process. While the ...

  • SPONSOR SESSION (PSI Services): Solving Complex Testing Needs: Using Multi-Vendor Partnerships Friday, September 18, 2020 | 01:30:PM - 02:10:PM ET Not all testing needs are created equal - and in the case of some programs, they can be incredibly complex. In this session, PSI, TrueAbility, and one of our shared clients will discuss how we addressed their testing needs through using the strengths of both ...

  • Third-Party Risk Management: How Do You Know Your Data Is Secure? Friday, September 18, 2020 | 01:30:PM - 02:10:PM ET Look beyond a standard checklist and take a proactive approach to protecting your testing business. Security incidents have immediate and lasting repercussions, but by actively identifying and addressing security risks, you can improve your testing program integrity. Join two industry-leading testing companies, the National Board ...

  • Twenty Years in the Making; Transitioning to Automated Essay Scoring in an Adult Education Assessment Program Friday, September 18, 2020 | 01:30:PM - 02:10:PM ET This session describes the transition from human scoring to AI scoring of writing in a unique adult education assessment context. Presenters explain the motivations and challenges related to embarking on this transition. The pilot study process and results are described, as well as lessons learned ...

  • Machine Learning Algorithms in the Time of Fairness and Transparency Friday, September 18, 2020 | 02:10:PM - 02:40:PM ET Are artificial intelligence and machine learning algorithms inherently biased against minority groups? How are test publishers handling calls for algorithm transparency? This peas in a pod discussion will give people the opportunity to discuss their experiences implementing AI/ML algorithms in the testing industry, including ...

  • The Year 2020 is Disrupting Testing As We Know It, and I Feel Fine Friday, September 18, 2020 | 02:10:PM - 02:40:PM ET Test publishers are displaying great flexibility in our efforts to continue testing activities while maintaining social distance. While 2020 has been difficult, every cloud has a silver lining. This Peas in a Pod session is a chance to share your experiences as you navigate the challenges ...

  • Too Much Stress Working from Home, How to Make your Management Aware? Friday, September 18, 2020 | 02:10:PM - 02:40:PM ET Balance at work and home is needed, how can we share ideas to keep a good outlook.

  • How Can We Use Examiner Behavior Data to Improve Exam Scoring and Inform Decision-Making? Friday, September 18, 2020 | 02:50:PM - 03:30:PM ET Do examiners score mainly in the evening? Does the time they mark affect the quality of marking? Are first-time examiners more reliable? When it comes to planning high-stake assessments and ensuring the quality of scoring, these questions are sure to have crossed your mind. Data ...

  • SPONSOR SESSION (Yardstick Assessment): Adapting Test Delivery in a Changing Environment - The Credentialing Program Perspective Friday, September 18, 2020 | 02:50:PM - 03:30:PM ET As organizations around the world deal with the realities of the recent pandemic, assessment and credentialing programs had to very quickly rethink their processes and operations as it relates to exam delivery. WIth test centers closed or at reduced capacity and taking test-taker health and ...

  • The Past, Present, and Future of Technology in Assessment Friday, September 18, 2020 | 02:50:PM - 03:30:PM ET In this presentation, we will describe the past, present, and future performance assessment by showcasing their use across three different organizations. We will start with the Medical Council of Canada describing the common performance assessment modalities in medicine and health professions; Microsoft will describe their ...

  • Using Targeted Content in a Longitudinal Assessment Framework to Support Learning Through Assessment Friday, September 18, 2020 | 02:50:PM - 03:30:PM ET Recently, several certifying bodies have begun offering a new assessment type designed to make summative recertification decisions and to provide test takers with formative feedback and educational materials. A defining feature of “longitudinal assessments” is that small sets of items are administered through a test ...

  • Best Practices for Translating and Adapting Computerized Tests and Exams in the 21st Century Friday, September 18, 2020 | 04:15:PM - 04:50:PM ET "It takes more than a good translation provider to ensure that an assessment can reliably measure cognitive and non-cognitive skills across languages and cultures. Yet, while most sophisticated testing environments support multiple languages, they often fail to enable workflows that harness the power of translation ...

  • Breaking Down Economic Barriers: Improving Test Score Gains Through Engaging Online Learning Friday, September 18, 2020 | 04:15:PM - 04:55:PM ET The cost of learning resources for standardized tests can hinder a students’ ability to get ready to take high-stakes exams. Learn how a mission-driven nonprofit organization developed a business model that enables learners to reach their highest potential, while also delivering a positive, trusted learning ...

  • Compromised Item Detection for Computerized Adaptive Testing Friday, September 18, 2020 | 04:15:PM - 04:55:PM ET Item leakage has been a serious issue in continuous, computer-based testing, especially computerized adaptive testing (CAT), as compromised items jeopardize the fairness and validity of the test. Strategies to detect and address the problem of compromised items have been proposed and investigated, but many solutions ...

  • The (Legal) Road Less Traveled Friday, September 18, 2020 | 04:15:PM - 04:55:PM ET Foundational legal rights protect your program’s exam content and score validity. Those rights - ranging from contract law to competition and criminal law - provide powerful tools for addressing concerns. But what actually happens when a testing program sues another person or entity, and ...

On Demand

  • Learn More about Incubator Headstart

  • 2020 Graduate Student Research Award: It’s All the Rage: Developing an Animated Screener for Postpartum Depression Postpartum depression presents a complication for mothers which can, in some cases, be severe and even life-threatening. Instruments commonly used to screen for this psychological condition have been challenged by an extensive body of literature, with many mothers being unidentified and even untreated for their ...