Telepractice and the KTEA-3

Computers and connectivity

Two computers with audio and video capability and stable internet connectivity—one for the examiner and one for the examinee—are required. A stationary web camera, microphone, and speakers or headphones are required for both the examiner and the examinee. It is recommended that the examiner have a second computer screen so that he or she can view the administration directions, but the paper format manual can also be used.

Teleconference platform

A teleconference platform with screensharing capability is required.

Video

High-quality video (HD preferred) is required during the administration. Make sure the full faces of the examiner and the examinee are seen using each respective web camera. The teleconference platform should allow all relevant visual stimuli to be fully visible to the examinee when providing instruction or completing items; the video of the examiner should not impede the examinee’s view of visual stimuli.

Screensharing digital components

Digital components are shared within the teleconferencing software as specified in Table 1. There are two ways to view digital components in the Q-global Resource Library: through the pdf viewer in the browser window or full screen in presentation mode. Always use full screen (i.e., presentation) mode for digital components viewed by the examinee. This provides the cleanest presentation of test content without onscreen distractions (e.g., extra toolbars). Refer to Using Your Digital Assets on Q-global in the Q-global Resource Library for complete directions on how to enter presentation mode.

Image/screen size

When items with visual stimuli are presented, the digital image of the visual stimuli on the examinee’s screen should be at least 9.7” measured diagonally, similar to an iPad or iPad Air. Some teleconferencing platforms shrink the size of images, so the facilitator should verify the image size prior to the testing session. Typically, computer screens used for teleconference assessment are a minimum of 15” measured diagonally. Smaller screens, such as those of iPad minis and smartphones, are not allowed for examinee-facing content as these have not been examined empirically and may affect stimulus presentation, examinee response, and validity of the test results. Similarly, presenting stimuli on extremely large screens has not been examined, so the same precaution applies. Prior to testing, ask the onsite facilitator to aim a peripheral camera or device (as described in the next paragraph) at the examinee’s screen to ensure that the examinee’s screen is displaying images in the correct aspect ratio and not stretching or obscuring the stimuli image.

Peripheral camera or device

A stand-alone peripheral camera that can be positioned to provide a view of the session from another angle or a live view of the examinee’s progress is helpful. Alternately, the onsite facilitator may join the teleconference from via a separate device (e.g., a smartphone with a camera or another peripheral device) and set it in a stable position to show the examinee’s pointing or written responses.

If using an onsite facilitator who is not in a professional role (e.g., parent/guardian), the examiner should use their professional judgment about the capacity of the facilitator to perform the required functions correctly and without interfering in the testing session. A parent/guardian does not typically remain in the room during testing except on rare occasions as described in the administration manual, so it is necessary to train them how to set the peripheral camera/device in a stable position before beginning the session.

The facilitator should silence the audio and mute the microphone on any peripheral devices to prevent feedback. Train the onsite facilitator to position the peripheral camera/device before written response tasks and subtests that elicit pointing or gestured responses (refer to Table 1) so you can view the examinee’s real-time responses. Instruct the facilitator not to capture video or take photos as this is a copyright violation.

During the COVID-19 outbreak, it may be necessary for examiners to think creatively about how to gain the optimal view of the examinee’s progress in a response booklet or when pointing at a screen. A document camera is the best solution if the examinee’s camera cannot be shifted to provide the correct view of response booklets (e.g., if the examinee’s camera is integrated into a laptop or computer screen). However, it is unrealistic to expect examinees to have document cameras within their homes. Online instructional videos (e.g., here) demonstrate how a smartphone may be used with common household objects (e.g., a tower or stack of books, paper weight, ruler, and rubber band or tape) to create an improvised document camera for use during tasks involving response booklets or manipulatives. While this is not optimal or a permanent solution for telepractice, it is functional in the current situation. Similarly, for multiple choice tasks, some examinees tend to point to responses rather than say the number or letter corresponding to their response, and other tasks (see Table 1) require the examinee to point at the stimuli. In this situation, other everyday household objects (e.g., books) could be used to form an improvised stand upon which to position the device to provide a second-angle view of the examinee pointing at the screen. Typically, devices provide the best view of the examinee’s screen and pointing responses when positioned in landscape format.

Gesturing

When gesturing to the stimulus books is necessary, display them as digital assets onscreen and point using the mouse. It may on occasion be necessary for the examiner to gesture to areas of a paper copy of the response booklets or to show how to respond on the examiner’s camera. Refer to Table 1 for specific instructions by subtest.

Capturing response booklet performance: Response booklets should be placed in the provided envelope immediately upon completion to ensure that no responses are lost or altered prior to scoring. For tasks that are simple to score, it is acceptable to ask the examinee or facilitator to show the completed response booklet on camera at the conclusion of the session before they are sealed into the envelope to be returned (as discussed in the Test/Test Materials section).

Audio considerations

High-quality audio capabilities are required during the administration. An over the head, two-ear, stereo headset with attached boom microphone is recommended for both the examiner and examinee.

Audio check

Test the audio for both the examiner and examinee prior to the administration to ensure a high-quality audio environment is present. This is especially critical for tasks where audio is critical, such as Phonological Processing and Listening Comprehension. Testing the audio should include an informal conversation prior to the administration where the examiner is listening for any clicks, pops, or breaks in the audio signal that distorts or interrupts the voice of the examinee. The examiner should also ask the examinee and facilitator if there are any interruptions or distortions in the audio signal on their end. Record any connectivity lapses, distractions, or intrusions that occurred during testing.

Manage audiovisual distractions

As with any testing session, make sure the examinee’s environment is free from audio and visual distractions. If you are unfamiliar with the examinee’s planned physical location, meet virtually with the facilitator in advance of the testing session. Ask the facilitator to show the intended testing room and provide a list of issues to address to transform the environment into an environment suitable for testing. For example, remove distracting items, silence all electronics, and close doors. Ask the examinee and facilitator to close all other applications on the computer, laptop, or other device, and to silence alerts and notifications on the peripheral device. Ensure radios, televisions, other cellular phones, fax machines, smart speakers, and equipment that emit noise are silenced and removed from the room.

Lighting

Establish good overhead and facial lighting for the examiner and examinee. Close blinds or shades to reduce sun glare on faces and the computer screens.

Practice

During the telepractice setup, and before administering to any actual examinee, practice the mechanics and workflow of every item in the entire test using the selected teleconference platform so that you are familiar with the administration procedures. For example, use a colleague as a “practice examinee.”

Standardized procedures

Follow the administration procedures of face-to-face administration as much as possible. For example, if a spoken stimulus cannot be said more than once in face-to-face administration, do not say it more than once in a telepractice administration unless a technical difficulty precluded the examinee from hearing the stimulus.

Facilitator role and training

The onsite facilitator’s role in a telepractice session is largely to manage audiovisual needs and materials. Train the facilitator to troubleshoot audiovisual needs that arise during the testing session, including camera angle, lighting, and audio checks. The facilitator’s role is not to manage rapport, engagement, or attention during the testing session and they are not to interfere with the examinee’s performance or responses.

If using an onsite facilitator who is not in a professional role (e.g., parent/guardian), the examiner should use their professional judgment about the capacity of the facilitator to perform the required functions correctly and without interfering in the testing session. The examiner should communicate expectations about the facilitator’s role in testing tasks immediately prior to the testing session when the examinee is not present to ensure that nothing is disclosed to the examinee about the tasks. Do not allow the facilitator to show or warn the examinee about any portion of the test. Instruct the facilitator not to open any materials until you provide instructions to do so, if applicable. Expect to provide verbal guidance to the facilitator during the testing session.

For subtests that involve written responses or that elicit pointing responses (see Table 1), train the facilitator to position the peripheral camera/device to allow you to view the examinee’s progress and responses. Instruct the facilitator to watch the onscreen video shown by the peripheral camera/device and to listen to feedback from you to guide camera angle adjustment as they position the camera/device. Refer to the Audio/Visual Equipment section and to Table 1 for specific subtest telepractice considerations for the facilitator.

Any other roles and responsibilities for which an examiner needs support, such as behavior management, should be outlined and trained prior to the beginning of the testing session. The examiner is responsible for documenting all behaviors of the facilitator during test administration and taking these into consideration when reporting scores and performance.

Appropriateness

Ensure that a telepractice administration is appropriate for the examinee and for the purpose of the assessment. Use clinical judgement, best practice guidance for telepractice (e.g., Interorganizational Practice Committee, 2020), information from professional organizations, existing research, and any available federal or state regulations in the decision-making process.

Preparedness

Before initiating test administration, ensure that the examinee is well-rested, able, prepared, and ready to appropriately and fully participate in the testing session.

Facilitator role

Explain the role of the facilitator to the examinee so participation and actions are understood.

Headset

It may not be appropriate or feasible for some examinees to use a headset due to behavior, positioning, physical needs, or tactile sensitivities. Use clinical judgement on the appropriate use of a headset in these situations. If a headset is not utilized, ensure your microphone and the examinee’s speakers are turned up to a comfortable volume.

Mouse

On some teleconference platforms, you can pass control of the mouse to allow the examinee to point to indicate responses; this is acceptable if it is within the capabilities of the examinee. Best practice guidelines provide cautions about this, however (IOPC, 2020).

Copyright

Obtain permission for access to copyrighted materials (e.g., stimulus books, response booklets) as appropriate. Pearson has provided a letter of No Objection to permit use of copyrighted materials for telepractice via non-public facing teleconferencing software and tools to assist in remote administration of assessment content during the COVID-19 pandemic.

Response booklets

Provide the correct printed copies of response booklets to the facilitator in advance of the testing session and communicate the plan for securing and forwarding/returning materials, real-time and after testing. For example, seal the response booklets in separate envelopes that are clearly labeled and have the facilitator open the envelopes on camera only after requested to do so, and return the original response booklets to the examiner in prepaid envelopes to ensure test security is not compromised and test records can be maintained.

Digital assets

Practice using the digital assets until the use of the materials is as smooth as a face-to-face administration.

Considerations

Review Table 1 for the specific telepractice considerations for each subtest to be administered.

Input and output requirements and equivalence evidence

Consider the input and output requirements for each task, and the evidence available for telepractice equivalence for the specific task type.

Telepractice Versus Face-to-Face Administration

Although there are no published studies that examine the equivalence of telepractice and face-to-face administration and scoring of the KTEA–3 specifically, a number of studies support equivalence of tasks that are highly similar to the KTEA–3 subtest with respect to constructs assessed and input/output demands. These studies include nonclinical examinees (Galusha-Glasscock et al., 2016; Sutherland et al., 2017; Wright, 2016, 2018), as well as examinees with specific learning disabilities, (Hodge et al., 2019), intellectual disability (Temple et al., 2010), and other clinical conditions (Cullum et al., 2006; Galusha-Glasscock et al., 2016; Grosch, Weiner, Hynan, Shore, & Cullum, 2015; Hildebrand, Chow, Williams, Nelson, & Wass, 2004; Ragbeer et al., 2016; Stain et al., 2011; Temple et al., 2010; Wadsworth, Dhima, et al., 2016; Wadsworth, Galusha-Glasscock, et al., 2018).

Digital Versus Traditional Format

Telepractice involves the use of technology in assessment as well as viewing onscreen stimuli. For these reasons, studies that investigate assessment in digital versus traditional formats are also relevant.

A number of investigations of tasks with similar input/output demands from the Wechsler Individual Achievement Test–Third Edition (WIAT–III; Pearson, 2009), the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV; Wechsler, 2003) and the Wechsler Intelligence Scale for Children–Fifth Edition (WISC–V; Wechsler, 2014) have produced evidence of equivalence when administered and scored via digital or traditional formats to examinees without clinical conditions (Daniel, 2012; Daniel, 2013; Daniel et al., 2014; Raiford, Zhang, et al., 2016). In addition, equivalence has been demonstrated for tasks with similar input/output demands with examinees with clinical conditions, such as intellectual giftedness or intellectual disability (Raiford et al., 2014, Raiford, Zhang, et al., 2016), attention-deficit/hyperactivity disorder or autism spectrum disorder (Raiford, et al., 2015; Raiford, Zhang, et al., 2016), or specific learning disorders in reading or mathematics (Raiford, Drozdick, et al., 2016; Raiford, Zhang, et al., 2016).

Evidence by Subtest

Table 2 lists each KTEA–3 subtest, the input and output requirements, and the evidence of subtest equivalence in telepractice–face-to-face and digital–traditional investigations for similar tasks. The numbers in the evidence columns correspond to the studies in the reference list, which is organized alphabetically in telepractice and digital sections. For clarity, each study is denoted either T or D, with T indicating the study investigated telepractice–face-to-face mode, and D indicating the study addressed digital–traditional format.

State in your report that the test was administered via telepractice, and briefly describe the method of telepractice used. For example, “The KTEA–3 was administered via remote telepractice using digital stimulus materials on Pearson’s Q-global system, and a facilitator monitored the administration onsite using a printed response booklet during the live video connection using the [name of telepractice system, e.g., Zoom] platform.”

Make a clinical judgment, similar to a face-to-face session, about whether or not you are able to gather the examinee’s best performance. Report your clinical decision(s) in your report and comment on the factors that led to the decision to report (or not report) the scores.

For example, “The remote testing environment appeared free of distractions, adequate rapport was established with the examinee via video/audio, and the examinee appeared appropriately engaged in the task throughout the session. No significant technological problems were noted during administration. Modifications to the standardization procedure included: [list]. Tasks highly similar to the KTEA–3 subtests have received initial validation in several samples for remote telepractice and digital format administration, and the results are considered a valid description of the examinee’s skills and abilities.”