Many UX teams rely on remote usability testing to efficiently get design feedback from users. There are两种类型的远程用户测试

  • 适度的远程测试involves a researcher meeting with a participant via remote screen-sharing software, which allows the researcher to provide instructions, observe the user’s interaction with the design in real time, and ask followup questions specific to that participant’s session.
  • Unmoderated remote testing不需要研究人员参加每个测试会议;相反,软件应用程序为用户提供指令,记录其操作,并可能会询问它们预定的后续问题。

Is Unmoderated Testing Right for Your Project?

未经发言的研究不包括研究人员和研究参与者之间的任何直接互动,这都是他们最大的好处和最大的缺点。

因为无需安排每个参与者的个人会面,unmoderated testing is usually much fasterthan a moderated study. It may be possible to launch a study and receive results within just a few hours. Unmoderated studies also allow you to collect feedback from dozens or even hundreds of users simultaneously. And for international studies, you don’t have to get up at an ungodly hour to match users’ time zone.

However, there are importantlimitations of unmoderated usability testing:

  • Early-prototype testing is difficult没有主持人解释和帮助参与者从原型的错误或局限恢复。
  • Without a moderator, participants tend to be less engaged and behaveless realistically in tasks这取决于想象力,决策或情绪反应。

为了更好地理解这一限制,例如,思考差异购物andpurchasing购物可以包括许多不同类型研究与比较 - 没有单一的“正确”的方式。为了现实地购物,参与者必须首先想象自己需要该产品,然后注意细节并进行比较。一个参与者pretendsto shop and isn’t very motivated often will glance only at a few products and quickly select one that seems reasonable. But, in real life, consumers who are spending their own money on a product they actually need behave very differently. Of course, a moderated study is not totally realistic either, but because participants are aware that the moderator is observing them, they will be socially motivated tofully engage with the task。Thus, this s表面压力补偿了患者研究中缺乏个人动机-and even more so in in-person studies.

Unmoderated studies work best for evaluating live websites and apps or highly functional prototypes. They are appropriate for studying activities that don’t require a lot of imagination or emotion from participants.

未经发言的研究需要更细致的规划,即一个受体的研究,因为您不能依赖人类判断,以便在飞行中调整研究程序。对于未经拨出的可用性研究,您需要通过以下所有步骤进行:

List of 6 steps for unmoderated user testing: Define study goals, Select testing software, Write tasks and questions,Pilot test, Recruit participants, and analyze results.

1. Define Study Goals and Participant-Recruitment Criteria

Choosing software should not be the first step in unmoderated research. Before you decide which testing software to use, you should get a clear idea of what you hope to accomplish by doing the study. Then you can select a tool which has the capabilities best suited for your research goals, rather than limiting your study to fit within the technical constraints of a particular tool. Clearly articulated study goals allow you to identify must-have requirements for the testing software.

如果研究目标是......

研究工具必须能够:

Compare how long it takes people to complete a signup and checkout process on your site vs. your competitors

  • Measure time on task
  • Aggregate time on task data
  • 显示个人时间来检查异常值
  • Export time data to a standard spreadsheet format (e.g., .cvs)

帮助大型团队了解为什么用户努力在移动应用程序中完成结账过程

  • Record screen and audio of native mobile applications
  • 高质量的视频捕获,查看详细的互动
  • Easily create clips and compilations of videos to share with the team
Examples of how study goals may affect tool selection

Goals vary from study to study, and a tool that is well suited to one study might be not at all effective for another. Knowing the study objectives is essential in order to make a good tool choice.

在这个阶段,您还需要考虑您想要在您的学习中包含的参与者类型。他们的人口应该是什么?他们需要在哪里?它们是否成为您的系统或专家的用户?他们会匹配一个特定的persona.or user group in your target audience? These questions usually are determined by your study objectives and can also inform your tool selection.

2. Select Testing Software

对于未修改的研究,管理测试的软件对于获得有用的结果绝对至关重要。软件必须通过会话指导参与者并记录发生的事情。它还可以控制研究参与者的选择。

幸运的是,有可提供许多不同的未经寄存器测试服务,不同价格点的功能组合不同。过多的选择意味着您现在可以在选择适合项目要求的工具方面更为重要。(对于许多未修改的测试工具,频繁的功能和定价更改,因此请务必将它们与您在学习时的需求进行比较。)

It’s definitely worth your time to thoroughly investigate and pilot test tools, because migrating a study to a different system due to a technical limitation discovered after you launch a study is not fun! (Neither is trying to integrate data collected by two different tools.)

3. Write Task Instructions and Followup Questions

Many unmoderated testing services include study templates with generic example tasks. Don’t blindly copy them. The tasks that you give participants to do on your site or application should be highly specific to your situation. Generic tasks, such as “What is the purpose of this site,” are unlikely to give you good insights: to really assess the usability of your system you will need to write your own tasks.

In our experiencetraining other companies to run their own remote usability tests那writing任务是大多数研究人员失败的地方in getting the results they need from their studies.

To write good task instructions for an unmoderated study, first articulate what it is you want the user to accomplish (such as: use the help section to answer a question, upgrade an account, or save an article to read later). Then,用特定,现实和可操作的指令描述目标-没有包括使任务变得太容易的提示。你需要different types of task instructions depending on whether you’re doing a qualitative or quantitative study

In unmoderated studies, the activities that you want the participants to conduct have to be evenmore carefully written适用于审核会话的任务。Participants cannot ask for clarification if they don’t understand the instructions and you can’t ask them to try again if they do the wrong thing. If users misinterpret your instructions and perform the wrong task, your test is wasted. Unmoderated task instructions should also explicitly tell users when they should stop; remember, the moderator won’t be there to ask them to move to a different task.

You should also meticulously plan any followup questions. These can include quantitative questions, in which participants rate the subjective difficulty or satisfaction of an activity. Or you may askopen-ended questionswhich prompt users to describe specific parts of the experience. Carefully choose how you phrase your questions; broad wording such as “How would you describe this brand?” may lead unmoderated participants to talk about their past experiences instead of the system they just used.

4.试点测试

A pilot test is a trial session that you run before your actual study begins, in order to discover any problems with your study design or procedure.Pilot testing is a good idea for all user studies但不加节制的螺栓是尤其重要的ies, because there won’t be a moderator available to fix problems while the study is running. Even the most thorough pilot testing can’t catch everything, but you can often detect and fix problems with:

  • 人们误解的任务说明是因为措辞含糊不清
  • 丢失的任务或以错误的顺序呈现(特别是在具有许多任务的复杂研究中)
  • 缺少功能或研究所需内容的原型
  • Incompatibilities or technical limitations which prevent your testing software from capturing the data you want to record — especially if you are trying to test an intranet or native mobile application
    (If your testing software records data via a web-browser extension, then browser or website restrictions may prevent it from capturing entire pages or sites or lead to low-quality recordings that are difficult to analyze. If you’re using a testing service for the first time, do a quick test of the recording process before you even bother setting up your tasks.)

您可以通过研究自己发现一些问题,因为您是参与者,但其他人不会明显,直到您使用自己的设备。确保分析试点研究中收集的数据!

5.招聘参与者

There’s no point in watching people use your system if they don’t match your target audience. Make sure you have some control over who participates in the test, either through screening questions, or by recruiting your own participants.

某些工具仅提供与他们面板的参与者的未修改用户测试,而其他工具则将为您提供分发的URLown pool of testers

使用提供的参与者面板快速简便(特别是如果您的产品与广泛的消费者受众相关的东西)。小组参与者也熟悉研究软件,并且可以在有时间之后立即参加您的考试。几乎每个面板都包括一些基本的人口统计过滤,但大多数研究如果您提供更好的结果屏幕参与者使用关于他们的行为的问题不仅仅是关于他们的年龄或性别。如果您测试的经验仅对符合特定标准的人有关,例如经常在线驾驶汽车或在线购物,请务必使用工具,使您可以编写自己的自定义问题。

The downside of using panel participants is that many do these studies so frequently that they’ve learned to focus on certain aspects of the design and look for things to critique. To compensate for possible “professional testers,” recruit extra participants and exclude data from people who didn’t seem honestly engaged with the activities.

在线意味着您可以像街道另一边的人一样轻松地测试全球的用户。许多未经寄存的测试服务现在具有包括来自世界各地的参与者的面板。如果您的网站目标国际客户那unmoderated testing is a great way to reach a wide variety of locations. Just remember that if you recruit participants to complete the test in other languages, you’ll also need a researcher or translator fluent in that language to interpret the results.

总之:

  • B2B sites, sites that target elite or rich customers, and other services with narrow target audiences usually can’t use panels and must recruit the test participants themselves.
  • B2C站点和其他针对广泛观众的服务通常通过使用面板获得更快和更便宜的结果。
  • 您并不总是需要最大的面板,但如果您要求来自特定区域的用户,有些面板可能需要很长时间才能满足您的学习。(如果您真的需要特定的受众,请提前讨论您的测试服务或面板管理器的要求,如果它们是含糊不清的,请考虑使用不同的服务。)

6.分析结果

Unmoderated studies can quickly accumulate a LOT of data, so you’ll need an organized, analytical approach to turn this data into actionable insights about your design.

如果您收集定性数据,例如参与者操作和评论的视频和录音,则需要查看每个会话录制。用户的口头和书面评论可能是误导性的,所以你必须观看他们的行为,以便理解有效或没有。在一个受体化的研究中,您可以随时随地与参与者一起进行,但在未经检测的研究期间,您需要能够随后观看录制。

Screen recordings are helpful, but, in the absence of an audio recording, it is easy to miss为什么某些行为发生。If nothing is happening on the screen, is it because the participant is reading, or is she thinking about where to click next? The audio recording ofparticipants verbalizing their actionsis essential. (Recordings of the participants’ webcam to capture their facial expressions are also nice to have, but not essential. It can be more difficult to recruit participants who have a webcam and are willing to be recorded, so make it a secondary requirement.)

在每次录像中,您需要识别对设计的问题,问题和对策和消极反应。如果您只有几个录音来审查,但对于具有数十名参与者的大型研究,视频分析变得非常耗时。如果您希望分析大型研究(或进行几种不同的较小研究),请寻找一个未经置信的测试工具,提供具有稳健功能的视频分析,特别是:

  • Tagging videos with timestamped notes as you watch them
  • Aggregating, exporting, sharing, and visualizing the notes you’ve added to your recordings
  • Producing short clips or highlight compilations of important moments in your recordings

Of course, youcan制作笔记和视频剪辑,即使此功能未构建到未拨出的测试工具 - 但包含这些功能的未拨出测试服务,这些功能并不一定花费多于缺少它们的鞍座工具。如果您将分析超过几个小时的录音,那么对加速数据分析的工具来说,值得注意的是要稍微支付。

对于极大的定性研究,考虑可以收集一些定量措施或提供自动转录的工具。These features don’t eliminate the need to carefully review your recordings, but they can certainly expedite the process by directing your attention to specific recordings that are likely to be significant (such as recordings where users had low satisfaction ratings or recordings where particular keywords were mentioned).

If your study is primarilyquantitative那your analysis will be quite different. Metrics such as success rate, task time, and subjective ratings will be automatically collected by your study tool. But to ensure your conclusions are accurate you’ll need to review your data and:

  • Clean the data by identifying and excluding inaccurate values. For example, if a few task times are much shorter or longer than the others, investigate why, andexclude the outliers根据您的分析,如果值不准确,因为参与者没有完全完成任务,或者做错了任务。
  • Perform统计测试评估结果的重要性(especially when your goal is to compare multiple designs or tasks).
  • Generate data visualizations to help communicate your findings to others.

一些未经寄存的测试工具可以自动化排除异常数据点的过程,并且许多都有内置数据可视化图表。但是,由于Cookie-Cutter Charts并不总是显示最重要的结果,如果打算进行定量分析,请确保您的工具包含导出数据的能力,以便您可以使用Excel或专业统计软件执行自己的分析。

Summary

未经发出的研究需要比会议期间的中等检测更少的工作,但在研究开始之前需要细致的提前计划。您可以在全天内了解有关远程用户测试的更多信息和此方法和人员测试之间的关系Usability Testing培训课程,包括关于写作任务,促进会话等等的实践细节。