Many UX teams rely on remote usability testing to efficiently get design feedback from users. There are两种类型的远程用户测试：
- 适度的远程测试involves a researcher meeting with a participant via remote screen-sharing software, which allows the researcher to provide instructions, observe the user’s interaction with the design in real time, and ask followup questions specific to that participant’s session.
- Unmoderated remote testing不需要研究人员参加每个测试会议;相反，软件应用程序为用户提供指令，记录其操作，并可能会询问它们预定的后续问题。
Is Unmoderated Testing Right for Your Project?
因为无需安排每个参与者的个人会面，unmoderated testing is usually much fasterthan a moderated study. It may be possible to launch a study and receive results within just a few hours. Unmoderated studies also allow you to collect feedback from dozens or even hundreds of users simultaneously. And for international studies, you don’t have to get up at an ungodly hour to match users’ time zone.
However, there are importantlimitations of unmoderated usability testing:
- Early-prototype testing is difficult没有主持人解释和帮助参与者从原型的错误或局限恢复。
- Without a moderator, participants tend to be less engaged and behaveless realistically in tasks这取决于想象力，决策或情绪反应。
为了更好地理解这一限制，例如，思考差异购物andpurchasing。购物可以包括许多不同类型研究与比较 - 没有单一的“正确”的方式。为了现实地购物，参与者必须首先想象自己需要该产品，然后注意细节并进行比较。一个参与者pretendsto shop and isn’t very motivated often will glance only at a few products and quickly select one that seems reasonable. But, in real life, consumers who are spending their own money on a product they actually need behave very differently. Of course, a moderated study is not totally realistic either, but because participants are aware that the moderator is observing them, they will be socially motivated tofully engage with the task。Thus, this s表面压力补偿了患者研究中缺乏个人动机-and even more so in in-person studies.
Unmoderated studies work best for evaluating live websites and apps or highly functional prototypes. They are appropriate for studying activities that don’t require a lot of imagination or emotion from participants.
1. Define Study Goals and Participant-Recruitment Criteria
Choosing software should not be the first step in unmoderated research. Before you decide which testing software to use, you should get a clear idea of what you hope to accomplish by doing the study. Then you can select a tool which has the capabilities best suited for your research goals, rather than limiting your study to fit within the technical constraints of a particular tool. Clearly articulated study goals allow you to identify must-have requirements for the testing software.
Compare how long it takes people to complete a signup and checkout process on your site vs. your competitors
Goals vary from study to study, and a tool that is well suited to one study might be not at all effective for another. Knowing the study objectives is essential in order to make a good tool choice.
在这个阶段，您还需要考虑您想要在您的学习中包含的参与者类型。他们的人口应该是什么？他们需要在哪里？它们是否成为您的系统或专家的用户？他们会匹配一个特定的persona.or user group in your target audience? These questions usually are determined by your study objectives and can also inform your tool selection.
2. Select Testing Software
It’s definitely worth your time to thoroughly investigate and pilot test tools, because migrating a study to a different system due to a technical limitation discovered after you launch a study is not fun! (Neither is trying to integrate data collected by two different tools.)
3. Write Task Instructions and Followup Questions
Many unmoderated testing services include study templates with generic example tasks. Don’t blindly copy them. The tasks that you give participants to do on your site or application should be highly specific to your situation. Generic tasks, such as “What is the purpose of this site,” are unlikely to give you good insights: to really assess the usability of your system you will need to write your own tasks.
In our experiencetraining other companies to run their own remote usability tests那writing任务是大多数研究人员失败的地方in getting the results they need from their studies.
To write good task instructions for an unmoderated study, first articulate what it is you want the user to accomplish (such as: use the help section to answer a question, upgrade an account, or save an article to read later). Then,用特定，现实和可操作的指令描述目标-没有包括使任务变得太容易的提示。你需要different types of task instructions depending on whether you’re doing a qualitative or quantitative study。
In unmoderated studies, the activities that you want the participants to conduct have to be evenmore carefully written比适用于审核会话的任务。Participants cannot ask for clarification if they don’t understand the instructions and you can’t ask them to try again if they do the wrong thing. If users misinterpret your instructions and perform the wrong task, your test is wasted. Unmoderated task instructions should also explicitly tell users when they should stop; remember, the moderator won’t be there to ask them to move to a different task.
You should also meticulously plan any followup questions. These can include quantitative questions, in which participants rate the subjective difficulty or satisfaction of an activity. Or you may askopen-ended questionswhich prompt users to describe specific parts of the experience. Carefully choose how you phrase your questions; broad wording such as “How would you describe this brand?” may lead unmoderated participants to talk about their past experiences instead of the system they just used.
A pilot test is a trial session that you run before your actual study begins, in order to discover any problems with your study design or procedure.Pilot testing is a good idea for all user studies但不加节制的螺栓是尤其重要的ies, because there won’t be a moderator available to fix problems while the study is running. Even the most thorough pilot testing can’t catch everything, but you can often detect and fix problems with:
- Incompatibilities or technical limitations which prevent your testing software from capturing the data you want to record — especially if you are trying to test an intranet or native mobile application
(If your testing software records data via a web-browser extension, then browser or website restrictions may prevent it from capturing entire pages or sites or lead to low-quality recordings that are difficult to analyze. If you’re using a testing service for the first time, do a quick test of the recording process before you even bother setting up your tasks.)
There’s no point in watching people use your system if they don’t match your target audience. Make sure you have some control over who participates in the test, either through screening questions, or by recruiting your own participants.
某些工具仅提供与他们面板的参与者的未修改用户测试，而其他工具则将为您提供分发的URLown pool of testers。
The downside of using panel participants is that many do these studies so frequently that they’ve learned to focus on certain aspects of the design and look for things to critique. To compensate for possible “professional testers,” recruit extra participants and exclude data from people who didn’t seem honestly engaged with the activities.
在线意味着您可以像街道另一边的人一样轻松地测试全球的用户。许多未经寄存的测试服务现在具有包括来自世界各地的参与者的面板。如果您的网站目标国际客户那unmoderated testing is a great way to reach a wide variety of locations. Just remember that if you recruit participants to complete the test in other languages, you’ll also need a researcher or translator fluent in that language to interpret the results.
- B2B sites, sites that target elite or rich customers, and other services with narrow target audiences usually can’t use panels and must recruit the test participants themselves.
Unmoderated studies can quickly accumulate a LOT of data, so you’ll need an organized, analytical approach to turn this data into actionable insights about your design.
Screen recordings are helpful, but, in the absence of an audio recording, it is easy to miss为什么某些行为发生。If nothing is happening on the screen, is it because the participant is reading, or is she thinking about where to click next? The audio recording ofparticipants verbalizing their actionsis essential. (Recordings of the participants’ webcam to capture their facial expressions are also nice to have, but not essential. It can be more difficult to recruit participants who have a webcam and are willing to be recorded, so make it a secondary requirement.)
- Tagging videos with timestamped notes as you watch them
- Aggregating, exporting, sharing, and visualizing the notes you’ve added to your recordings
- Producing short clips or highlight compilations of important moments in your recordings
Of course, youcan制作笔记和视频剪辑，即使此功能未构建到未拨出的测试工具 - 但包含这些功能的未拨出测试服务，这些功能并不一定花费多于缺少它们的鞍座工具。如果您将分析超过几个小时的录音，那么对加速数据分析的工具来说，值得注意的是要稍微支付。
对于极大的定性研究，考虑可以收集一些定量措施或提供自动转录的工具。These features don’t eliminate the need to carefully review your recordings, but they can certainly expedite the process by directing your attention to specific recordings that are likely to be significant (such as recordings where users had low satisfaction ratings or recordings where particular keywords were mentioned).
If your study is primarilyquantitative那your analysis will be quite different. Metrics such as success rate, task time, and subjective ratings will be automatically collected by your study tool. But to ensure your conclusions are accurate you’ll need to review your data and:
- Clean the data by identifying and excluding inaccurate values. For example, if a few task times are much shorter or longer than the others, investigate why, andexclude the outliers根据您的分析，如果值不准确，因为参与者没有完全完成任务，或者做错了任务。
- Perform统计测试评估结果的重要性(especially when your goal is to compare multiple designs or tasks).
- Generate data visualizations to help communicate your findings to others.