## Announcements
- template for assignment 1 is [available](https://gitlab.comp.anu.edu.au/comp3900/2026/comp3900-2026-A1-prototyping)
- assignment 1 [due **Monday 24 August, 23:59 on GitLab**](https://canvas.anu.edu.au/courses/2781/assignments/11544)
- assignment 2 specification will be properly available after the A1 is in, [you can see the draft already on Canvas](https://smcclab.au/thirty-nine-hundred-hci/assessments/02-needs-finding.html)
- keep attending labs, if issues, apply for an [extension (see course policies on Canvas)](https://canvas.anu.edu.au/courses/2781/pages/policies)
- any questions, problems, [**use the forum**](https://edstem.org/au/courses/37143/discussion)!
- lab marks come out weekly via Canvas
**Who has a question about assignment 1?**
## Plan for the class
**New module: user research, data, analysis, evaluation**
1. plan data gathering sessions
2. plan and run an interview
3. design a questionnaire
4. understand observation studies
# Main issues in data gathering
- why and how is data gathered?
- what kind of data? (just ratings or more?)
- necessary both for discovering requirements and evaluation
- today: introduce the main issues and techniques
- later weeks: look at how to structure evaluation and analyse different kinds of data
Just ratings? or more? (Photo by Towfiqu barbhuiya on Unsplash)
## Setting Goals
Get information about people, their behaviour and experiences with technology.
- remember: **you are not the user!** user research aims at design-neutral descriptions of real users ([Hornbæk et al., 2025](#ref-hornbaek-intro-hci:2025))
- What information and why?
- Depends on research problem and phase of research/design process, e.g.,
- Comparing two alternative interfaces
- Understanding a context of use
- Measure time taken to complete a task
- Discover how users interact with an existing system
What are the goals? (Photo by Ricardo Arce on Unsplash)
## Identifying Participants
Who are the participants? How many are needed?
- Small group of stakeholders
- Criteria for involvement (“plays video games regularly”)
- Random sampling from large population
- Convenience/volunteer sampling from those available
- Users with specific skills/needs
- Snowball sampling (participants help find more)
- Researchers make a justified choice
- Number of participants: most common in HCI is 12 ([Caine, 2016](#ref-caine-sample-size:2016))
People in the studio (2025)
## Relationship between collector and provider
A data provider *gives* us data. What do they get back?
- Conduct research openly, ethically, [responsibly](https://policies.anu.edu.au/ppl/document/ANUP_007402)
- Informed consent
- Clear communication of benefits and risks
- Respect and acknowledgement
- Particular care for some groups, e.g., Aboriginal and Torres Strait Islander communities.
- Practical: building rapport and understanding (not just taking)
An example consent form for research.
## Ethical Considerations for Data
- data can have strings attached
- privacy
- personal information (including **name!**)
- sensitive information (embarrassing or harmful)
- storage requirements
- do we just use Google drive for everything?
- research needs a data management plan
We have to think about data. (Photo by Claudio Schwarz on Unsplash)
## Triangulation
- *data triangulation:* data is drawn from different sources, times, places, people, etc
- *investigator triangulation:* different researchers (observers, interviewers, etc)
- *triangulation of theories:* different theoretical frameworks
- *methodological triangulation:* different data gathering or research techniques
- this typology is a classic idea from sociology ([Denzin, 2017](#ref-denzin-research-act:2017)): no single method is bias-free, so combine them
- but debated: does combining methods assume there is one “true” picture for them to converge on? ([Blaikie, 1991](#ref-blaikie-triangulation-critique:1991))
Triangles support strong conclusions. (Photo by Charles 2010)
Blaikie’s critique: triangulation quietly imports a positivist assumption — that different methods are views of one underlying reality and should converge on it. Interpretivists reply that different methods constitute different objects of study, not angles on the same one, so “agreement” between methods isn’t automatically validation. Hold this thought — we come back to epistemology later in the course.
## Pilot Studies
A small initial study to help plan a larger study.
- Could involve limited number of participants
- Limited interface or study parameters
- Checks that *expected* data can be obtained
- Can be called a “formative study” (to form the goals of the main study)
- E.g., assess the game platform used in a larger study ([Mohaddesi & Harteveld, 2020](#ref-mohaddesi-pilot-studies:2020))
# Interviews
> Conversation with a purpose ([Kahn & Cannell, 1957](#ref-kahn1957dynamics))
## Ask the users
- Interviews make a lot of sense in HCI
- What should a system do? Just ask.
- How should it work? Just ask.
- Did it work well? Ask away.
- How would you change it? Ask.
An interview. (Photo by Sam McGhee on Unsplash)
## Unstructured Interviews
- exploratory, similar to conversations
- go into depth on experiences
- questions are open: no expectation on the content of answers or subsequent questions
- **probing:** can you tell me more about …?
- benefit: generate rich, complex data
- limitation: time consuming to analyse
> Could you tell me about your experience using the NewWidgetApp?
> You mentioned that you enjoyed (feature X), why was that?
> Can you explain more about what happened when you used (feature X)?
## Structured Interviews
- questions are **predetermined**
- questions need to be short and clear
- questions can be closed (answers from specific options)
- typically, whole interview is scripted
- low skill to deliver
- fast to deliver
- useful when goals are clearly understood and questions and responses can be identified
> What is your most used code editor: VSCode, nvim, emacs, notepad.exe, or something else?
> How often do you push your assessments to Gitlab: every minute, every hour, every day, just once?
## Semi-structured Interviews
- features of both **structured** and **unstructured**
- typically: script with questions on main topics but discussion and probing follows each question
- intended to be somewhat replicable
- **probe:** neutral questions to gain more detail
- **prompt:** reminder of some part of the topic to gain specific information
- careful: prompting can preempt answers (bias!)
> How are things going with the interaction experience?
> Are there any \[initial\] impressions you want to share?
> Can you describe the connections between your movements and the resulting sound?
([Reed, 2023](#ref-reed2023imagining))
## Focus Groups
- interview with multiple participants (could be called a group interview)
- facilitator prompts discussion
- group members can influence each other (good or bad?)
- good for talking to lots of people.
- raising diverse viewpoints
- **facilitation:** need to be careful
- **groupthink:** generally considered harmful
(I don’t find the “Focus Group” term super useful in my academic research—but they are widely used in industrial/government research.)
A group! Could be focussed. (Photo by Antenna on Unsplash)
## Activity: Plan interview questions
**Do:** Let’s plan a *semi-structured* interview!
> What questions should be asked in a semi structured interview about student’s user experience with the “catchbox” (soft microphone used during lectures)?
Use the poll everywhere link to suggest interview questions and vote on the best ones.
**Ideate** for 2-3 minutes, **vote** for 1 minute, then let’s discuss.
## Developing Interview Questions
- long questions are confusing
- jargon / technical language may be confusing (e.g., popover, jumbotron, nav bar, onboarding popup)
- keep questions neutral (e.g., “How much do you love this lecture?” is a leading question)
- questions need to support data gathering **goals**
- may not need to ask questions not related to goals
**Careful:** it’s easy to (accidentally) write bad interview questions!
## Running the Interview
- usually have a script for introductions, consent, etc (read verbatim)
- listen more than talk
- respond with sympathy but without bias
- enjoy the experience
- interviewing is **hard work**!
Yichen Wang and Marlene Radice in 2024
## Capturing Data
- usually interviewing involves a combination of:
- notes
- audio
- video
- different data have different issues and needs
- notes: type up, handwriting (bias?)
- audio: convert format, [transcribe (suggest Aiko)](https://sindresorhus.com/aiko)
- video: edit, store (large file sizes, anonymity)
Editing videos from a study in 2015.
# Questionnaires
## Structure and Format
- written method of gathering structured data
- ask questions of a subset of people to generalise to a wider population ([Müller et al., 2014](#ref-mueller-survey-research:2014))
- sometimes called a “survey”, technically a survey is the whole study
- *questionnaire* or *survey instrument* is the paper form with questions
- questions can be open or closed
- often questionnaire used for demographic data
- standard survey instruments are often questionnaires
Photo by Nguyen Dang Hoang Nhu on Unsplash
## Open text questions
Questions might be something like:
> What were the strengths of this course?
> Please provide any suggestions about how this course could be improved?
- the researcher might have little control over how seriously these questions are taken.
- potential to gather rich data
- **careful:** use these only with really open concepts
Wide open (2010)
## Closed form questions
- closed form questions have preset responses from which the respondent must select.
- unordered responses
- rating scale questions
- **careful:** closed form questions design should be neutral and include most likely or relevant responses
> What is your favourite fruit (select one answer): plum, tomato, pineapple”
> How interesting is this lecture? Select a number from 1 (interesting) to 10 (amazingly interesting).
## Likert or Lump It
- Likert scale questions are very common in questionnaires
- named after [Rensis Likert](https://en.wikipedia.org/wiki/Rensis_Likert) (social scientist) ([Likert, 1932](#ref-likert1932technique))
- The question includes a statement, e.g.: “The workload was appropriate for this course”
- A number of levels of agreement are provided, e.g.: “strongly disagree, disagree, neutral, agree, strongly agree.”
- 5-points is common, 3, 7, 9 or a continuous slider is also possible.
Some typical agreement scale questions
## Semantic Differential Scale
- ratings of an object, concept, situation, etc
- the answer is a point between two opposite concepts
- e.g., describe your experience of using the (insert system here)
- difficult to use — easy to use
- boring — fun
- slow to learn — fast to learn
## Existing Survey Instruments
- Researchers sometimes choose to use well-known existing survey instruments rather than create their own.
- Existing surveys may be validated by having been tested and applied in many other studies.
- Sometimes the survey instrument comes with built-in instructions for analysing results.
- Let’s look at some examples that are typical in HCI:
- System Usability Scale (SUS) ([Brooke, 1995](#ref-brooke-sus:1995); [UIUX Trend, 2024](#ref-uiuxtrend2024sus))
- NASA Task Load Index (TLX)
- Creativity Support Index (CSI) ([Carroll & Latulipe, 2009](#ref-carroll-csi:2009); [Cherry & Latulipe, 2014](#ref-cherry-csi:2014))
## System Usability Scale Questions
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
## NASA Task Load Index Questions
1. How mentally demanding was the task?
2. How physically demanding was the task?
3. How hurried or rushed was the pace of the task?
4. How successful were you in accomplishing what you were asked to do?
5. How hard did you have to work to accomplish your level of performance?
6. How insecure, discouraged, irritated, stressed and annoyed were you?
[Worksheets provided!](https://ntrs.nasa.gov/citations/20000021488) ([Hart & Staveland, 1988](#ref-hart-nasa-tlx:1988)). Extra questions [weight the ratings](https://humansystems.arc.nasa.gov/groups/tlx/), though most studies now skip weighting and use “Raw TLX” ([Hart, 2006](#ref-hart-tlx-20-years:2006)).
NASA TLX
## Creativity Support Index
- **Idea:** measure how well a system can support creativity ([Carroll & Latulipe, 2009](#ref-carroll-csi:2009); [Cherry & Latulipe, 2014](#ref-cherry-csi:2014))
- **creativity support tools:** writing, visualisation, video editing, music tools etc
- creativity is a bit hard to define, but the CSI includes:
- exploration, expressiveness, immersion, enjoyment, results worth effort, collaboration
- inspired by TLX, 2 questions per factor + 15 paired comparisons.
- It’s hard to write survey questions! Beginners may do better with a standard questionnaire.
- More questions means more analysis work—counter-productive without a way to aggregate them.
- Rating scale data isn’t continuous: use non-parametric significance tests—more powerful and consistent than t-tests at typical HCI sample sizes ([Kaptein et al., 2010](#ref-kaptein-likert-analysis:2010))—though statisticians still argue about this ([Norman, 2010](#ref-norman-likert-parametric:2010)).
- The distribution matters: prefer box plots over mean and standard error charts.
Some boxplots from a survey.
The Likert analysis question is a live argument, not settled: Kaptein, Nass & Markopoulos (CHI 2010) show non-parametric tests are more powerful and consistent at typical HCI sample sizes, while Geoff Norman (2010, health sciences education) argues parametric tests are robust to ordinal data and skewed distributions, and that the “you can’t do that with Likert data” objection is overblown. Both papers are from the same year and both are highly cited. Students will meet both positions in the literature — the honest advice is to know why you chose your test, not to treat either rule as gospel.
## Activity: Write a questionnaire
**Talk:** 🙋🏽♀️🗣️ Let’s write a questionnaire about ANU students’ class enrolment experience.
- What questions should we ask on a questionnaire about ANU student’s enrolment experience? If closed list options, if open write “(open question)”
Talk for 2-3 minutes and add some examples on [PollEverywhere](https://PollEv.com/charlesmarti205), then let’s have some discussion.
ANU course enrolment.
# Observation
## Direct Observation in the Wild
- watching people for science
- observing system use in its normal context
- e.g., join a tour group to observe a navigation app
- e.g., watch a concert to observe music technology
- “in the wild”: everyday contexts, not the lab ([Chamberlain et al., 2012](#ref-chamberlain-research-wild:2012))
### Example framework from Rogers et al. ([2023](#ref-rogers-beyond-hci:2023)):
- **The person:** who is using the technology?
- **The place:** where are they using it?
- **The thing:** what are they doing with it?
We don’t actually stalk people. (Photo by Philippe Bout on Unsplash)
## Ethnography
- Literally “study of culture”
- immersion and participation in a research context.
- e.g., embed in a factory (talk to workers, perform tasks) to create a factory management system
- often applied to workplaces: factories, hospitals, etc
- study participants in their own context rather than in a lab/classroom
- its job in HCI is debated: “implications for design” vs “what people actually do” vs interpretation ([Crabtree et al., 2009](#ref-crabtree-ethnography-harmful:2009); [Dourish, 2006](#ref-dourish-implications:2006); [Suchman, 1987](#ref-suchman:1987))
(Photo by Homa Appliances on Unsplash)
Suchman’s Plans and Situated Actions (1987, Xerox PARC) is where this tradition starts in HCI: work as actually done differs from work as officially described. Twenty years on, the field was still arguing about what ethnography is for. These two papers actually disagree — Dourish says ethnography’s value is analytic, not a requirements bullet-list; Crabtree et al. say the interpretive turn went too far and ethnography should stick to describing what people observably do.
## Direct Observation in Controlled Environments
- observation in a lab-based setting
- could be possible to identify close details
- record all aspects of technology use
- record data, audio, video as well as observations
Observation via video in a lab (2015)
## Think-Aloud Technique
- **Problem:** observers don’t know what participants are thinking
- **Solution:** ask participants to say everything they are thinking and trying to do when using an interface, so we know!
- theory from psychology: participants verbalise the contents of their working memory, concurrently or retrospectively ([Ericsson & Simon, 1980](#ref-ericsson-verbal-reports:1980))
- can produce very useful data
- hard work for the participant
- needs careful facilitation from the observer
- the standard tool of industry UX practice ([Fan et al., 2020](#ref-fan-think-aloud-industry:2020))
- but theory and practice disagree: the strict protocol is rarely followed in the field ([Boren & Ramey, 2000](#ref-boren-thinking-aloud:2000))
- more: [Thinking aloud, the \#1 usability tool](https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/)
Ericsson & Simon’s theory demands a strict protocol: participants verbalise only, no probing, and the only permitted intervention is “keep talking”. Boren & Ramey watched practitioners at work and found they systematically break these rules — chatting, probing, reassuring — so the method as actually practised isn’t licensed by the theory it cites for validity. Their proposal: ground it in speech communication theory instead, since the observer is unavoidably a listener in a conversation, and manage that role deliberately rather than pretending to be absent. Another science-vs-practice tension, like Raw TLX.
## Video-cued Recall
- it can be difficult to remember details from a long sequence of tasks.
- techniques like video cued recall can help participants provide a commentary on an experience.
- the idea is: you record video of the participant completing an an interaction and then have an open-ended discussion while watching it back.
Videos of an interaction for discussion.
## Indirect Observation
- We can observe without being present
- Useful for
- embedding systems in people’s everyday life
- interacting with participants remotely
- tracking more participants than you could directly observe
- **nonreactive:** avoids *reactivity*, where being studied changes what participants do ([Hornbæk et al., 2025](#ref-hornbaek-intro-hci:2025))
## Diaries
- participants write about their experiences with a system regularly in a diary
- easy for the researcher at the collection stage
- takes continuous effort from the participants (reminders? structure?)
- relies on participant’s memory and subjective account
- video and photos can reinforce written accounts
- the capture medium (photos, audio, tangible objects) shapes what participants record and later recall ([Carter & Mankoff, 2005](#ref-carter-diary-media:2005))
The participant just writes down their experience! (2025)
## Logs, Analytics, Scraping
- **Interaction log:** system data showing exactly what the participant did and when
- e.g., key presses, mouse moves, GUI events, sensor data
- time spent on actions or software
- unobtrusive and automatic, but lots of data: needs visualisation or analysis to develop findings
- scraping public sources (e.g., social media) can also be observation
**Careful:** scraping raises ethical concerns! Few social media users know researchers use their public posts, and most think consent should be asked ([Fiesler & Proferes, 2018](#ref-fiesler-twitter-ethics:2018)).
Interaction data from a concert (2014)
## Activity: Would you rather (observation edition)
**Talk:** 🤷🧠🗣️ As an HCI researcher, would you rather:
1. observe directly in the wild
2. observe directly in a lab
3. observe indirectly through a diary
4. observe indirectly through data analytics
Discuss for 2-3 minutes in groups, rank on [PollEverywhere](https://pollev.com/charlesmarti205), and then we will hear some responses!
# Data Gathering in Practice
Back to our issues:
1. goals
2. participants
3. ethics, consent with participants
4. ethics of data
5. triangulation
6. pilot studies vs main studies
## Choosing Techniques ([Rogers et al., 2023](#ref-rogers-beyond-hci:2023))
| technique | good For | kind of data | issues |
|----|----|----|----|
| interviews | exploring issues | mostly qualitative | skilled work but high value |
| focus groups | multiple viewpoints | mostly qualitative | efficient, groupthink risk |
| questionnaires | specific questions | quantitative and qualitative | lots of participants, hard to design |
| direct observation in the wild | context of use | mostly qualitative | very useful data, very time consuming |
| direct observation in a controlled environment | capturing details | quantitative and qualitative | situation can be artificial/unrealistic |
| indirect observation | automatic or remote | quantitative and qualitative | hard to analyse lots of data, can be authentic |
## Adapting for Different Participants
- data gathering should adapt to different participants
- **children:** think and react differently to adults, e.g., adapt scales to visual representations ([Putnam et al., 2020](#ref-putnam-sus-children:2020))
- **people with disabilities:** caregivers might be involved to adapt interview questions
- **animals(!):** hard to interpret behaviour, need to design interfaces to work without hurting animals ([Mancini et al., 2017](#ref-mancini-aci:2017))
ACI is real! (Photo by Alison Pang on Unsplash)
## Gathering Data Remotely
- expectations for remote data gathering changed completely during pandemic!
- can access participants in different countries, age groups, abilities, specific or expert users
- observation could be recorded and then analysed later
- best practices might include ([Mastrianni et al., 2021](#ref-mastrianni-remote:2021)):
- run pilot tests before sessions
- have backups in place in case of issues
- inform participants of technical requirements
- use retrospective questioning if think-aloud has issues
Interviews and observations can happen remotely! (Photo by Chris Montgomery on Unsplash)
## Questionnaires vs Interviews
- Questionnaires, while very useful can only tell us information that we know to ask about.
- Interviews let us find out information that didn’t know to begin with.
- Questionnaires are fast and can address lots of participants—a broad approach
- Interviews are slow but are a deep approach.
- Interviews also require good conversational skills and experience to conduct well.
- Both collect what people *say*: beware the **say–do gap** between reports and behaviour ([Hornbæk et al., 2025](#ref-hornbaek-intro-hci:2025))—observation fills it in.
## Questions: Who has a question?
**Who has a question?**
- I can take *cathchbox* question up until 2:55
- For after class questions: meet me outside the classroom at the bar (for 30 minutes)
- Feel free to ask about **any aspect of the course**
- Also feel free to ask about **any aspect of computing at ANU**! I may not be able to help, but I can listen.
Meet you at the bar for questions. 🍸🥤🫖☕️ Unfortunately no drinks served! 🙃
# References
Blaikie, N. W. H. (1991). A critique of the use of triangulation in social research. *Quality and Quantity*, *25*(2), 115–136.
Boren, T., & Ramey, J. (2000). Thinking aloud: Reconciling theory and practice. *IEEE Transactions on Professional Communication*, *43*(3), 261–278.
Brooke, J. (1995). SUS: A quick and dirty usability scale. *Usability Eval. Ind.*, *189*.
Caine, K. (2016). Local standards for sample size at CHI. *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, 981–992.
Carroll, E. A., & Latulipe, C. (2009). The creativity support index. *CHI ’09 Extended Abstracts on Human Factors in Computing Systems*, 4009–4014.
Carter, S., & Mankoff, J. (2005). When participants do the capturing: The role of media in diary studies. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 899–908.
Chamberlain, A., Crabtree, A., Rodden, T., Jones, M., & Rogers, Y. (2012). Research in the wild: Understanding “in the wild” approaches to design and development. *Proceedings of the Designing Interactive Systems Conference*, 795–796.
Cherry, E., & Latulipe, C. (2014). Quantifying the creativity support of digital tools through the creativity support index. *ACM Trans. Comput.-Hum. Interact.*, *21*(4).
Crabtree, A., Rodden, T., Tolmie, P., & Button, G. (2009). Ethnography considered harmful. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 879–888.
Denzin, N. K. (2017). *The research act: A theoretical introduction to sociological methods*. Routledge.
Dourish, P. (2006). Implications for design. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 541–550.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. *Psychological Review*, *87*(3), 215–251.
Fan, M., Shi, S., & Truong, K. N. (2020). Practices and challenges of using think-aloud protocols in industry: An international survey. *Journal of Usability Studies*, *15*(2), 85–102.
Fiesler, C., & Proferes, N. (2018). “Participant” perceptions of Twitter research ethics. *Social Media + Society*, *4*(1).
Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 20 years later. *Proceedings of the Human Factors and Ergonomics Society Annual Meeting*, *50*(9), 904–908.
Hart, S. G., & Staveland, L. E. (1988). *Development of NASA-TLX (task load index): Results of empirical and theoretical research*. *52*, 139–183.
Hornbæk, K., Kristensson, P. O., & Oulasvirta, A. (2025). *Introduction to human-computer interaction*. Oxford University Press.
Kahn, R. L., & Cannell, C. F. (1957). *The dynamics of interviewing; theory, technique, and cases*. John Wiley & Sons.
Kaptein, M. C., Nass, C., & Markopoulos, P. (2010). Powerful and consistent analysis of likert-type rating scales. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 2391–2394.
Likert, R. (1932). A technique for the measurement of attitudes. *Archives of Psychology*.
Mancini, C., Lawson, S., & Juhlin, O. (2017). Animal-computer interaction: The emergence of a discipline. *International Journal of Human-Computer Studies*, *98*, 129–134. https://doi.org/
Mastrianni, A., Kulp, L., & Sarcevic, A. (2021). Transitioning to remote user-centered design activities in the emergency medical field during a pandemic. *Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems*.
Mohaddesi, O., & Harteveld, C. (2020). The importance of pilot studies for gamified research: Pre-testing gamettes to study supply chain decisions. *Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play*, 316–320.
Müller, H., Sedley, A., & Ferrall-Nunge, E. (2014). Survey research in HCI. In J. S. Olson & W. A. Kellogg (Eds.), *Ways of knowing in HCI* (pp. 229–266). Springer.
Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. *Advances in Health Sciences Education*, *15*(5), 625–632.
Putnam, C., Puthenmadom, M., Cuerdo, M. A., Wang, W., & Paul, N. (2020). Adaptation of the system usability scale for user testing with children. *Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems*, 1–7.
Reed, C. N. (2023). *Imagining & sensing: Understanding and extending the vocalist-voice relationship through biosignal feedback* \[PhD thesis, Queen Mary University of London\].
Rogers, Y., Sharp, H., & Preece, J. (2023). *Interaction design: Beyond human-computer interaction, 6th edition*. John Wiley & Sons, Inc.
Suchman, L. A. (1987). *Plans and situated actions: The problem of human-machine communication*. Cambridge University Press.
UIUX Trend. (2024). *Measuring system usability scale (SUS)*. UIUX Trend.
