## Announcements
- assignment 2 due next Tuesday (6 October — Monday is a public holiday)
- remember to use your tutorial time for meeting research clusters and collecting data.
- remember to follow the step-by-step guide.
- go fork [the repo!](https://gitlab.comp.anu.edu.au/comp3900/2026/comp3900-2026-A2-needs-finding)
**Markdown Formatting Check:** There is a CI/CD job that checks your markdown formatting using the [`markdownlint-cli`](https://github.com/igorshubovych/markdownlint-cli) tool. Syntax rules are listed [here](https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md) in our script, rules `MD013` and `MD041` are disabled. All other rules are active.
## Plan for the class
- research questions
- about evaluation: from a question to a yardstick
- types of evaluation
- planning evaluations
- evaluation by inspection, with a case study: the Hawaii false missile alert
- activity: two ways to evaluate nanojam
# Research Questions
For the final project you will have to choose a research question to explore.
This is a clear question (one sentence) that guides the design of your research project.
RQs have been called [survival beacons](https://lennartnacke.com/how-to-choose-a-good-research-question/#research-questions-are-survival-beacons) because they should guide all aspects of our research plans.
How do we choose a research question and write it clearly?
Important skill for any research activity.
## Four steps to write a research question
This framework inspired by Lennart Nacke, [everybody’s favourite HCI writing coach on LinkedIn](https://lennartnacke.com/how-to-choose-a-good-research-question/#research-questions-are-survival-beacons).
1. Outline a broad area of interest
2. Identify a problem that needs solving
3. Justify solving this problem
4. Write the question
To be clear, a research question starts with a question word (what, how, why, can, do, should) and ends with a question mark. It can just be one sentence.
Seems too easy… let’s try it together.
Lennart Nacke’s 4-step research question framework
## A worked research question example
1. haptic wearable interfaces.
2. keeping focussed in complicated meetings
3. lack of awareness in meeting can lead to poor work performance and embarrassment
4. here we go:
> What effects can a haptic wearable interface have on lack of awareness during meetings and later work performance?
Encodes the broad area, the problem, the justification, the context, etc.
## Is the problem worth solving?
The four steps get you a well-formed question. They don’t tell you if it’s a question worth asking.
Two useful ways to find a gap ([Hornbæk et al., 2025, Section 45.2](#ref-hornbaek-intro-hci:2025)):
- **Knowledge gap:** what do we not understand yet, given our present understanding?
- **Capability gap:** what are we not able to *do* yet?
Try working backwards: instead of starting with a problem and then asking who cares, start from **what people need to understand or do better**.
> “Good research produces results that improve stakeholders’ capacity to solve important problems.”
## Three ways a research question goes wrong
**1. Wrong level of abstraction.** Too specific and almost nobody is affected:
> Why do aging users make errors in Microsoft Word 17.3 when sitting near a pool in Belize?
Raise it until it covers many people — but not so far that it can’t be answered in one study.
## Three ways a research question goes wrong (2)
**2. Confounded with a solution.** This one has the method baked in before any thinking has happened:
> What does the *interview method* tell us about children’s motivation to use tablets in schools?
Better — solution-neutral, so the method is still an open choice:
> What motivations do children have for using tablets in schools?
## Three ways a research question goes wrong (3)
**3. Not refutable.** A question that presupposes its own answer isn’t a question.
Very easy to do accidentally when you’re attached to a pet theory, or want to show your system beats the baseline. Your RQ should allow **more than one** answer — including one you won’t like.
And on novelty:
> “Off-hand comments about a problem being novel are almost always wrong.” ([Hornbæk et al., 2025, Section 45.2](#ref-hornbaek-intro-hci:2025))
Nearly every HCI problem has precursors. Finding them is your job, not an optional extra.
## Research Question Bingo
**Problems to Solve:**
1. family meal planning
2. language learning while commuting
3. caring for houseplants
4. focus during remote work
5. medication schedules
6. teaching kids about conservation
7. organising hobby collections
8. non-verbal communication
9. practicing music in small spaces
10. tracking community events
## Activity: Write a Research Question
**Do:** Let’s write a research question!
> Together, let’s [Spin the wheels](https://smcclab.au/thirty-nine-hundred-hci/resources/research-question-wheels.html) to decide on a broad area and a problem.
Then, decide on a “justification” and write a research question. Remember that the RQ should include the broad area, the problem, and the justification.
Use the poll everywhere link to suggest research questions and vote on the best ones.
**Write** for 2-3 minutes, **vote** for 1 minute, then let’s discuss.
**Extension:** Before you vote, run the candidates past the three failure modes: is the **abstraction** right, is a **solution** smuggled in, and is it **refutable**?
# About Evaluation
Where we are in the course, following Rogers et al. ([2023](#ref-rogers-beyond-hci:2023)):
- **Week 4:** how to collect data (interviews, questionnaires, observation, think-aloud, logs)
- **Weeks 5 and 6:** how to analyse it
- **This week:** how to design the *study* — which setting, which kind of study, which variables, and whether you need users in the room at all
Same collection and analysis techniques as needs-finding. What changes is that now there is an artefact, and a question about it.
## What is evaluation?
- **Evaluation:** collecting and analysing data from user experiences with an artefact.
- **Goal:** to improve the artefact’s design.
- **Addresses:** functionality, usability, user experience
- Appropriate for all different kinds of artefacts and prototypes.
- Methods vary according to goals.
- **Understanding people**
- Users may not have the same experiences or perspectives as you do
- Different users use software differently
- **Understanding designs**
- Proof that ideas work
- Understand limitations, affordances, applications
- **Business**
- Invest in the right ideas
- Find problems to solve (before production, before next iteration, etc.)
- **Research**
- Evidence for new interactive systems
- Empirical proof of hypotheses
- New knowledge to answer research questions
## What should you evaluate/measure?
> Does the design do what the users need and want?
Examples:
- **Game App Developers:** Whether young adults find their game fun and engaging compared to other games
- **Government authority:** Whether their online service is accessible to users with a disability
- **Children’s talking toy designers:** Whether six-year-olds enjoy the voice, feel of the soft toy, and can use safely
Usability goals within user experience goals, after Rogers et al. (2023)
## Usability and Usability Goals
Six usability goals:
- Effective to use (effectiveness)
- Efficient to use (efficiency)
- Safe to use (safety)
- Having good utility (utility)
- Easy to learn (learnability)
- Easy to remember how to use (memorability)
([Rogers et al., 2023](#ref-rogers-beyond-hci:2023))
Image: dtravisphd on Unsplash
## Where should you evaluate your design?
Depends on your evaluation goal!
- Lab studies (controlled settings)
- In-the-wild studies (natural settings)
- Remote studies (online behaviour)
Image: Unsplash, UX Indonesia
## Formative vs Summative Evaluation
*When* you evaluate changes what the evaluation is for ([Rogers et al., 2023, Section 14.2](#ref-rogers-beyond-hci:2023)):
- **Formative evaluation:**
- during design, checking the product still meets users’ needs
- from early sketches to a nearly finished design
- feeds the next iteration
- **Summative evaluation:**
- assessing whether a finished product is successful
- against a target: a usability goal, a baseline, a standard
For the final project you will almost certainly be doing a **formative** evaluation of a prototype.
## Overview: from a goal to a yardstick to a study
Set the **yardstick** (what counts as “good”) first, and the kind of study follows ([Hornbæk et al., 2025](#ref-hornbaek-intro-hci:2025), table 40.2).
| Yardstick | The system should… | Kind of study |
|----|----|----|
| No critical usability problems | not make users err, get lost, or give up | think-aloud study |
| Comply with guidelines | match known traits of good systems | heuristic evaluation |
| Meet usability goals | hit quantitative targets | summative usability test |
| Compare favourably to X | beat a baseline on some measure | experiment |
| Compatibility with user practices | be adopted into everyday life | field study |
No data-collection column: that was week 4. The yardstick picks the study; the study picks the techniques.
## Example: three questions about one app
nanojam (week 7): three questions, three yardsticks, three studies.
1. *Can a first-time user make a repeating pattern in the command line without help?* **No critical usability problems.** Think-aloud in the lab, six people, log where they stall.
2. *Is the pads interface faster to a first pattern than the command line?* **Compare favourably to X.** Experiment, same participants on both, counterbalanced, time to first pattern.
3. *Does anyone keep using it after the lecture?* **Compatibility with user practices.** Deployment, usage logs, follow-up interview.
Each is the right study for *its* question.
nanojam’s pads interface.
# Types of Evaluation
## Controlled settings (e.g., Usability testing)
Image Source: Usability Testing (interactiondesign.org)
A controlled evaluation setting is not the normal place for using a technology or for the user to be.
## Usability Testing
- **Measures:** numbers or time (e.g., tasks completed, errors made, time taken)
- **Techniques:** combined from week 4: think-aloud during set tasks, a questionnaire afterwards (SUS, NASA-TLX), then a short interview
- **Data:** variety of data depending on the techniques (e.g., video, audio, facial expressions, key presses, verbal feedback)
- **Settings:** lab + observation room, mobile usability kit, university classroom
- **Number of participants:** 5-12 baseline but more is better
- **What is new this week:** the *tasks* are set by you, and the *setting* is controlled by you
A controlled setting at ANU for testing music apps. Interaction data, timing, audio, video, surveys and interviews were recorded (Martin et al., 2016)
## Usability Testing Example
Schaadhardt et al. (2021)Understanding Blind Screen-Reader Users’ Experiences of Digital Artboards
## Natural settings (e.g., Field studies)
Evaluating a technology or context of use in the normal setting for the user.
Field studies can:
- Help identify opportunities for new technology
- Establish the requirements for a new design
- Facilitate the introduction of technology or inform deployment of existing technology in new contexts
Helps to establish [ecological validity](https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction/ecological-validity).
## Field Studies
- Goals:
- Understanding how people interact with technologies in “messy worlds”, how technologies will be integrated into contexts
- Studying use of existing technologies and impacts of introducing new ones
- Techniques: mostly qualitative (observation, interviews, diaries, interaction logging) rather than set tasks and timings
- Duration: No fixed length- can be seconds, months, years
- Paying attention to: Use situations, problems/errors, distractions, patterns of behaviours
- How does your presence and involvement shape engagement? Observation vs participant observation
- Findings: thematic analysis, vignettes, narratives, critical incident analysis (weeks 5 and 6)
## Field Studies Example
Co-Designing with Orangutans: Enhancing the Design of Enrichment for Animals (Sarah Webber, Marcus Carter, Wally Smith, and Frank Vetere) Proc. DIS ’20 (Webber et al., 2020)
Design objective 1: Develop a digital installation to provide enhanced, varied enrichment for orangutans at Melbourne Zoo
## Opportunistic Evaluations
- quick feedback about a design idea in the early design process
- confirm whether it’s worth developing an idea into a prototype
- informal and doesn’t require lots of time or resources
- not a replacement for formal evaluation
- **care required** with ethics in research (Hons, Master, PhD). Asking supervisors and colleagues for advice vs collecting data to establish findings.
**E.g.,** designers ask colleagues for design feedback: Yichen Wang’s *arMIDI* system early design process with supervisor and colleagues ([Wang et al., 2025](#ref-wang2025seeingsound)).
Development process for arMIDI
## Which methods to choose from?
The evaluation setting guides certain dimensions of developed artefacts.
- **Combinations of methods** are used for a richer understanding. E.g., usability testing is combined with observations to identify usability problems and how users use the system.
### Pros and Cons
- **Controlled settings** allow hypotheses testing on specific features for generalised results.
- **Uncontrolled settings** offer unexpected insights into perception and experience of new technologies in daily life and work.
**Talk:** 🗣️🎤⭐️ You’re all HCI researchers and we need to evaluate this interactive toy.
We need to choose:
- how we will evaluate the toy?
- in what environment?
- what information do we need and why?
- what *research questions* are being asked?
Talk for 2-3 minutes and then we will hear some answers.
Where and why will we evaluate this toy? (Photo by COSMOH LOVE on Unsplash)
# Planning Evaluations
What do we need to keep in mind to plan evaluations?
## Design and Conduct Issues
- **Reliability:** how well it produces the same results on separate occasions under the same circumstances
- **Validity:** whether the evaluation method measures what it intended to measure
- **Ecological validity:** how the environment in which an evaluation is conducted influences or distorts results
- **Bias:** occurs when the results are distorted
- **Scope:** how much of the findings can be generalised
## Ethical Issues
- **Risks:** what are the risks to participants? (e.g., physical harm, reputational risk, distressing conversations, being identified etc)
- …and how are risks mitigated…
- **Benefits:** what are the benefits to participants? (e.g., none, helping research, fun experience, getting paid, course credit, etc)
- **Consent:** how is informed consent established? (e.g., a participant information sheet and a written form)
- **Data:** how is data stored and who has access to it? what will happen to it over time?
Universities have [processes](https://services.anu.edu.au/research-support/research-ethics-integrity-compliance/human-ethics) to *approve* the ethical aspects of research that collects data from humans following established rules ([National Health and Medical Research Council (NHMRC) et al., 2025](#ref-national-statement-ethical-conduct:2025)).
We don’t go deeply into research ethics in this course but the four issues above are the core ones.
## Developing an evaluation plan
- Evaluation Goal/Aims
- Participants
- Setting
- Data to collect
- Methods
- Ethical Considerations and Consent
- Data capture, recording, storage
- Analysis method
- Output(s) of evaluation process
How to evaluate this app?
## Prepare what you need before the first participant arrives
The plan is a document. This is the checklist for the room (or the Zoom call).
- a place for the participant and for you: tables, chairs, power
- the prototype, working, in the state the tasks assume
- printed instructions and task list, so everyone hears the same thing
- data capture ready and tested: video, audio, screen, logs
- consent form and information sheet, signed *before* recording
Then **pilot** it (week 4). Most study failures are “the laptop died” or “we didn’t press record”.
Yichen Wang’s human-AI musical collaboration research study setup at School of Music.
## Experimental Variables
- **Independent variable:** the condition the researcher controls.
- **Dependent variable:** the outcome we are measuring.
- **Independent vars in HCI:** different interfaces, input devices, software, colours, computer type
- **Dependent vars in HCI:** efficiency, accuracy, subjective satisfaction, ease of learning, physical/cognitive demands
### Variables shape your study
- **Tasks:** completing specific tasks, or freely using a technology?
- **Interfaces:** just using one interface, or comparing two (or more!) different styles
Feels like we doing chemistry or biology… (Photo by CHUTTERSNAP on Unsplash)
## From a research question to a hypothesis
A hypothesis is a **prediction**: it says what you will change, what you will measure, and which way the result will go.
> RQ: *Is the pads interface faster to a first pattern than the command line?*
>
> H1: *Participants reach a first repeating pattern faster with pads than with the command line.*
- The RQ allows more than one answer. The hypothesis picks one and dares the data to disagree.
- A hypothesis can only be dismissed, never proven: null and alternative guide the test.
- Dismissing one needs a controlled design and significance testing (week 10).
Not every RQ should become a hypothesis. nanojam questions 1 and 3 are better off without one.
## Experiment Design
Which participants test which conditions?
1. Different-participant design: each participant sees one condition.
2. Same-participant design: everybody sees each condition.
3. Matched-participant design: matched groups of participants with a shared trait put into each group
- **Balanced Ordering** is important to counter learning effects.
- **Design choices** affects validity and reliability
- **Data collection:** think back to week 4 lecture, but often includes task performance, completion time, errors, subjective satisfaction etc.
Focus is on the experiment not the design!! (Photo by Girl with red hat on Unsplash)
## Table of Experimental Designs
| Design | Advantages | Disadvantages |
|----|----|----|
| Different participants (between-participants design) | \- No order effects | \- Requires many participants - Individual differences can affect results - Random assignment helps minimize differences |
| Same participants (within-participants design) | \- Eliminates individual differences between conditions | \- Requires counterbalancing - Risk of order effects (e.g., learning or fatigue) |
| Matched participants (pair-wise design) | \- No order effects - Reduces impact of individual differences | \- Time-consuming to find matched pairs - May miss other influential variables |
## In-the-Wild Studies
- Natural setting, minimal control over participants
- reflecting real-world use unpredictable and complex
- Ethical and practical challenges are greater, e.g., participant consent, privacy, equipment issues, and environmental factors.
> Reveal insights about actual use and long-term integration that lab studies often miss.
## People are complicated
HCI is hard. To do a study, you usually need to:
- Plan the study
- Find participants
- Manage communication with them
- Figure out what to do if they don’t show up
- Managing a study requires some social skills! It’s hard work!
Is there any way to do evaluation *without* users?
# Evaluation by Inspection
Skip the “users”! Just evaluate against established principles (heuristics) and standards.
## Expert Evaluation
- Conducted by designers and design “experts” rather than with end users
- Inspection methods – expert role plays user
- **Heuristic evaluation:** Researchers evaluate whether aspects design adhere to established usability principles (see over)
- **Cognitive walkthroughs:** Simulating user reasoning and problem solving at each step in an interaction sequence (evidence, availability, accessibility of correct action)
Two more ways to evaluate with **nobody in the room**, covered after the case study:
- **Predictive models:** compute the user’s performance instead of measuring it (Fitts’ law)
- **A/B testing:** thousands of real users, none of whom know they are in a study. Not “without users” so much as without *recruiting* them.
## Nielsen’s 10 Usability Heuristics
1. **Visibility of system status:** keep the user informed
2. **Match between system and real world:** system uses language and communication familiar to the user, information is natural and logical
3. **User control and freedom:** users make mistakes, there should be “emergency exits” to cancel and return quickly
4. **Consistency and standards:** users should not wonder whether words, situations or actions mean the same thing, follow conventions
5. **Error prevention:** eliminate error-prone conditions, or check with user before they occur
6. **Recognition rather than recall:** make elements, actions, and options visible
7. **Flexibility and efficienty of use:** shortcuts to speed up for experts, allow tailored experiences
8. **Aesthetic and minimal design:** less is more, no unnecessary information
9. **Help users recognise, diagnose and recover from errors:** error messages need plain language, and suggest solutions
10. **Help and documentation:** best if explanation is not needed, if it is, make it good
## Activity: Inspect This Interface
An internal web app a state emergency agency uses to send public alerts. The officer picks a template, then confirms.
**Do:** You’re the expert evaluator. Log every violation of Nielsen’s 10 you can see, numbered by heuristic. 3 minutes, then we compare lists.
Decide what’s wrong *before* you find out what happened.
- 8:07am, mid-drill: the officer took the drill call for a real one and sent the live template. Same prompt for both, no preview.
- “THIS IS NOT A DRILL” reached 1.4 million phones. No correction template existed; the correction took **38 minutes** ([Public Safety and Homeland Security Bureau, 2018](#ref-fcc-hawaii-false-alert:2018)).
**The test that would have caught it:** the one you just ran. Heuristics 5 and 9, one evaluator, one hour.
The fix, from Appendix B of the FCC report: drill relabelled, a “False Alarm” template added.
## Cheapest first, never cheapest only
Inspection has a high false-negative rate, and different evaluators find markedly different problems. It “should not be trusted for complex or safety-critical systems” ([Hornbæk et al., 2025, Section 40.3](#ref-hornbaek-intro-hci:2025)).
Hawaii’s alert system *was* safety-critical. The heuristic pass finds the problems on the face of the screen; it does not find the ones that only appear when a tired officer is mid-drill at 8am.
In Hornbæk’s full yardstick table, “comply with guidelines” maps to one method. “Safety” is the row marked *multiple*.
## Shneiderman’s Eight Golden Rules of Design
1. Strive for consistency
2. Seek universal usability
3. Offer informative feedback
4. Design dialogs to yield closure
5. Prevent errors
6. Permit easy reversal of actions
7. Keep users in control
8. Reduce short-term memory load
## Activity: Two ways to evaluate nanojam
**Do (3 minutes):** open on your own device. You are the expert evaluator. Log every violation of Nielsen’s 10 you can find in the command line interface, numbered by heuristic.
**Watch (3 minutes):** one volunteer, projected, thinks aloud (week 4) while trying to make a repeating pattern on the **pads** interface, no help from me.
Then compare: what did the inspection find that the user never hit? What did the user hit that no heuristic covers?
Two studies, one app, ten minutes.
**Think:** which of the three research questions from earlier does each of these studies actually answer?
## A/B Testing
- An online controlled experiment: users randomly get A (control) or B (new design), usually without knowing. Between-participants, at web scale; week 10 has the statistics.
- **A/A test first:** same page to both groups, so any difference is a setup bug. Office Online’s failed because only the treatment was redirected; the delay alone cost page views ([Kohavi et al., 2009](#ref-kohavi-online-experimentation-microsoft:2009)).
- **Or don’t test, and find out later:** Office Online swapped a yes/no feedback question for five stars, lost 80% of responses, and took eight months to notice.
Bing search ad: control (top) and treatment with a second line of ad text moved into the title (bottom) (Kohavi et al., 2020). More in week 10.
## Predictive Models
Estimate user performance without needing real users, using formulas to assess task efficiency — useful in early design stages or when testing with users is difficult.
Fitts’ Law ([Fitts, 1954](#ref-fitts1954information)):
- predicts how long it takes to point at a target based on its size and distance
- helps designers optimize button placement, size, and spacing on screens and devices.
- applications: input methods (e.g., touch, gaze, tilt), mobile and VR, simulating interactions for users with motor impairments
## Evaluation after deployment: adoption, use, and non-use
The last yardstick in the table, **compatibility with user practices**, can only be measured after the thing is out in the world. A whole literature asks what “success” means then:
- Adoption/Appropriation/Design-in-use ([Ehn, 2008](#ref-ehn-participation:2008))
- Technology acceptance ([Davis, 1989](#ref-davis1989perceived))
- Non-use ([Satchell & Dourish, 2009](#ref-satchell-beyond-user:2009))
- Technology habitation ([Soro et al., 2016](#ref-soro-older-users:2016))
- Technology individuation ([Ambe et al., 2017](#ref-ambe-individuation:2017))
(Ambe et al. 2017)
# Questions: Who has a question?
**Who has a question?**
- I can take *cathchbox* question up until 2:55
- For after class questions: meet me outside the classroom at the bar (for 30 minutes)
- Feel free to ask about **any aspect of the course**
- Also feel free to ask about **any aspect of computing at ANU**! I may not be able to help, but I can listen.
Meet you at the bar for questions. 🍸🥤🫖☕️ Unfortunately no drinks served! 🙃
# References
Ambe, A. H., Brereton, M., Soro, A., & Roe, P. (2017). Technology individuation: The foibles of augmented everyday objects. *Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems*, 6632–6644.
Benford, S., Greenhalgh, C., Crabtree, A., Flintham, M., Walker, B., Marshall, J., Koleva, B., Rennick Egglestone, S., Giannachi, G., Adams, M., Tandavanitj, N., & Row Farr, J. (2013). Performance-led research in the wild. *ACM Trans. Comput.-Hum. Interact.*, *20*(3).
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. *MIS Quarterly*, *13*(3), 319–340.
Ehn, P. (2008). Participation in design things. *Proceedings of the Tenth Anniversary Conference on Participatory Design 2008*, 92–101.
Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. *Journal of Experimental Psychology*, *47*(6), 381.
Hornbæk, K., Kristensson, P. O., & Oulasvirta, A. (2025). *Introduction to human-computer interaction*. Oxford University Press.
Kohavi, R., Crook, T., Longbotham, R., Frasca, B., Henne, R., Ferres, J. L., & Melamed, T. (2009). Online experimentation at Microsoft. *Third Workshop on Data Mining Case Studies and Practice Prize, KDD 2009*.
Kohavi, R., Tang, D., & Xu, Y. (2020). *Trustworthy online controlled experiments: A practical guide to A/B testing*. Cambridge University Press.
Martin, C., Gardner, H., & Swift, B. (2014). Exploring percussive gesture on iPads with ensemble metatone. *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems*, 1025–1028.
Martin, C., Gardner, H., Swift, B., & Martin, M. (2016). Intelligent agents and networked buttons improve free-improvised ensemble music-making on touch-screens. *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, 2295–2306.
National Health and Medical Research Council (NHMRC), Australian Research Council (ARC), & Universities Australia. (2025). *The national statement on ethical conduct in human research (2025)*.
Public Safety and Homeland Security Bureau. (2018). *Report and recommendations: Hawaii emergency management agency January 13, 2018 false alert*. Federal Communications Commission.
Rogers, Y., Sharp, H., & Preece, J. (2023). *Interaction design: Beyond human-computer interaction, 6th edition*. John Wiley & Sons, Inc.
Satchell, C., & Dourish, P. (2009). Beyond the user: Use and non-use in HCI. *Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7*, 9–16.
Schaadhardt, A., Hiniker, A., & Wobbrock, J. O. (2021). Understanding blind screen-reader users’ experiences of digital artboards. *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems*.
Soro, A., Brereton, M., & Roe, P. (2016). Towards an analysis framework of technology habituation by older users. *Proceedings of the 2016 ACM Conference on Designing Interactive Systems*, 1021–1033.
Wang, Y., Xi, M., Adcock, M., & Patrick Martin, C. (2025). Seeing the sound: Supporting musical collaboration with augmented reality. *Proceedings of the 2025 Conference on Creativity and Cognition*, 99–112.
Webber, S., Carter, M., Smith, W., & Vetere, F. (2020). Co-designing with orangutans: Enhancing the design of enrichment for animals. *Proceedings of the 2020 ACM Designing Interactive Systems Conference*, 1713–1725.
