AI Needs Finding

Dr Charles Martin

Understanding how your classmates really use AI. (Photo by UX Indonesia on Unsplash)

Outline

Due date: 2026-10-06 23:59 AEDT
Mark weighting: 30%
Submission: submit your assignment according to the instructions below
Policies: Late submission is not permitted for this assessment. This is an individual assessment task.
Gitlab Template Repository: https://gitlab.cecs.anu.edu.au/comp3900/2026/comp3900-2026-A2-needs-finding

Description

Assignment 2 involves conducting a small-scale needs finding study focused on your classmates’ use of AI tools in their creative and learning work. The research challenge is as follows:

AI for Learning and Life: TEQSA — Australia’s higher education regulator — just got in touch. They’re drafting new guidelines on AI in tertiary education and need real data about what students actually need from AI tools, not just what they’re worried about. As a contracted HCI researcher, your job is to run a small needs finding study with your fellow students. The research question you will investigate is:

“What are the unmet needs, frustrations, and desired capabilities that students experience when using AI tools in a specific context of their life?”

Students use AI for studying, working, coding, creating, socialising, healing, and managing all sorts of things in their lives. You need to choose one specific context to investigate. To address the research question you will collect and analyse data and present your findings as a needs analysis. Your study should involve at least one form of quantitative and one form of qualitative data and analysis.

Your study documentation will include research plan, analysis, results, and conclusions under the following headings:

Context, Research Plan, and Conclusions: describe the AI use context you are investigating and why it is worth studying, your study plan including data gathering strategies, analysis framework(s), and justification from HCI academic research, and summarise the overall conclusions — including a set of clearly articulated user needs derived from your data.
Quantitative Results: present analysis of your quantitative data and your findings from this data, include excerpts from data, plots, tables, or other presentation approaches to help articulate your findings.
Qualitative Results: present analysis of your qualitative data and your findings from this data, include excerpts from data, plots, tables, or other presentation approaches to help articulate your findings.

You should include all collected data from your study in the materials folder of the GitLab template, and you may refer to it from your main document.

To accomplish this task, you will need a strong understanding of data gathering, quantitative analysis, and qualitative analysis as discussed in weeks 5–8 of the course. Your submission must demonstrate sophisticated engagement with these concepts.

Specification

Your submission must:

clearly describe the AI use context you are studying and justify its relevance to the research question
include a clear study plan with justification from HCI academic research
include a clear summary of the overall conclusions derived from your data and analysis, expressed as a set of user needs
include data from 3–5 participants: all participants must be students in COMP3900/6390! (list them in the acknowledgements section)
include results based on quantitative and qualitative data and analysis techniques
include at least five references to external sources in ACM format
be written in markdown format with correct syntax and formatting
be a maximum of 1800 words + 10% acceptable buffer (i.e., no more than 1980 words)
must be uploaded to your fork of the comp3900-2026-A2-needs-finding repository on Gitlab by the due date

N.B. to eliminate doubt: all participants in your study must be current students in COMP3900/6390. The best way to find and study your classmates is to attend all lectures and tutorials and participate in assignment planning activities.

Submission Process

Fork the template repository.
Follow appropriate data gathering and analysis processes to address the research challenge.
Upload research data and analysis scripts as well as other relevant materials in your fork of the template repository.
Write your study documentation following the template in the repository.
Don’t forget to include at least five references to external sources in ACM format in your documentation.

Notes

Here’s how to get started with the work in this assignment:

Choose a context to study. Pick one specific situation in your classmates’ lives where they use AI tools — for example: writing code with an AI assistant, using AI to revise written work, using AI to understand a difficult concept, using AI to manage stress or plan their week, or using AI in a creative project. You will need to explain why this context is interesting and how it connects to the research question.
Create a study plan. Use your research skills to find examples of needs finding studies in HCI (e.g., contextual inquiry, diary studies, semi-structured interviews at CHI or CSCW). Use these to help plan your study. A needs finding study focuses on understanding what people do, what they struggle with, and what they wish they could do — rather than on testing a specific system. Make sure your study plan is realistic.
Find participants – they must be current COMP3900/6390 students. Attend all classes and find 3–5 people in your tutorial who will participate in each others’ studies. Don’t leave this until the last minute!
Collect data with your participants. Consider using a short survey for quantitative data (e.g., frequency of AI tool use, types of tasks, perceived helpfulness ratings) and a semi-structured interview or a short diary task for qualitative data (e.g., a description of a recent experience using AI, open-ended reflections on what worked or what was missing). You will need to participate in other people’s studies as well as ask them to participate in yours. (Don’t make up the data.)
Analyse your data. Use the quantitative and qualitative analysis techniques covered in classes. For qualitative data, reflexive thematic analysis is particularly well-suited to identifying needs and patterns across participants.
Articulate your findings as user needs. Your overall conclusions should go beyond summarising what participants said — synthesise the data into a set of clearly stated user needs that could inform future design. For example: “Students need a way to check whether AI-generated explanations are accurate without leaving their learning context.”
Write up your project in the correct format in your fork of the GitLab repository. Make sure you are using correct markdown syntax and have included all data and analysis files in your repository.
Acknowledge your research participants by listing them in your acknowledgements section.

Here’s some general advice:

Don’t forget to ask for help or advice on the course forum: Edstem discussions
Generative AI is not banned in this course, but submitted assignments are expected to be primarily your own work. Any use of generative AI must be explained in your Acknowledgements section including the scope to which it is applied. Use of generative AI without appropriate explanation and referencing comes under the N category in the marking rubric.
You may use generative AI to help with aspects of data analysis (see lectures!) but you are responsible for the output and whether findings are justified or not; we may re-run analyses to check your data.

Marking Rubric

CRITERIA	HD	D	CR	P	N
Sophistication and clarity of the study plan and articulation of user needs. (10 marks)	Excellent to outstanding study plan and articulation of user needs demonstrating consideration of HCI needs-finding methods that goes beyond learning materials. Needs statements demonstrate clear synthesis and evaluative judgment. Level of communication and referencing is excellent.	Very good study plan and articulation of user needs applying HCI needs-finding methods that follows the specification, but not beyond learning materials. Needs statements demonstrate evaluative judgment. Level of communication and referencing is excellent.	A study plan and needs findings following HCI research methods at the level of learning materials. Needs statements are present but synthesis and judgment may be limited. Level of communication and referencing is good.	Some effort to follow the specification for a study plan and derive user needs. May touch on HCI research methods. Needs statements may be weak or merely restate participant quotes. Level of communication and referencing may have some errors.	A study plan and conclusions with little connection to HCI needs-finding methods or one that is below acceptable standards. May not follow the specification or contain serious errors in communication and referencing.
Sophistication of quantitative data collection, analysis and articulation of findings. (10 marks)	Excellent to outstanding quantitative data collection, analysis and articulation of findings demonstrating in-depth understanding of HCI approaches. Excellent adherence to the assessment format.	Very good quantitative data collection, analysis and articulation of findings demonstrating in-depth understanding of HCI approaches. Excellent adherence to the assessment format.	Good quantitative data collection, analysis and articulation of findings that may not show sophisticated understanding of HCI approaches. Good adherence to the assessment format.	Satisfactory quantitative data collection, analysis and articulation of findings that may have limited incorporation of HCI approaches. The adherence to the submission format may be poor. Data analysis may have some errors.	Quantitative data collection, analysis and articulation of findings that is below acceptable standards. May have very poor adherence to submission format. Data collection and analysis may contain serious errors.
Sophistication of qualitative data collection, analysis and articulation of findings. (10 marks)	Excellent to outstanding qualitative data collection, analysis and articulation of findings demonstrating in-depth understanding of HCI approaches. Excellent adherence to the assessment format.	Very good qualitative data collection, analysis and articulation of findings demonstrating in-depth understanding of HCI approaches. Excellent adherence to the assessment format.	Good qualitative data collection, analysis and articulation of findings that may not show sophisticated understanding of HCI approaches. Good adherence to the assessment format.	Satisfactory qualitative data collection, analysis and articulation of findings that may have limited incorporation of HCI approaches. The adherence to the submission format may be poor. Data analysis may have some errors.	Qualitative data collection, analysis and articulation of findings that is below acceptable standards. May have very poor adherence to submission format. Data collection and analysis may contain serious errors.

Note that serious errors in referencing, communication, data collection and analysis are aligned with the N category in the assessment rubric. Serious errors may include: references to non-existent sources, research findings or conclusions that rely on non-existent data or analyses, fabricated data, fabricated results from analysis processes.