Rememo

An assistive tool for caregivers conducting dementia reminiscence therapy, using AI-image generation as a medium to bridge the gap between personal photos and archive images
About the Project
For people with dementia, reminiscence therapy with personal or archive photos is either too specific or general. Rememo bridges the gap using AI images, created through a card-based storytelling activity, giving the senior autonomy over the session while improving efficiency for carers. 
Conceptual Dissertation
Design Research Paper
NUS CDE Innovation & Research Award
Year
2023 - present
Platform
Figma, Midjourney, OpenAI, Stable Diffusion, Python, Arduino, Raspberry Pi
Timeframe
Ongoing
Team
Celeste Seah
Dr Yen Ching Chuian, Thesis supervisor
What I did
Research plan
Observations & interviews
Stakeholder management
Lo-fi prototyping
Service design
Hi-fi prototyping
User testing
Results
Experimented with prompt engineering to generate photorealistic AI images of memories
Conducted design research at two care centres
Designed and built a functional prototype
User-tested prototypes with 6 participants

Approach

Initial motivation
The journey into using AI-generated images to enhance dementia care began during a group project in the Netherlands. Intrigued by the potential of art therapies and AI, our exploration was validated through interviews revealing that personal photos often failed to evoke memories in dementia patients, instead causing distress.
Local context
Upon returning to Singapore, I delved deeper into the local care context through discussions with professionals and direct engagement in daycare centres.

User research

User interviews
To get a better understanding of the context surrounding potential users, I sought out 2 individuals with cognitive impairments and interviewed them in the presence of their caregivers. The responses were organised into an affinity map.
Care systems
I also conducted observational studies at 2 nursing homes and sat in on one reminiscence session involving 1 facilitator and 5 participants in varying stages of cognitive decline. I summarised the findings in a Service Blueprint.
From my conversations with the staff as well as observations, it was noted that manpower was a pressing issue for them as each client requires a lot of attention to engage and care for their needs, including moving them around for activities or even helping them to the bathroom.
  • Occupational therapists spend considerable time compiling materials for reminiscence therapy.
  • Generic materials often fail to engage seniors meaningfully.
  • Caregivers typically remain uninformed about their loved one’s daily activities unless noted by staff
  • Understanding each senior's unique history relies heavily on the tacit knowledge of therapists.
Because we think reminiscence stimuli needs to be specific yet general to an individual’s experiences, we think that using AI-assistive tools in RT will allow seniors to have deeper conversations independently with others. We’ll know this is true or false when we see people with cognitive impairments reminiscing together with little staff facilitation and more self-direction.

Prompt engineering

Prompt formula
The development of a reliable prompt formula was central to this project. This formula, designed to produce evocative memory-like photographs, comprises four key elements: 'who', 'what', 'where', and 'when'. The effectiveness of this formula lies in its ability to co-create images that resonate deeply with individual experiences, thus extending the therapeutic process.
Composition
Exploration into different types of image composition was done to determine the most effective approach in triggering memories and promoting dialogue among dementia patients. Two primary styles were examined:
Contextual Images
These images set a broad scene but do not include a specific subject. Feedback indicated that while these images helped set a tone, they often lacked a focal point for patients to engage with, potentially complicating the cognitive processing for those with diminished capacities.
Subject-Centric Images
Images focusing on a clear subject engaged in an activity were found to be more effective. They allowed patients to relate the depicted actions to their own experiences. However, the inclusion of human faces could sometimes disrupt the immersion, making it difficult for some patients to connect the images with their memories.
Style
Stylistically, several art styles can be tested and would primarily serve the purpose of preventing clients from confusing the photorealistic images with their own memories. However, since the point of the process is to immerse them in context to trigger memories, photorealism would appear to be more productive for the goal.

Lo-fi prototyping

Ideation
Idea 1: Small group interactive simulated presence therapy with live AI images to facilitate reminiscence
  • Depth & detail of reminiscence
  • Quality of conversation with other seniors in the group
  • Manpower and resources to deploy high-quality RT for a large group at one time
Idea 2: Workshop for seniors to co-create personalised memory books using AI to visualise their memories
  • Depth & detail of reminiscence
  • Quality of conversation with other seniors in the group
  • Longevity of memory book outcome for at-home RT with family
A decision was made to develop Idea 2 further over Idea 1 for the following reasons.
  • Higher engagement levels – principle of reminiscence as being relationship-based
  • IKEA effect – Cognitive bias where people place disproportionately higher value on things they helped to create
  • Take home artefact extends the reminiscence for repeated use at home with caregivers & family beyond the daycare
User testing
I user tested a scrappy version of the service with a 74-year-old man with severe cognitive impairment. I used  thick chalkboards for the prompt cards, in hopes that the tactility of writing with chalk may trigger more memories.
  • A lot of prompting and explaining to the point of the facilitator(s) leading the conversation
  • Tactility of the thicker cards is helpful for user to hold
  • User is unable to write responses on the cards but was able to write his own name
  • User said he cannot see the printed photograph
  • User did not want to elaborate on the memory and instead talked more about his associated opinions

Design refinement

Input methods
Early testing with the lo-fi prototype revealed that handwriting as an input method posed significant challenges, including the technical difficulties of optical character recognition (OCR). Alternative methods such as voice recognition were also considered but deemed unsuitable due to the diverse languages and dialects spoken by seniors in Singapore, often characterized by non-standard enunciation. Consequently, a card-based input method was chosen for its simplicity and versatility. Cards can support multiple languages and include illustrations, making the system accessible even for illiterate seniors.
Form Analogy and User-Centered Design
User-centredness is paramount when designing for elderly users with cognitive impairments. As such, to cater to this demographic, the decision was made to focus on analog interactions over digital interfaces, citing challenges the elderly face when interacting with digital screens. By eliminating digital interactions and the refinement process typical of AI-image generators, the user interaction is more accessible and less intimidating. 
Conceptual metaphors were also considered to allude to devices familiar to the elderly such as television sets and cameras, making it more approachable by reducing the learning curve by tapping on existing mental models and affordances.
In refining the service delivery mechanism, attention was given to optimizing the user experience, especially in group therapy contexts. Initial considerations of using a Polaroid Camera metaphor were set aside due to practical concerns over fine motor skills and memory recall challenges associated with inserting and remembering cards. 
Renders of Polaroid Camera concept
Render of Card Rack concept
Instead, a shift towards a game-like interaction model was made, prioritizing ease of use and engagement over conceptual metaphorical significance. This model involves laying out cards on a table for selection, thereby promoting a more communal and interactive session.

Service

The proposed service blueprint for reminiscence therapy introduces significant enhancements in therapy delivery efficiency and caregiver engagement.

Technical implementation

The system is built around the Raspberry Pi 4, chosen for its processing capabilities to interface with a cloud-based AI-image generator through an API. It controls the RFID readers that read the prompt cards, each embedded with an RFID tag. When all four categories in the prompt formula are satisfied with the appropriate cards, image generation starts. The generated image is then sent to a compact photo printer (Instax Wide Link) over Bluetooth to be printed.
Several leading AI image generation engines were evaluated, including Midjourney, DALL-E, and Stable Diffusion, for their effectiveness in text-to-image generation. Stable Diffusion was chosen for its ease of use through an API, good aesthetic appeal and photorealism.

User testing

Structured user testing sessions were conducted with 6 residents at ECON Care Residence (Henderson). Participants were selected by severity of cognitive impairment as the intervention is intended for seniors with milder cases. For ease of data collection and transcribing of the session, only English-speaking participants were considered. The three sessions, two Individual and two Groups of 2, were facilitated by Occupational Therapists who were briefed about the system and how to use it. 
During user testing, quantitative and qualitative data were gathered on the system's effectiveness in enhancing reminiscence therapy. This included measuring users’ understanding of the system, engagement levels (in comparison to their baseline levels), depth of reminiscence and change in mood. Additionally, therapists’ insights on the system's usability and its impact on their workflow are critical in evaluating the project for refinement.

Further developments

Business viability
Rememo offers a unique solution to enhance dementia care efficiency and quality. It serves a dual market:
  • B2B model: Targeting care centres and nursing homes with comprehensive training for therapists and an ongoing supply of themed card packs.
  • B2C adaptation: For tech-savvy seniors at home, a digital app replicates the Rememo experience, suitable for personal use or integrated into home-care services provided by existing care centres.
Continued work
Further work is ongoing to develop the project in the following areas. The project is open to collaboration with relevant organisations who may be interested. 
  • Expanding the range of themes and resources available as requested by the therapists
  • Fine-tuning and training the AI engine on local contexts such as traditional games and local foods. 
  • Longitudinal testing on how the images can continue to be used even as a person’s condition progresses

You may also like these projects

Kagenda: What if meetings could be smarter?

As a workplace consultant, hybrid setup has become an increasing requirement post pandemic. How will you create productive and meaningful solutions for new tenants? How will you trigger sustainable behaviour throughout the tenant's journey?
UX research
UI design
Coursework
Read more

i12 Katong Shoppers' App

i12 Katong Mall will reopen in Q4 2021 after closing for major asset enhancement works in early 2020. The mall operator, Keppel Land, seeks to transform the mall into a future-ready retail hub that will shape the increasingly phygital world while tapping into the rich Peranakan heritage of the district.
UX research
UI design
Internship
Read more

Supernova Ignite: Branding the first post-COVID campus concert

NUS Supernova: Ignite 2022 (12 - 13 Aug 2022) was a two-day music festival featuring student performers, held on UTown Green and attended by over 5000 participants. It marked the start of the first post-COVID semester at NUS with the return of student life.
Branding
Project management
Graphic design
Freelance
Read more
LinkedInInstagram
Designed and built by Celeste Seah
Copyright © 2024