Co-designing a framework for evaluation of assistive technology: learning from people with acquired brain injury and the assistive technology advisors they engage
Libby Callaway
A B * , Grahame Simpson C D E , Reem Rendell E F G H , Josh Taylor I and Em Bould A B
A
B
C
D
E
F
G
H
I
Abstract
Assistive technology can change the way executive function support is provided to people with acquired brain injury (ABI). However, tools to guide consideration and selection of technology for cognitive support are lacking. The aim of this study was to co-design and co-produce a tool that can be used by assistive technology advisors collaborating with people with ABI to evaluate assistive products used for cognitive support.
McKercher’s six-phase co-design process was used. A small circle co-design group was purposively constructed including people with ABI lived experience, practicing health professional and researchers, and injury and disability insurers. The small circle co-design work was coupled with large circle stakeholder engagement across three online health professional workshops and one in-person workshop.
A Framework for the Evaluation of Assistive Technology was drafted, tested and refined using detailed co-design and co-production processes across six phases. Workshop feedback was iteratively incorporated to finalise the framework. The Framework for the Evaluation of Assistive Technology is now available for use via a free website, called My Technology Space.
Co-design of the Framework for the Evaluation of Assistive Technology across multiple phases of work enhanced its utility and drew on perspectives of people with ABI who use assistive technology, as well as health professionals and funders of assistive products and services.
Keywords: assessment, assistive technology, co-creation, co-design, co-production, cognition, community, health professionals.
Introduction
Acquired brain injury (ABI) can cause enduring impairments of executive functioning, including difficulties with planning, organising, initiating, maintaining attention within or remembering activities (Jeffay et al. 2023). Difficulties in executive functioning, known as executive dysfunction, have been identified to exert a profound effect on resuming pre-injury activities of daily living and life roles following ABI (Tate et al. 2014; Jamieson et al. 2020). Effective management of executive dysfunction can increase the participation and well-being of individuals with brain injury within everyday life; however, research has demonstrated that health professionals may find it challenging to address these changes in executive functioning (Pilli et al. 2023).
Assistive technology is an umbrella term that includes both assistive products and their related systems and services, and can help to maintain or improve functioning related to cognition, communication, hearing, mobility, self-care for vision and health, well-being, inclusion, and participation (World Health Organization 2024). There is growing recognition that emerging technologies can offer benefits, while also posing potential challenges, for people with disability (Bridge et al. 2021; Smith et al. 2023). Assistive products for cognition are devices and software designed to meet the specific needs of persons with executive dysfunction, and have been viewed as holding promise in supporting executive functioning following ABI (Leopold et al. 2015; Andelic et al. 2019). As an example of this, the advent of the smartphone over a decade ago has led to widespread adoption of these accessible and convenient devices among people with ABI (Chuah et al. 2024), and acceptability of their use as a memory and organisational aid (Wong et al. 2017). More broadly, assistive technology has been found to be effective in improving activities of daily living of people with ABI, by helping to compensate for cognitive impairments (Leopold et al. 2015; Nam and Kim 2018).
Beyond the increasing use of smart devices for cognitive support, advances in technology have led to the development of new cognitive and social communication treatment approaches, including via both mainstream and emerging technologies (e.g. smart home, wearable and mobile technologies; Togher et al. 2023). These advances require health professionals to develop new competencies to work within the digital and virtual health space, and advance knowledge and performance competencies in this field (Brunner et al. 2018). Given the burgeoning growth of smart home and mobile assistive technology available on the market, the challenge for health professionals (including occupational therapists, speech pathologists and neuropsychologists, as the main assistive technology advisory workforce) is to know what works for who, and in what context, and to fully understand each product’s features (Bridge et al. 2021).
Two classes of assistive technology that are recognised in the Australian New Zealand Standard for assistive products – and can be used as interventions to address cognitive functioning within everyday activities following ABI – are assistive products for measuring, stimulating or training physiological and psychological functions (Class 04), and assistive products for communication and information management (Class 22; International Standards Organization 2023). Class 04 includes assistive products for time, memory and planning support, such as electronic clocks, calendars and reminder systems (e.g. for medications). Class 22 includes assistive technology for signalling, alarming and localisation, such as wearables enabling a nominated support person to be called for assistance; and products for locating or tracking a person. Products that may be used for executive function support in this class include personal GPS trackers enabling real-time location; and computer and mobile devices with adaptive features to support cognitive changes, including computers, smart phones, tablets, and associated software and apps.
Although assistive technology used for cognitive support can be classified using this internationally recognised terminology, a problem exists for the funders of assistive products and service for people with ABI, as well as the individual themselves and the health professional workforce advising them. This problem relates to each of these stakeholder’s knowledge and skills regarding what can be achieved via the use of assistive technology for cognitive support, and which of the large number of products available within each class are safe, sustainable and effective for use (Bridge et al. 2021). Evidencing this issue, in a survey of 123 health professionals working in the field of ABI rehabilitation in NSW, researchers found that although 92% of respondents felt they had ‘adequate’ or ‘expert’ knowledge about mainstream technology for their personal use, 65% reported having ‘no’ or ‘minimal’ knowledge about the latest emerging assistive technology; 59% ‘no’ or ‘minimal’ knowledge about evidence for benefits, harms or cost-effectiveness of assistive technology for their clients; and 49% ‘no’ or ‘minimal’ knowledge about essential factors to consider when selecting assistive technology for clients. Reflecting this knowledge gap further, less than half (44%) rated themselves as ‘fairly’ or ‘very confident’ in gathering evidence to determine the suitability of assistive technology for their clients (Pilli et al. 2023). Given the evidence gaps and practice challenges that exist, the current study was designed to draw on co-design and co-production processes across multiple phases of work to address the knowledge and skills problem identified – that is, how to evaluate, compare, trial and select assistive products used for cognitive support with people with ABI.
A co-design approach to participatory action research includes active collaboration between stakeholders in the design of solutions to a pre-specified problem (McKercher 2020). Co-design proposes an approach to designing with – not for – people (Vargas et al. 2022), can be an outcome-generating methodology (Benz et al. 2024), and is viewed as an important part of good assistive technology research and practice (Layton et al. 2022). Co-production includes working in partnership, and sharing power between people by implementing previously determined solutions to a previously agreed problem with emphasis on the most efficient use of existing resources and assets (Vargas et al. 2022; Social Care Institute for Excellence n.d.). Although the actual finalised solution to be designed was not known at the start of this research, the study aim was to co-design and co-produce a tool that could be used by health professionals and other assistive technology advisors in collaboration with people with ABI and their other key supporters to choose and evaluate assistive products used for cognitive support within everyday activities following brain injury. This paper describes the six-stage co-design process used to co-design and co-produce the tool.
Materials and methods
A small circle co-design group was purposively constructed to include people with ABI lived experience, practicing health professionals and researchers, and injury and disability insurers. This small circle co-design group met for the duration of the project, which used McKercher’s (2020) six phase co-design process – see Fig. 1.
Co-design activities were co-constructed and implemented across each phase (and are outlined in each of the phases described below), with the small circle co-design research team and large circle stakeholder engagement to offer a ‘multi-dimensional’ and ‘systems’ perspective (McKercher 2020, p. 58). Phases 1–4 guided the co-design methods, and Phases 5 and 6 led to the co-produced results. Activities completed in Phases 1–4 are detailed below, with findings from work undertaken in Phases 5 and 6 reported in the results section.
Phases 1–4, guiding the co-design methods
To build the conditions, the small circle co-design team was purposefully constructed with the two-person support team (LC, GS) ‘guiding and looking after’ people taking part in co-design throughout the duration of the project (McKercher 2020, p. 76). Examples of support offered by this two-person team included preparation of a plain text agenda, and a simple slide pack using images and key words to guide each small circle discussion; the offer to meet with individuals from the co-design team individually either before or after a small circle co-design discussion; and provision of verbal explanation of written information presented in group or individual meetings. Follow-up check ins via telephone or email were also undertaken with the co-design team members with ABI after a specific session, in case further thoughts, questions or suggestions arose.
People with lived experience of ABI were invited into the small circle co-design team by the research team sending details of the study to community-based health professionals working in the field of ABI. These health professionals were invited to forward details of this co-design project to people aged >18 years with ABI who met the following criteria: (1) they were able to provide informed consent for co-design participation; (2) the health professional determined that the individual would be interested in participating in co-design processes as a team member in receipt of renumeration for their time; and (3) the person was using technology for cognitive support. If the person wanted to hear more about the project, or consider being contracted as a paid co-designer within it, they were invited to contact the chief investigator (the first author). Three adults with ABI who met the study inclusion criteria responded, and the chief investigator met with each of them to discuss the role and time demands. All three agreed to join the co-design team following these meetings and provided informed consent. Health professionals were invited into the small circle co-design team by the research team sending details of the study to their professional networks. There was a focus on ensuring the professions of occupational therapy, speech pathology and neuropsychology were all represented (given they are the main assistive technology advisory workforce in the area of cognitive support and executive function). Seven health professionals responded to the email and provided informed consent.
The small circle co-design team included people with lived experience of ABI who use technology for cognitive support (n = 3, two men and one woman); health professionals working in the field of ABI who were involved in the design and implementation of assessments and interventions or researching in the area of assistive technology used for cognitive support (n = 7, one man, five women and one non-binary person); and representatives from state-based injury insurance and Commonwealth disability insurance agencies funding assistive products and services (n = 3, two men and one woman).
Parameters of engagement and payment of each of the people with lived experience of ABI, as well as health professionals working in the small co-design team, were established at the start of the project. Two of the people with lived experience of ABI were contracted to Monash University as casual research assistants. One person with ABI stated a preference to volunteer their input, with the option of being provided gift cards to acknowledge their time and expertise contributed to the project. The health professionals were paid per hour by them invoicing the University.
To further build the conditions for the subsequent phases of co-design and co-production work, key activities were mapped and finalised through a consensus process with the small circle team. These activities were designed to address the knowledge and skills problem identified to exist in relation to consideration and selection of technology for cognitive support. The subsequent phases of co-design work aimed to draw on the support team, small circle co-designers, and other people involved in co-design (i.e. the large circle stakeholders, being the project advisory group and co-design workshop participants) to co-produce a tool that could be used by health professionals and other assistive technology advisors in collaboration with people with ABI to consider, compare, select and evaluate assistive products used for cognitive support within everyday activities.
The second phase, ‘immerse and align’, was guided by a set of semi-structured questions (see McKercher 2020, p.147), with one-to-one meetings held with each of the co-designers with lived experience of ABI. These one-to-one meetings included one of the researchers from the support team (LC or GS), and were used to understand and document their individual perspectives on the use of technology for cognitive support following ABI; products used; how the person would assess, trial, select, fit or customise, use and evaluate the effectiveness of assistive products; and the workforce they drew on for advice and decision making.
Small group meetings were also held with each of the co-design representatives from state-based injury insurance and Commonwealth disability insurance agencies funding assistive products and services to understand and document their professional perspectives on the use of technology for cognitive support following ABI; how they make decisions regarding the investment in assistive product types; approaches to assessment, trial, selection, fitting or customisation, use or evaluation of the effectiveness of assistive products; and the workforce they contract for advice and decision making. Tacit knowledge of practicing health professionals and researchers in the small circle co-design group were also documented at this point (Malterud 2001).
As part of discovery work, an international environmental scan of assistive products that could be used for executive functioning support was undertaken, identifying 195 products. This scan was undertaken using a Google advanced search in English, and search terms were based on those specified in Liddle et al. (2021), which had been developed with a subject matter librarian. Based on review of product manuals, 143 of these were deemed to be able to be used in Australia and were subsequently mapped to 12 assistive product categories (for details, go to the My Technology Space website). A scoping review of literature published between 2009 and 2019 on electronic assistive technology used to support executive functioning was also undertaken, with a focus on extracting and reporting clinical implications that could be applied in the initial tool drafting (Liddle et al. 2021). Findings from this discovery phase of work were presented to, and discussed with, the small circle co-design team. This involved one-to-one meetings with the co-designers with lived experience of ABI, and group meetings with health professionals and representatives from state-based injury insurance and Commonwealth disability insurance agencies.
The design phase involved establishing a project advisory group, inclusive of health professionals working in the field of ABI; professionals involved in either the design and implementation of assessments and interventions, or undertaking research, in the area of assistive technology used for cognitive support; and representatives from disability and injury insurance agencies funding assistive products and services. The membership structure of this group was proposed by the research team at project inception, with advisory group members invited via the research team’s and project funder’s professional networks. The project advisory group met a total of eight times from 2021 to 2024 inclusive.
The three researchers with lived experience of ABI proposed individual meetings, hosted either online using Zoom or in person at a University library (based on each person’s stated preference). Six individual meetings were held, each for between 60 and 90 min duration. The first meeting involved hearing each person’s perspectives on barriers and enablers to the use of technology for cognitive support, and the types and categories of technology used. Subsequent meetings took a focus on a component of framework development (e.g. consideration of key questions to ask when exploring technology for cognitive support; views on how to categorise these questions; tools that could be useful to assist the consideration of technology for cognitive support; drafting of a glossary of terms; and digital and written instructions to guide use of a framework). Each meeting was recorded, transcribed verbatim and analysed by the first author using inductive content analysis (Vears and Gillam 2022).
The design of the first version of a tool drew on and incorporated findings from the environmental scan, literature review, small group co-design investigator meetings and consensus work, and project advisory group feedback. This first version was drafted for use by a person with ABI, their key informal supporter/s (e.g. family or friends) and/or assistive technology advisors inputting to product exploration. In that first version, nine domains (being effectiveness; design and functionality; reliability; value for money; technical specifications; sustainability; service delivery; privacy and security; and risk) were operational defined, and a set of 58 statements (linked to the nine domains) were drafted. Version 1 of the tool was named by the co-design research team as the Performance Enhancing Technology Framework and presented at an international assistive technology conference in September 2020 for peer testing (Layton and Callaway 2020). Version 1 of the framework was converted into an interactive PDF document, a Word document and a fillable Excel spreadsheet. The spreadsheet version was designed to convert statements into questions. For example, under the domain ‘effectiveness’, the statement, ‘What is the description of the assistive technology as per the manufacturer’s claims?’ was converted to ‘description of assistive technology (as per manufacturer’s claims) meets user’s specific needs’. For each question, there was a drop down selection box of ‘yes’, ‘no’, ‘not applicable’ or ‘don’t know’ and an open text box for comments available across each question in each of the nine domains. By the user making these selections for two or more products, the Excel spreadsheet autocalculated a ranking of products across the nine domains based on the answers provided. The selection of ‘don’t know’ also caused the spreadsheet to identify where more information was required to be sought. This framework, associated tools, a glossary of terms, and digital and written instructions to guide use were all developed through the small circle co-design processes, and then went into Phase 5 focused on testing and refining. A brief survey of the characteristics of people who registered to use the framework and proposed goals of use was developed by the academic researchers in the team.
Participant recruitment for testing and refining the tool (Phase 5) and implementing and learning (Phase 6)
To test and refine the first iteration of the framework developed by the small circle co-design team, large circle stakeholder engagement (McKercher 2020) was undertaken at national professional development conferences with health professionals who offer clinical advice on assistive technology to people with ABI. Conferences had a focus on health professional practice, including in the areas of assistive technology and acquired brain injury. Workshop 1 was hosted at the Australiasian Society for the Study of Brain Impairment (ASSBI) Conference in November 2020, Workshop 2 was hosted at the Occupational Therapy Australia National Conference in June 2021 and Workshop Three was hosted at the ASSBI Conference in May 2022.
Health professionals were recruited into one of the three co-design workshops, and/or the final implementation workshop, using a structured abstract and public domain advertising for each workshop. The abstracts for all three co-design workshops stated that attendees would be offered the opportunity to test and provide feedback on tools being developed by the small circle co-design team. Attendance at each of the four workshops, and an online registration form to download and access the Framework, was accepted as implied consent for study participation as approved by Monash University Human Research Ethics Committee (Project ID: 37284).
The three co-design workshops were hosted online using the Zoom platform. The final implementation workshop (Workshop 4) was held in person at the Australian Assistive Technology Conference hosted by the Australian Rehabilitation and Assistive Technology Association (ARATA) at the Gold Coast Convention and Exhibition Centre in Queensland, Australia, in November 2024. Results from the workshops delivered during the phases of testing and refining the tool (Phase 5), and the online registration form filled out to download and access the Framework (as part of Phase 6: implementing and learning) will now be presented.
Results
Phases 5 and 6 co-production results
Using iterative large circle stakeholder engagement, Phases 5 and 6 delivered the final version of the co-designed and co-produced framework, as follows:
A total of 74 people attended the three 1-hour co-design workshops. In Workshop 1, the 13 participants included six occupational therapists, three neuropsychologists, two speech pathologists and two ‘other’ professionals. In Workshop 2, all 38 attendees were occupational therapists. For Workshop 3, the 23 participants included eight neuropsychologists, six occupational therapists, six speech pathologists, two psychologists,and one ‘other’ professional.
One of the researchers with lived experience of ABI (author 4 on this paper) and two of the practicing health professionals inputted to the co-design of the content for all three workshops with the researchers. They also contributed via pre-recording of videos or provision of digital images and verbatim written content to be used in each workshop. Each of the three 1-hour workshops used an online polling system (Mentimetre) to survey and collect feedback from participants on both the drafted tool domains, as well as how the use of the tool within lived experience scenarios presented in the workshop impacted each attendee’s level of self-rated confidence and capability to support people with ABI to consider assistive technology used for cognitive support.
Open-ended responses were invited using the online polling system across the three workshops to give direct feedback on other framework design considerations. These open-ended responses were presented back to the small circle co-design group after each workshop for discussion. They included suggestions of questions to be added under specific domains (n = 19), clarification of questions in the drafted framework (n = 7) and suggestions of ways to train others to use the framework (n = 4 being short videos, written fact sheets, scenarios demonstrating use with people with ABI and podcasts). Suggestions of questions to be added or clarified were used to cross-check and/or directly mark-up additions to the draft version of the framework at each timepoint. A subsequent iteration of the framework was then presented to and discussed with the small circle co-design team after each workshop. Drawing on this combined small circle and large group feedback, edits were iteratively made to produce subsequent framework versions for further large group review and input.
Based on workshop feedback, the small circle co-design team agreed to a change of name of the framework from the Performance Enhancing Technology Framework to a Framework for Evaluation of Assistive Technology (FEAT). To complete the co-design and co-production process – and implement and learn about experiences of use of the FEAT – a 3-hour implementation workshop was hosted in person at the Australian Assistive Technology Conference in November 2024 with health professionals and other assistive technology advisors working with people who use assistive technology for executive function support. Thirty-one people attended the final 3-hour implementation workshop (Workshop 4), including 22 occupational therapists, three representatives from funding bodies, one speech pathologist, one neuropsychologist, two assistive technology peer mentors (i.e. people with disability lived experience who use assistive technology in their own lives and mentor others on its use) and five ‘other’ professionals. As suggested by previous large group co-design participants, short videos, written fact sheets and scenarios demonstrating use of the FEAT with people with ABI were used in this implementation workshop.
Following these four co-design and co-production activities, the FEAT was finalised as a fillable Excel spreadsheet with an associated PDF and Word Version fact sheet produced and made available on the My Technology Space website. Based on the iterative co-design processes, there are now 62 items (across the nine domains) on the FEAT, all of which have ‘yes’, ‘no’ or ‘don’t know’, and an open text box for comments. For 28 items, there is also the option to select ‘not applicable’. An additional tab was also added to the final Excel spreadsheet, to provide a summary of the text comments, and an option to print the summary as a pdf file. Multiple products can be compared on the FEAT, leading to a percentage ranking of the product with the most affirmative responses, and highlighting for those products where there were entries by the respondent that they did not know the answer to a question, and thus may be needed to get more information. The FEAT can be downloaded and used for free by completing a short survey via the Selecting Assistive Technology page on the My Technology Space website. This short survey enables the co-design research group to learn about user characteristics and goals for use of the FEAT.
Discussion
In the context of rapid growth of both mainstream and assistive technology, tools to guide consideration and selection of technology for cognitive support following ABI have been lacking. Using a small circle co-design team, and large group co-design and co-production processes within a six-stage co-design process, this approach has co-produced a tool called the Framework for Evaluation of Assistive Technology (FEAT). The FEAT can be used by assistive technology advisors in collaboration with people with ABI and other key supporters to evaluate assistive products that may be used for cognitive support, and has gone through multiple testing and iterations. Drawing on both the small circle team members and iterative large circle co-design processes, input has been received from the assistive technology stakeholder groups central to good assistive technology outcomes. These groups include assistive technology users, health professionals working with them, and policy makers and funders in the field of assistive products and services.
The two classes of assistive technology that are recognised in the Australian New Zealand Standard that can be used as interventions to address cognitive functioning within everyday activities following ABI (being Class 04 and Class 22) have within them a combined 32 subclasses of assistive products that may be relevant to consider by people with ABI and advisors working with them (International Standards Organization 2023). The FEAT is a resource that can assist to consider both the features and use of such products. It also addresses recent calls for resources to build health professionals’ capability, and the identified gap in clinician implementation of, assistive technology in brain injury rehabilitation (Pilli et al. 2023). Person-centred assistive technology approaches are recognised as good practice (Layton et al. 2022). Although health professionals may have previously made decisions regarding assistive products or other types of interventions for individuals, there have been more recent calls for consumers and providers to be actively involved in the design of such interventions (Benz et al. 2024, p. 1). However, the large number of products, and burgeoning area of technology development, in retail markets means that decision making can be challenging. The FEAT has been developed to assist to address this challenge, by posing questions about products that can be explored through both examining the product/s under consideration, and by guiding conversations between the person with ABI, their key supporter/s and an assistive technology advisor.
Co-design is proposed to be ‘outcome-generating methodology that proactively enables the inclusion of people with disability and service providers through community-based participatory research and action’ (Benz et al. 2024, p. 1). Co-design and co-production processes have been useful in the current project to actively involve people with ABI, as well as health professionals and funders of services, in an iterative process of Framework design. This input – undertaken via input from both the small circle co-design team and large group co-design engagement – has been ‘outcome-generating’, with the framework being renamed, undergoing multiple iterations and testing, and also expanded to a set of 62 items (after an initial 58 drafted statements) across nine domains. However, authentic co-design is recognised to take both extended timeframes and resources (McKercher 2020), and this longer time allowance – as well as both human and financial resources – was certainly necessary for the current project.
Beyond the time and resources taken, another important finding of the co-design and co-production processes used in this study was the multiple domains – and expansive set of questions – that may need to be explored when considering technologies used for cognitive support following brain injury. This finding adds further evidence to past research by Bridge et al. (2021), which identified that a person considering assistive technology for cognitive support may benefit from discussion to make more informed decisions about technology use, as ‘… end-users are often unaware of how the [assistive product] data is used, and the privacy settings on many smart home ecosystems are not tailored to privacy needs’ (Bridge et al. 2021, p. 11). Large circle engagement led to both health professionals and funders indicating that technology decisions are multi-factorial in nature, and trade-offs can occur. Thus, responses to the finalised nine FEAT domains must combine to inform product selection, trial and evaluation of effectiveness.
Although mainstream technologies have been identified to hold promise in supporting executive functioning following ABI (Andelic et al. 2019), aligned with recent research by Wong et al. (2017), findings from the current study have reiterated that there are multiple factors to consider for successful uptake and use of technologies, including ease of use, effectiveness and availability of technical support over time. Given these multiple factors, the FEAT also demonstrates through its 62 items that it can take significant time to plan and implement such interventions. The FEAT offers a process for ensuring that such planning is linked to in-depth questioning of the product use, performance and associated impacts on the technology user experience upon trial or over time within ongoing use.
Limitations and further research
This study does have limitations. First, those people with lived ABI experience who joined the small circle co-design group were invited by sending details of the study to community-based health professionals working in the field of ABI. This may have limited the diversity of lived experience co-designers engaged, and thus perspectives from this important stakeholder group – that is, assistive technology users with ABI. Although the FEAT was designed for use by assistive technology advisors in conversation with people with ABI and their other key supporters (like family members), future research that expands testing of the FEAT with, and gathers broader feedback from, people with ABI would be useful to further evaluate the framework utility with assistive technology users.
The large circle stakeholder engagement was undertaken within existing professional development conferences that attendees self-selected to attend, rather than via purposive and prospective targeted participant recruitment. In addition, all engagement was undertaken in group work, and used scenarios from the small circle co-design team members with lived experience of ABI and health professionals working with them. This may have impacted both the level of engagement of each stakeholder and the perspectives gained to inform the final framework version. Future studies that purposively recruit assistive technology advisors to test and evaluate the FEAT performance within clinical practice would be useful.
Finally, although co-design and co-production has led to the version of the Framework now available, and the short survey completed to access the tool will offer some evidence of user characteristics and goals for use of the framework, further research on the FEAT is required. Specifically, research that tests and evaluates the use of the FEAT to inform assistive technology decision making – including the impact of the framework on both knowledge and skills of assistive technology advisors – would be valuable. Any such research should utilise rigorous research designs that include both subjective and objective measures of impact on the behaviours of health professionals collaborating with people with ABI, as well as the experiences and outcomes of people using the final selected AT for cognitive support.
Conclusion
The FEAT is a co-designed and co-produced tool that can be used by health professionals and other assistive technology advisors in collaboration with people with ABI and their other key supporters. The FEAT has been developed to assist people to choose and evaluate assistive products used for cognitive support within everyday activities following brain injury. The co-design and co-production processes used in this project offered iterative testing and refinement through multiple versions, enhancing the finalised framework. The FEAT is a free tool that is available for use, and will undergo further testing and evaluation from the perspectives of assistive technology advisors, as well as assistive product users with ABI.
Data availability
Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data are not available.
Conflicts of interest
Grahame Simpson is an Editor-in-Chief for Brain Impairment but was not involved in the peer review or decision-making process for this paper. The authors have no further conflicts of interest to declare.
Declaration of funding
This research was funded by the Transport Accident Commission with icare NSW (Grant # T012: Trial of smart home technology following Traumatic Brain Injury).
Ethics
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and international committees on human experimentation, and with the Helsinki Declaration of 1975, as revised in 2008.
Acknowledgements
The authors thank Libby Witts and Scott Thomas for the valuable lived experience expertise they offered to this project; Sue Sloan, Jan Mackey and Sheridan Vines for their extensive clinical perspectives offered across neuropsychology, speech pathology and occupational therapy; Jade Hurst, Rhys Ashpole and Lloyd Walker for their diverse and comprehensive injury and disability insurance policy and funding perspectives; and Lisa Licciardi, Brendan Worne, Kate Mather and Robyn Tate for the application of the Framework for Evaluation of Assistive Technology within single case experimental design work delivered within other phases of the T012 Trial of smart home technology following traumatic brain injury project.
References
Andelic N, Løvstad M, Norup A, Ponsford J, Røe C (2019) Editorial: Impact of Traumatic Brain Injuries on participation in daily life and work: Recent research and future directions. Frontiers in Neurology 10, 1153.
| Crossref | Google Scholar | PubMed |
Benz C, Scott-Jeffs W, McKercher KA, Welsh M, Norman R, Hendrie D, Locantro M, Robinson S (2024) Community-based participatory-research through co-design: supporting collaboration from all sides of disability. Research Involvement and Engagement 10, 47.
| Crossref | Google Scholar | PubMed |
Bridge C, Zmudzki F, Huang T, Owen C, Faulkner D (2021) Impacts of new and emerging assistive technologies for ageing and disabled housing: AHURI Final Report No. 372. Melbourne, Australia: Australian Housing and Urban Research Institute Limited. doi:10.18408/ahuri7122501
Brunner M, McGregor D, Keep M, Janssen A, Spallek H, Quinn D, Jones A, Tseris E, Yeung W, Togher L, Solman A, Shaw T (2018) An eHealth capabilities framework for graduates and health professionals: Mixed-methods study. Journal of Medical Internet Research 20(5), e10229.
| Crossref | Google Scholar | PubMed |
Chuah SL, Whiting DL, Gates TM, Simpson GK (2024) Factors associated with device, Internet and videoconferencing use amongst individuals with moderate-to-severe traumatic brain injury. Healthcare 12(14), 1371.
| Crossref | Google Scholar | PubMed |
Jamieson M, Jack R, O’Neill B, Cullen B, Lennon M, Brewster S, Evans J (2020) Technology to encourage meaningful activities following brain injury. Disability and Rehabilitation: Assistive Technology 15(4), 453-466.
| Crossref | Google Scholar | PubMed |
Jeffay E, Ponsford J, Harnett A, Janzen S, Patsakos E, Douglas J, Kennedy M, Kua A, Teasell R, Welch-West P, Bayley M, Green R (2023) INCOG 2.0 Guidelines for Cognitive Rehabilitation Following Traumatic Brain Injury, Part III: Executive Functions. Journal of Head Trauma Rehabilitation 38(1), 52-64.
| Crossref | Google Scholar | PubMed |
Layton N, Callaway L (2020) Person-centred approaches to evaluation of assistive technology products and outcomes: An Australian policy and practice perspective. Presentation at the Rehabilitation Engineering Society of North America Annual Conference (online event). Available at https://www.resna.org/sites/default/files/conference/2020/GAATO/144Layton.html
Layton N, Bould E, Buchanan R, Bredin J, Callaway L (2022) Inclusive research in health, rehabilitation and assistive technology: Beyond the binary of the ‘Researcher’ and the ‘Researched’. Social Science 11, 233.
| Crossref | Google Scholar |
Leopold A, Lourie A, Petras H, Elias E (2015) The use of assistive technology for cognition to support the performance of daily activities for individuals with cognitive disabilities due to traumatic brain injury: The current state of the research. NeuroRehabilitation 37(3), 359-378.
| Crossref | Google Scholar | PubMed |
Liddle J, Callaway L, Simpson G (2021) Occupational Therapy Australia, 29th National Conference and Exhibition, “Inspired Insights for Brighter Futures”, 23–25 June 2021, Virtual Conference and Engagement Hubs. Australian Occupational Therapy Journal 68(S1), 7-91.
| Crossref | Google Scholar | PubMed |
Malterud K (2001) The art and science of clinical knowledge: Evidence beyond measures and numbers. Lancet 358(9279), 397-400.
| Crossref | Google Scholar | PubMed |
Nam JH, Kim H (2018) How assistive devices affect activities of daily living and cognitive functions of people with brain injury: a meta-analysis. Disability and Rehabilitation: Assistive Technology 13(3), 305-311.
| Crossref | Google Scholar | PubMed |
Pilli K, Worne B, Simpson GK (2023) Clinician experiences with using assistive technology in brain injury rehabilitation: a survey of clinician capability, attitudes, and barriers. Brain Impairment 24(2), 185-203.
| Crossref | Google Scholar | PubMed |
Smith EM, Graham D, Morgan C, MacLachlan M (2023) Artificial intelligence and assistive technology: risks, rewards, challenges, and opportunities. Assistive Technology 35(5), 375-377.
| Crossref | Google Scholar | PubMed |
Social Care Institute for Excellence (n.d.) Co-production: what it is and how to do it. Available at https://www.scie.org.uk/co-production/what-how/
Tate R, Kennedy M, Ponsford J, Douglas J, Velikonja D, Bayley M, Stergiou-Kita M (2014) INCOG recommendations for management of cognition following traumatic brain injury, part III: executive function and self-awareness. The Journal of Head Trauma Rehabilitation 29(4), 338-352.
| Crossref | Google Scholar | PubMed |
Togher L, Douglas J, Turkstra LS, Welch-West P, Janzen S, Harnett A, Kennedy M, Kua A, Patsakos E, Ponsford J, Teasell R, Bayley MT, Wiseman-Hakes C (2023) INCOG 2.0 Guidelines for Cognitive Rehabilitation Following Traumatic Brain Injury, Part IV: Cognitive-Communication and Social Cognition Disorders. The Journal of Head Trauma Rehabilitation 38(1), 65-82.
| Crossref | Google Scholar | PubMed |
Vargas C, Whelan J, Brimblecombe J, Allender S (2022) Co-creation, co-design and co-production for public health: A perspective on definitions and distinctions. Public Health Research & Practice 32(2), e3222211.
| Crossref | Google Scholar | PubMed |
Vears DF, Gillam L (2022) Inductive content analysis: A guide for beginning qualitative researchers. Focus on Health Professional Education: A Multi-Professional Journal 23(1), 111-127.
| Crossref | Google Scholar |
Wong D, Sinclair K, Seabrook E, McKay A, Ponsford J (2017) Smartphones as assistive technology following traumatic brain injury: a preliminary study of what helps and what hinders. Disability and Rehabilitation 39(23), 2387-2394.
| Crossref | Google Scholar | PubMed |
World Health Organization (2024) Fact sheets: assistive technology. World Health Organization: Geneva, CH. Available at https://www.who.int/news-room/fact-sheets/detail/assistive-technology
