Evaluation of an integrated housing and recovery model for people with severe and persistent mental illnesses: the Doorway program*David R. Dunt A H , Andrew W. Benoy B , Andrea Phillipou C , Laura L. Collister D , Elizabeth M. Crowther D , Julian Freidin E and David J. Castle F G
A Centre for Health Policy, Melbourne School of Population and Global Health, The University of Melbourne, Vic. 3010, Australia.
B Nous Group, Level 10, 485 Bourke Street, Melbourne, Vic. 3000, Australia. Email: Andrew.Benoy@nousgroup.com.au
C Department of Mental Health, St Vincent’s Hospital, Level 2, 46 Nicholson Street, Fitzroy, Vic. 3065, Australia. Email: Andrea.firstname.lastname@example.org
E Homeless Outreach Psychiatric Service, Department of Psychiatry, Alfred Hospital, PO Box 315, Prahran, Vic. 3181, Australia. Email: Julian.email@example.com
F St. Vincent’s Hospital, Melbourne, Nicholson Street, Fitzroy, Vic. 3065, Australia. Email: David.firstname.lastname@example.org
G The University of Melbourne, PO Box 2900, Fitzroy, Vic. 3065, Australia.
H Corresponding author. Email: email@example.com
Australian Health Review 41(5) 573-581 https://doi.org/10.1071/AH16055
Submitted: 26 February 2016 Accepted: 23 August 2016 Published: 21 October 2016
Journal Compilation © AHHA 2017 Open Access CC BY-NC-ND
Objective The Doorway program is a 3-year pilot integrated housing and recovery support program aimed at people with a severe and persistent mental illness who are ‘at risk’ or actually homeless. Participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support. This arrangement is highly innovative, differing from widely favoured arrangements internationally involving congregate and scattered-site housing owned or managed by the support program. The aim of the present study was to determine the effects of the Doorway program on participants’ health, housing, service utilisation and costs.
Methods A pre-post study design was used with outcome measures consisting of a number of question inventories and their costs (where relevant). The principal inventories were the Behaviour and Symptom Identification Scale 32 (BASIS-32), a consumer-oriented, self-report measure of behavioural symptoms and distress, the Health of the Nation Outcome Scale (HoNOS), an interviewer-administered measurement tool designed to assess general health and social functioning of mentally ill people and the Outcomes Star (Homelessness) system which measures various aspects of the homelessness experience. Baseline measurements were performed routinely by staff at entry to the program and then at 6-monthly intervals across the evaluation period.
Results For 55 of 59 participants, total mean BASIS-32 scores (including as well three of five subscale scores) improved significantly and with moderate effect size. Four of the 10 domain scores on the Outcome Star (Homelessness) inventory also improved significantly, with effect sizes ranging from small–medium (three domains) to large (one domain). Mean usage of bed-based mental health clinical services and general hospital admissions both significantly decreased (with overall net savings of A$3096 per participant per annum). Overall cost savings (including housing) to government ranged from A$1149 to A$19 837 depending on the housing type comparator.
Conclusion The Doorway program secured housing for this vulnerable group with additional benefits in client outcomes, including reduced use and cost of health services. These findings, if confirmed in larger studies, should have widespread applicability internationally.
What is known about the topic? Beneficial effects of housing and recovery programs (Housing First) on people with severe and persistent mental illness and who are ‘at risk’, or actually homeless, are being demonstrated in Northern America. These effects include housing security, well being, health service utilisation and cost effects on government. However, these beneficial effects can only be regarded as settled for housing security. The highly innovative Doorway care model in which participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support, has not been investigated previously.
What does this paper add? This paper adds new data on the Doorway care models, it’s effects and costs, particularly with regard to participant behavioural distress and social functioning.
What are the implications for practitioners? The beneficial effects of this innovative model, if confirmed in larger studies, should have widespread applicability internationally.
Additional keywords: health funding and financing, health services research, mental health, models of care.
It is of concern that the recent Australian Study of High Impact Psychoses (SHIP) found that 5.2% and 12.8% of individuals with a psychotic illness had been homeless in the previous month and previous year respectively, the latter with a mean of 155 homeless days.2,3 Homeless people frequently do not have the financial resources or appropriate references to gain private housing tenure. Public housing waiting lists are often long and often availability is not in locations preferred by applicants. This further alienates them from social contacts and engenders a sense of isolation and disengagement from society. The importance of housing tenure in contributing to the recovery of individuals with a severe and persistent mental illness (SPMI) is clear.4
Doorway is a 3-year pilot integrated housing and recovery support program delivered by the MI Fellowship.1 It is designed to enhance the capacity of individuals with an SPMI requiring services and who are homeless, or at risk of homelessness, to lead independent, healthy and meaningful lives in housing and communities of their choice. Homelessness in contemporary Australia should be understood to consist of several (identified) arrangements for shelter below a minimum community standard. This is a small rented flat with a separate bathroom and kitchen and an element of security of tenure.5 The Doorway program integrates interventions to improve the client’s housing situation with efforts to improve social inclusion and support recovery.
The Doorway model builds upon and adapts the Housing First model that was pioneered in the US in the early 1990s.6 These models are built on the assumption that stable housing plays a critical role in the recovery of people with SPMI. Several large randomised control trials based on the Housing First model have now been conducted in Canada as part of the 5-cities At Home/Chez Soi Trial and the studies indicate consistent improvements with residential stability, social inclusion and reduced contact with the justice system.7,8 Effects on levels of quality of life and substance abuse show more mixed results.9–11 The research literature overall makes clear that the Housing First model has demonstrated real and substantial social benefits to participants.
A key difference between the Doorway model and other iterations of the original Housing First model is that participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support. This integrated care model is highly innovative, differing from widely favoured arrangements internationally that involve congregate and scattered-site housing owned or managed by the support program. The Doorway model also differs from other programs that have operated in Australia that have involved securing housing, but only through facilitating contact with other housing-specific agencies, such as the Housing and Accommodation Support Initiative (HASI) in New South Wales, the Journey to Social Inclusion (J2SI) in Victoria and Project 300 in Queensland.12–14
The aims of the present study were to determine the effects of the Doorway program, an innovative and little-studied care model where participants source and choose properties through the open rental market, on participants’ well being (health, access to housing, employment and social inclusion), as well as to perform an appraisal of the costs of participants’ health services and housing use (in the presence and absence of Doorway).
The Doorway program
As stated, the Doorway model supports participants to choose, access and sustain their own private rental accommodation by subsidising their rental payments where required. In addition, Doorway’s housing and recovery workers (H&RWs) support participants to, for example, develop tenancy skills and build natural support networks. The Doorway H&RWs are graduate-level staff, intentionally including some people with a lived experience of mental illness and/or homelessness. Integrated support teams are created for each Doorway participant. To do this, Doorway H&RWs are embedded in the public sector Acute Mental Health Services (AMHSs) within the relevant hospital catchment areas (see below) and provide housing and recovery inputs to care. AMHS staff also form part of these integrated support teams, providing clinical care, including case management. Other community-based health services may also be involved for specialised purposes. ACMHs, and specifically the case manager, exercise governance for these different program inputs into an individual participant’s care.
The Doorway pilot was implemented with three clinical partners across inner city and suburban Melbourne as well as country Victoria between July 2011 and June 2014. These clinical partners were Austin and Repatriation Medical Centre as part of Austin Health (covering the cities of Banyule and Nillumbik), St Vincent’s Hospital as part of St Vincent’s Health (covering the City of Yarra) and Latrobe Regional Hospital as part of Gippsland Health (covering Baw Baw Shire and Latrobe City Council). The demographic characteristics, levels of social and economic disadvantage, rental affordability and mental health characteristics of participants differ across each region, sometimes being better, sometimes worse than those in Victoria overall.1 The program was funded by the Department of Health and Human Services of Victoria for a 3-year period (30 June 2011–30 June 2014).
The present study used a pre-post design and encompassed both quantitative and qualitative dimensions. The evaluation period concluded in November 2013, 7 months before the end of the 3-year funded period at the request of the funding body.
Inclusion criteria for admission to the program were: (1) severe mental illness requiring services from an adult mental health service; (2) homelessness or at risk of imminent homelessness; (3) currently case-managed by an adult mental health service; and (4) receiving a Disability Support Pension (DSP) or Newstart Allowance.
Operational inclusion criteria were: willing to accept support; willing to give consent for members of the integrated team to share information with each other; living in doorway catchment areas; and needing to consent to data collection, sign a lease and contribute to rental payments.
Baseline measurements were performed at entry to the program and then at 6-monthly intervals across the evaluation period.
Outcomes measurement tools
The Behaviour and Symptom Identification Scale 32 (BASIS-32) is a consumer-oriented, self-report measure of behavioural symptoms and distress; it has five subscales and 32 items, each rated on five-point scales.15,16 The Health of the Nation Outcome Scale (HoNOS) is an interviewer-administered measurement tool designed to assess general health and social functioning of mentally ill people.17 The HoNOS has 12 subscales, bundled under four headings (Behavioural problems, Symptomatic problems, Impairment, Social problems). Each item is rated on a five-point scale.
Outcomes Star (Homelessness) (http://www.outcomesstarsystem.org.uk, accessed 2 November 2015) has 10 domains measuring various aspects of the homelessness experience, namely internal motivation, social networks, managing money, offending, mental health, physical health, living skills, meaningful use of time, managing tenancy and substance abuse.
Doorway staff collected both quantitative and qualitative data relevant to housing, employment and social inclusion. For housing, these data included, for example, the proportion of a participant’s housing preferences met, the incidence and amount of rental arrears, and breach of duty notices. For employment, variables collected included engagement in paid and unpaid employment, steps taken to find work and seeing an employment consultant, such as an Australian Government-funded disability employment service, accessing education and vocational training opportunities and receiving qualifications. For social inclusion, data collected included the composition of participants’ natural support networks, contacts with and attendance at court and contact with police.
Health system utilisation datasets
De-identified data from three Victorian Government Department of Health (DoH) datasets were provided for participants in the program. These datasets were: (1) the Client Management Interface/Operational Data Store (CMI/ODS), which provides data on Victorian public mental health service usage;18 (2) the Victorian Admitted Episodes Dataset (VAED) for data on Victorian hospital use; and (3) the Victorian Emergency Minimum Dataset (VEMD), which provides data on emergency department (ED) use. Data were not available for ambulance call-outs, use of drug and alcohol services or general practitioner (GP) consultations.
Costs of mental health and general health services were derived from these client contact data in conjunction with unit cost data derived from the Victorian Government DoH, as well as other published sources (see Table 1) (Victorian Government Department of Health, pers. comm.).18–21 Doorway program costs were derived from Doorway program funding documents (see Table 1), including Doorway client support costs, rental subsidy support and program management and operational costs.22,23 These clinical and housing components were valued at A$10 136 and A$7937 per participant per annum respectively. Together with program management and operation costs of A$1228, these totalled A$19 300 per participant per annum.
Selected social and demographic characteristics of participants
Social and demographic characteristics included sex, age (mean), country of birth, Aboriginal or Torres Strait Islander status, identified carer (at point of referral), parent (single or couple), receiving DSP payments at January 2013 and mean fortnightly income at January 2013.
Analyses of HoNOS, BASIS-32 and Outcomes Star Homelessness measures compared scores for individuals at their entry to and exit from the program (relevant quarter year ± 3 months if scores unavailable). If multiple scores were in the same quarter, mean scores were used. Null values assumed to correspond with a rating of 0.
The time of exit used varied slightly for individual measures (details are provided in table footnotes). Utilisation and cost data was also compared at program entry and end. Following normality checking and removal of outliers, statistical analysis of continuous variables was conducted using paired-sample t-tests, where paired continuous data existed. Categorical data were assessed with Chi-squared tests, generally with unpaired data. All analyses were performed using IBM SPSS version 22.0. Some data were available during but not before the program and are marked ‘Post only’. Cost data based on official estimates of average costs (multiplied by the number of contacts of use) was not suited to statistical analysis, which was not then performed. Unless indicated otherwise, data are given as the mean ± s.d. or as frequency (percentage).
Effect sizes using Cohen’s d-test were also estimated from results of pre-post change in outcome measures with statistically significant results to indicate whether changes were, in relative terms, ‘small’, ‘medium’ or ‘big’. Statistical tests were not performed on some data, such as cost data that were based on best estimates rather than directly measured values.
Ethics approval was granted for the project by Austin Health’s Ethics Committee, with matching approval from ethics committees at both St Vincent’s and Latrobe Hospitals. On entry to the program, participants provided signed approval for access to their clinical and other relevant data for program operation purposes. These data, following de-identification and which were also not re-identifiable, was judged by the human research ethics committee not to require further consent when used for research purposes. Victorian Data Linkages (VDL), established by the Victorian and Australian governments as part of the National Collaborative Research Infrastructure Strategy (NCRIS) to develop new data linkage capacity in Victoria, provided these de-identified data to the study team in encrypted form. No participants withdrew from the program as a result of the requirement to share information for program operation purposes.
Study population and throughput
During the evaluation period, 77 people went through the intake process and, of these, 59 took up residence in private rental properties under the program, with 50 still in residence at the end of the evaluation period (Fig. 1). Analyses are based on available data for 55 of the 59 participants (Austin, n = 20; St Vincent’s, n = 12; Latrobe, n = 23). As recorded by Doorway program staff, nine of these people left the program early. Reasons for early exit included social-improved outcomes (reconnecting with family), financial sustainability (living with someone else etc.), health reached recovery sustainability and increased support needs. Reasons for leaving early were judged to be evenly balanced between positive steps to recovery and ongoing recovery challenges.
Selected social and demographic characteristics of the participants are given in Table 2. These show that participants were most commonly Australian-born, single middle-aged males receiving a DSP. Schizophrenia (49%), followed by depression (25%), was by far the most common primary mental health diagnosis. More than one-third of participants had multiple mental health diagnoses. Based on their Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) scores, (http://www.who.int/substance_abuse/activities/assist/en/, accessed 15 September 2016) the most common problems were tobacco products (60% moderate risk; 20% high risk) and alcoholic beverages (60% moderate risk; 7% high risk). The use of cannabis and amphetamine-type stimulants was also common.
Prior to entering the program, based on the Chamberlain definition of homelessness (which refers to the following arrangements for shelter below a minimum community standard), 17% of participants were ‘primary homeless’ (i.e. without conventional accommodation), 50% were ‘secondary homeless’ (i.e. moving between various forms of temporary shelter), 21% were ‘tertiary homeless’ (i.e. living in single rooms in private boarding houses and 10% were ‘marginally housed’.5 This last category refers to living in housing situations close to minimum standard such as caravan parks.
Twenty-eight per cent of participants were on the public housing waiting list. The most common primary cause for their homelessness was their mental illness (50%), followed by relationship breakdowns (15%). Three participants had previously been long-term residents of a state-funded Continuing Care Unit (a community-based treatment facility).
Total mean BASIS-32 scores decreased significantly (improved) across the evaluation period from 1.3 ± 0.8 BASIS points before Doorway to 0.8 ± 0.6 points after Doorway (Cohen’s d = 0.7; 38% improvement in baseline levels; P = 0.04; n = 23). Scores decreased (i.e. improved) for each of the five subscales, with three subscales showing statistically significantly improvements, namely Relation to self/others (Cohen’s d = 0.6, P < 0.001), Depression/anxiety (Cohen’s d = 0.5, P = 0.01) and Daily living/role function (Cohen’s d = 0.5, P = 0.02; Fig. 2).
The mean total HoNOS score decreased (improved) from 10.0 ± 4.9 before housing to 8.8 ± 5.1 after housing (of a maximum score of 48; n = 35), but the change was not significant. Mean scores also improved across three of the four HoNOS domains, although none reached statistical significance.
Over the period of engagement with the program, there was one incident relating to a possible overdose, one relating to self-harm and six relating to medical concerns, physical assault and antisocial behaviour.
Homelessness and housing outcomes
There were statistically significant improvements for four of the Homelessness Star’s 10 domain scores, namely Motivation and taking responsibility (16.4% improvement from baseline; Cohen’s d = 0.43, P = 0.00), Managing money (23% improvement from baseline; Cohen’s d = 0.32, P = 0.01), Emotional and mental health (14.3% improvement from baseline; Cohen’s d = 0.36, P = 0.01) and Meaningful use of time (21.4% improvement from baseline; Cohen’s d = 1.29, P = 0.01). There were improvements in all six of the other domains, but these did not reach statistical significance (Fig. 3).
The period of time that participants were housed in rental accommodation ranged from 3 to 21 months (Post only).
The majority of participants’ housing preferences (proximity to family, health services and community resources) were met in the rental accommodation that they eventually occupied. By the end of the evaluation period, 31 (56%) participants had received 12-month lease extensions by their Property Managers, with three more on a month-to-month basis (all Post only).
The mean rental gap paid by Doorway to participants was A$194 ± 159 per fortnight at the end of the evaluation period, down from $376 ± 174 per fortnight at the beginning of the period (P < 0.001).
With regard to adverse events, 11 (20%) of Doorway participants had fallen into rental arrears on at least one occasion. There had been 10 breach of duty notices issued, numerous verbal warnings and some complaints from neighbours, three instances of property damage and six instances of lease breaks. Only one participant had any utilities disconnected during the evaluation period (all Post only).
Employment and training outcomes
There were modest improvements in outcomes for the proportion of participants engaged in paid and unpaid employment, taking steps to find work, seeing an employment consultant, accessing education and vocational training opportunities and receiving qualifications for their vocational training. However, these improvements were not statistically significant.
Social inclusion and antisocial behaviour outcomes
The composition of participants’ natural support networks evolved over time. The increase in contacts with ‘Others’, such as neighbours, work colleagues and local shop and café owners, rose from 14% to 59% and was significant (χ12 = 7.72, P < 0.01). There was also a large increase in contacts with ‘Friends’, from 45% to 68%, but this was not significant (χ12 = 1.42, P = 0.23).
Eleven (20%) Doorway participants had had 19 reported contacts with the court system related to criminal or civil matters, with the majority resulting in positive outcomes, such as intervention orders not being placed or lifted. Eleven (20%) participants had 21 reported contacts with the police with charges laid in only one case (all Post only).
Utilisation and cost of health services including the Doorway program
Admission to bed-based mental health services (clinical and community) decreased substantially (from 1.2 ± 2.1 to 0.5 ± 1.4 admissions per participant per annum; t50 = 3.01, P < 0.1). There was also a substantial decrease in total average bed days of bed-based mental health services from 20.0 ± 34.5 to 7.4 ± 26.5 days per year per participant (n = 51; t = 2.81, P < 0.01). Cost savings based on the reduction in bed-day usage were estimated at A$7355 per participant per annum (Table 3).
Contact with ambulatory mental health professionals decreased from a mean of 39.4 to 33.5 h per year per participant. The greatest reductions were in services provided by Mobile Support and Treatment Teams (MSTT) and Continuing Care Teams (CCT). There was an estimated saving of A$1882 per participant per annum.
Doorway participants’ decreased their use of specialised mental health supports (e.g. case managers (Fisher P < 0.01) and psychiatrists (Fisher P < 0.001)) over time.
Sixteen (33%) of the 48 participants enrolled in the Doorway program at the end of the evaluation period had been formally discharged from their adult mental health services to GP or similar care. Six of the nine participants who had been subject to Community Treatment Orders (CTO) at program intake had had these lifted (all Post only).
The total number of ED presentations decreased from 1.94 to 1.42 per participant annually (t50 = 1.75, P = 0.09). ED presentations leading to admission decreased from 0.50 to 0.21 per participant annually and ED presentations not leading to admission decreased from 1.48 to 1.23 per participant annually. Using available costs estimates, savings associated with ED presentations were A$349 per participant per annum (Table 3).
The total number of general hospital admissions decreased (from 0.45 to 0.12 annually per participant; t50 = 3.12, P < 0.01). The largest reduction in use was in the area of General Medicine, decreasing from 0.24 to 0.05 annually per participant. Estimated savings in hospital admissions per participant annually were A$1447 (Table 3).
Summary for all health service usage cost savings
Based on the above estimates of health service utilisation costs, savings relating to all health service usage (excluding Doorway direct client care services) were estimated at A$11 033 per participant per annum. When Doorway direct client care services, estimated at A$7937, were included, cost savings were reduced to A$3096 per participant per annum.
Costs associated with housing
Annual housing costs per Doorway participant can be compared against the costs of different types of social housing (Table 4). Where the cost of capital (to government) is available and included, the total housing cost per Doorway participant per annum of A$10 136 (as part of the total Doorway funding of A$19 300 per participant per annum) was lower than the annual costs of all types of social housing. Approximately one-third of Doorway participants resided in some form of social housing before joining the program.
Total cost savings also take into account Doorway managerial and operational costs (A$1228 per participant annually). These are shown per participant per annum in comparison with various forms of social housing in Table 4. The data in Table 4 indicates that Doorway produced overall net savings, the magnitude of which depended on the form of housing type used as a comparator. These savings were estimated to be at least A$1149 (for community housing). This is an underestimate, because government investment, in capital costs for community housing, were not available and were excluded. Cost savings were as high as A$19 837 when crisis accommodation housing was used as the comparator.
The program reported here was effective in securing subsidised rental housing for 59 people from this vulnerable group. It had additional benefits in terms of significantly improved client outcomes (as evidenced by total mean BASIS-32, including 3 of the 5 domains scores as well as four of the 10 domain scores of the Outcome Star Homelessness Scale). Results for BASIS-32 were clinically significant, all with moderate size effects. Results for the Outcome Star Homelessness Scale were also clinically significant ranging from small–medium (three domains) to large (one domain). Other improvements occurred in the areas of housing, employment and training, social inclusion and antisocial behaviour; some improvements were statistically significant, others were not. There was also a reduction in the use and cost of mental and general health services, as well as in the costs of housing. However, at the time of evaluation, mean levels of rental subsidies had yet to decrease substantially. Adverse events associated with tenancy, as well as antisocial behaviour, continued among a minority of participants.
These findings relating to improvement in behavioural function and distress of participants make an important contribution to the Housing First literature. This is because the previously unstudied Doorway model, where participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support, is shown in the present study to have beneficial effects.
The cost savings in both health service usage and housing type should be compared with other studies. A well-designed randomised control trial reported that offering housing and case management to homeless adults resulted in fewer hospital days and ED visits compared with usual care, although this involved a homeless population with chronic medical illnesses, not chronic mental illnesses.26 Savings (to government) in relation to homeless people with chronic mental illness are relatively few and, if anything, demonstrate additional costs to government.11,27–29 It is unclear to what extent the cost findings in the present study relate to the Doorway model, rather than the usual Housing First model.
It should be noted that these cost savings are to government, not participants. This is in contrast with the benefits of the Doorway program, which are considered above and should not be double counted. Further, these cost savings to government are potential, not actual. For benefit to people with SPMI and at risk of homelessness to occur, these newly available funds must be invested in relevant programs and replace economically inefficient programs where they currently exist.
The present study has some limitations, including the predominantly pre-post study design without a control group. Although it could be argued that it may have been possible to identify and possibly recruit a community-based population of homeless people with severe mental illnesses to form such a control group, this would have been very difficult. These include ethical difficulties if the study design involved withholding a program with benefits that are now established. They would also include controlling for the effects of potentially large numbers of confounding variables in two relatively small populations, either through randomisation, simple or propensity score matching. Therefore, it was necessary to accept an uncontrolled study design and to entertain the possibility that changes in outcome variables occurred for reasons other than entry to the Doorway program.
Other limitations of the study include a relatively small sample size of the Doorway participant population, making Type 2 errors possible. There were some missing paired data (pre-post) for some individual participants due principally to data gaps for the participants in the CMI/ODS. Some of these gaps may be related, directly or indirectly to early participant withdrawals from the program. More generally, however, numbers of participants withdrawing early from the program were related equally to ‘positive steps to recovery’ as to ‘ongoing recovery challenges’.
The Doorway model secured housing for this vulnerable group with additional benefits in client outcomes alongside reduced use and cost of health services. The model is eminently transferable to other countries. These findings, if confirmed in larger studies, should have widespread applicability internationally.
The authors thank Beth Fogerty, manager of the Doorway staff, other Doorway staff members and other clinical acute mental health service staff who gave generously of their time in the collection of de-identified data. The authors also thank David Carey, Kon Kon and Lachlan Rimes (Mental Health Branch) and Ying Chen (Victorian Data Linkages), all within the then Victorian Department of Health, for their assistance in the collection of de-identified data. The Doorway program was funded by the Department of Health, Victoria. The MI Fellowship funded the present evaluation study.
References Nous Group. Doorway – Summative Evaluation – November 2013. Melbourne: Nous Group; 2014. Available at: http://www.mifellowship.org/sites/default/files/Doorway%20-%20Summative%20Evaluation%20Report%20-%20FINAL%20-%20140205.pdf [verified 12 September 2016].
 Morgan V, Waterreus A, Jablensky A, Mackinnon A, McGrath JJ, Carr V, Bush R, Castle D, Cohen M, Harvey C, Galletly C, Stain HJ, Neil AL, McGorry P, Hocking B, Shah S, Saw S. People living with psychotic illness in 2010: the second Australian national survey of psychosis. Aust N Z J Psychiatry 2012; 46 735–52.
| People living with psychotic illness in 2010: the second Australian national survey of psychosis.CrossRef | 22696547PubMed |
 Morgan VA, McGrath JJ, Jablensky A, Badcock JC, Waterreus A, Bush R, Carr V, Castle D, Cohen M, Galletly C, Harvey C, Hocking B, McGorry P, Neil AL, Saw S, Shah S, Stain HJ, Mackinnon A. Psychosis prevalence and physical, metabolic and cognitive comorbidity. Data from the second Australian national survey of psychosis. Psychol Med 2014; 44 2163–76.
| Psychosis prevalence and physical, metabolic and cognitive comorbidity. Data from the second Australian national survey of psychosis.CrossRef | 1:STN:280:DC%2BC2c3ps1CjsA%3D%3D&md5=656cf923e4eac8e72af72f34404ee696CAS | 24365456PubMed |
 Drake RE, Whitley R. Recovery and severe mental illness: description and analysis. Can J Psychiatry 2014; 59 236–42.
| 25007276PubMed |
 Chamberlain C, MacKenzie D. Counting the Homeless 2001. Catalogue No. 2050.0. Canberra: Australian Bureau of Statistics; 2003.
 Tsemberis S, Gulcur L, Nakae M. Housing First, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. Am J Public Health 2004; 94 651–6.
| Housing First, consumer choice, and harm reduction for homeless individuals with a dual diagnosis.CrossRef | 15054020PubMed |
 Patterson M, Moniruzzaman A, Palepu A, Zabkiewicz D, Frankish CJ, Krausz M, Somers JM. Housing First improves subjective quality of life among homeless adults with mental illness: 12-month findings from a randomized controlled trial in Vancouver, British Columbia. Soc Psychiatry Psychiatr Epidemiol 2013; 48 1245–59.
| Housing First improves subjective quality of life among homeless adults with mental illness: 12-month findings from a randomized controlled trial in Vancouver, British Columbia.CrossRef | 23748928PubMed |
 Somers J, Rezansoff SN, Moniruzzaman A, Palepu A, Patterson M. Housing First reduces re-offending among formerly homeless adults with mental disorders: results of a randomized controlled trial. PLoS 2013; 8 e72946
| Housing First reduces re-offending among formerly homeless adults with mental disorders: results of a randomized controlled trial.CrossRef | 1:CAS:528:DC%2BC3sXhsVCrsrbK&md5=ac75e95127476dd793b00476ed4c0afcCAS |
 Kirst M, Zerger S, Misir V, Hwang S, Stergiopoulos V. The impact of a Housing First randomized controlled trial on substance use problems among homeless individuals with mental illness. Drug Alcohol Depend 2015; 146 24–9.
| The impact of a Housing First randomized controlled trial on substance use problems among homeless individuals with mental illness.CrossRef | 25465295PubMed |
 Somers JM, Moniruzzaman A, Palepu A. Changes in daily substance use among people experiencing homelessness and mental illness: 24-month outcomes following randomization to Housing First or usual care. Addiction 2015; 110 1605–14.
| Changes in daily substance use among people experiencing homelessness and mental illness: 24-month outcomes following randomization to Housing First or usual care.CrossRef | 26052657PubMed |
 Stergiopoulos V, Hwang SW, Gozdzik A, Nisenbaum R, Latimer E, Rabouin D, Adair CE, Bourque J, Connelly J, Frankish J, Katz LY, Mason K, Misir V, O’Brien K, Sareen J, Schütz CG, Singer A, Streiner DL, Vasiliadis HM, Goering PN, At Home/Chez Soi Investigators Effect of scattered-site housing using rent supplements and intensive case management on housing stability among homeless adults with mental illness: a randomized trial. JAMA 2015; 313 905–15.
| Effect of scattered-site housing using rent supplements and intensive case management on housing stability among homeless adults with mental illness: a randomized trial.CrossRef | 1:CAS:528:DC%2BC2MXkvFKnt70%3D&md5=a71e8c8b91f6ee6c742b7fe6349887e0CAS | 25734732PubMed |
 Bruce J, McDermott S, Ramia I, Bullen J, Fisher KR. Evaluation of the Housing and Accommodation Support Initiative (HASI) Final Report, for NSW Health and Housing NSW. Sydney: Social Policy Research Centre, UNSW; 2012. Available at: http://www.health.nsw.gov.au/mentalhealth/publications/Publications/pub-hasi-final-report.pdf [verified 12 September 2016].
 Johnson G, Kuehnle D, Parkinson S, Sesa S, Tseng Y-P. Sustaining exits from long-term homelessness: a randomised controlled trial examining the 48 month social outcomes from the Journey to Social Inclusion pilot program. Melbourne: Sacred Heart Mission; 2014. Available at: https://www.sacredheartmission.org/sites/default/files/publication-documents/j2si_sustaining_exits_from_longterm_homelessness_2015.pdf [verified 12 September 2016].
 Meehan T, Stedman T, Robertson S, Drake S, King R. Does supported accommodation improve the clinical and social outcomes for people with severe psychiatric disability? The Project 300 experience. Aust N Z J Psychiatry 2011; 45 586–92.
| Does supported accommodation improve the clinical and social outcomes for people with severe psychiatric disability? The Project 300 experience.CrossRef | 21534823PubMed |
 Eisen SV, Dill DL, Grob MC. Reliability and validity of a brief patient-report instrument for psychiatric outcome evaluation. Psychiatr Serv 1994; 45 242–7.
| Reliability and validity of a brief patient-report instrument for psychiatric outcome evaluation.CrossRef | 1:STN:280:DyaK2c3ksFWhtA%3D%3D&md5=70445e394a85cd8bbe7090b36e74971fCAS |
 Eisen SV, Grob MC, Klein AA. BASIS: the development of a self-report measure for psychiatric inpatient evaluation. Psychiatr Hosp 1986; 17 165–71.
| 1:STN:280:DyaL2s3jt12iug%3D%3D&md5=77c7b8ea1d201b385d69fabb9a52b40aCAS | 10282438PubMed |
 Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the Nation Outcome Scales (HoNOS). Br J Psychiatry 1998; 172 11–8.
| Health of the Nation Outcome Scales (HoNOS).CrossRef | 1:STN:280:DyaK1c7ps1ehtg%3D%3D&md5=f925fa53a6683e4df86fd98f4589e39dCAS | 9534825PubMed |
 Victorian Government Department of Health. Client Management Interface/Operational Data Store service contacts. 2015. Available at: https://www2.health.vic.gov.au/mental-health/research-and-reporting/reporting-requirements-for-clinical%20mental-health-services/service-contacts [verified 3 November 2015].
 Victorian Government Department of Health. Victorian health services policy and funding guidelines 2010–11: highlights. Melbourne: Department of Health; 2010.
 Independent Hospital Pricing Authority. National Hospital Cost Data Collection Australian Public hospitals cost report 2010–2011, round 15. Canberra: Commonwealth of Australia; 2013.
 Australian Institute of Health and Welfare (AIHW). Australian hospital statistics 2010–11. Health Services Series no. 43. Catalogue no. HSE 117. Canberra: AIHW; 2012.
 Mental Illness Fellowship. Proposal for housing pilot project – July 2009. Melbourne: Mental Illness Fellowship; 2009.
 Victorian Government Department of Health. Housing support and brokerage demonstration project. Melbourne: Victorian Department of Health; 2011.
 Steering Committee for the Review of Government Service Provision. Attachment Tables GA.19 (Indigenous CRA recipients, by payment type, 2012 (a), (b), (c)) and GA.22 (Median CRA entitlement, by location, 2012 (a), (b), (c)). In: Chapter 16 Housing and homelessness Sector overview Canberra: Productivity Commission; 2013. Available at: http://www.pc.gov.au/research/ongoing/report-on-government-services/2016/community-services/youth-justice/rogs-2016-volumef-chapter16.pdf [verified 12 September 2016].
 Zaretzky KP, Flatau P. The cost of homelessness and the net benefit of homelessness programs: a national study. Melbourne: Australian Housing and Urban Research Institute (AHURI); 2013.
 Sadowski L, Kee R, VanderWeele T, Buchanan D. Effect of a housing and case management program on emergency department visits and hospitalizations among chronically ill homeless adults: a randomized trial. JAMA 2009; 301 1771–8.
| Effect of a housing and case management program on emergency department visits and hospitalizations among chronically ill homeless adults: a randomized trial.CrossRef | 1:CAS:528:DC%2BD1MXls1Wjt78%3D&md5=a8007defd411cc3e96d2edc2df2f129eCAS | 19417194PubMed |
 Rosenheck R, Kasprow W, Frisman L, Liu-Mares W. Cost-effectiveness of supported housing for homeless persons with mental illness. Arch Gen Psychiatry 2003; 60 940–51.
| Cost-effectiveness of supported housing for homeless persons with mental illness.CrossRef | 12963676PubMed |
 Gilmer TP, Stefancic A, Ettner SL, Manning WG, Tsemberis S. Effect of full-service partnerships on homelessness, use and costs of mental health services, and quality of life among adults with serious mental illness. Arch Gen Psychiatry 2010; 67 645–52.
| Effect of full-service partnerships on homelessness, use and costs of mental health services, and quality of life among adults with serious mental illness.CrossRef | 20530014PubMed |
 Gilmer TP, Stefancic A, Tsemberis S, Ettner SL. Full-service partnerships among adults with serious mental illness in California: impact on utilization and costs. Psychiatr Serv 2014; 65 1120–5.
| Full-service partnerships among adults with serious mental illness in California: impact on utilization and costs.CrossRef | 24829104PubMed |
* *This paper is based on a technical report to the MI Fellowship by the Nous Group in February 2014 (http://www.mifellowship.org/sites/default/files/Doorway%20-%20Summative%20Evaluation%20Report%20-%20FINAL%20-%20140205.pdf, accessed 12 September 2016).1