THE STRUGGLE FOR TREATMENT INTEGRITY IN A "DIS-INTEGRATED" SERVICE DELIVERY SYSTEM
Chapter 8 in
A Book in the Wiley Series in Forensic Clinical Psychology
DRAFT 3: DO NOT CITE WITHOUT PERMISSION
The purpose of this chapter is to describe an innovative family preservation programme for delinquents, which exemplifies effective correctional treatment: Community Support Services of the St. Lawrence Youth Association in Ontario Canada. Over 7 years, the programme utilised the Teaching-Family Model's (Fixsen, & Blase, 1993) integrated clinical, administrative, evaluation and supervision systems to ensure quality and treatment integrity. These systems will be delineated, along with some of the challenges of implementing an integrated treatment within a fragmented children's' services delivery system. The intention is to do so from a "multilevel systems perspective" (Bernfeld, Blase and Fixsen, 1990), in which four levels of analyses are used to examine the delivery of human services: client, programme, agency, and societal. Finally, suggestions are made for contextual supports for innovative programmes, so as to foster their effectiveness, longevity and key role as catalysts for systemic change in children's services.
Hallmarks of Community Support Services include:
Taken as a whole, Community Support Services, like Alberta Family Support Services upon which it is modelled, offers "Contextual Therapy". Bernfeld et al. (1990) define this as "helping people learn to cope with their emotional and interactional issues in their own settings to maximise relevance, acquisition, and implementation and to minimise generalisation problems" (p.22). This report also provides more details on the treatment model, including an example of the treatment planning process, and case profiles. As well, it presents the extensive list of services provided by staff, that are documented in case files in order to track programme implementation.
The family-centred approach to treatment ensures that the treatment plan developed jointly with the family members "fits" their unique context. Staff also share their weekly goals and daily agendas with the family -- and, most importantly, are prepared to be flexible as families' needs change. The intensity, on-call support, and strength-oriented focus of the service help us build a solid relationship with family members and to facilitate behaviour change. Ultimately, our ability to work in the family's home depends on how we help the family accomplish its goals.
Bernfeld et al. (1995) describe the computerised Management Information System developed for Community Support Services. This system integrates the programme evaluation needs of the programme with supervision, management and administrative systems. This practical and cost-effective programme evaluation approach is integrated with routine service delivery. It generates automated monthly reports of the service's processes and outcomes. As an example of the former, time management data is collected on each staff, in order to track their different activities. These range from direct work with families (i.e. face-to face and on the phone), to indirect services (e.g. preparing for family visits, meetings, paperwork, travel, etc).
Outcome evaluation data collected at pre- and post-treatment, and at 3 month follow-up on 155 youths over 5 years is discussed in detail by Bernfeld et al. (1995) and will only be briefly summarised here. A home-based, family-centred treatment technology is evolving which seems to reduce the penetration of young offenders into the residential care system. While the results are encouraging, they should be interpreted with caution, given the lack of a comparison group. Bernfeld et al. (1990) discuss the principles that guided the development of the "in-house" programme evaluation model. These recognise the challenges in evaluating a flexible, strength-oriented programme, without compromising its implementation with families in crises. In depth analyses of these and other issues are provided by Pecora, Fraser, Nelson, McCrosky, and Meezan (1995).
Client benefits demonstrated by the programme include reliable or statistically significant reductions in youth behaviour problems, as measured by the Child Behavior Checklist (Acenbach, 1991) and parent-youth communication problems, on Robin and Foster's (1989) Conflict Behavior Questionnaire. Reliable improvements were also noted for the youths on the Social Skills Rating System (Gresham & Elliot, 1990).
Youth recidivism after a 15-month follow-up period (55%) compared favourably to the 67% rate reported by Hoge, Leschied, and Andrews (1993) based on 6 month follow-up for young offenders in open custody in Ontario. Moreover, the data indicated that Community Support Services achieved reliable reductions in the seriousness of offences, the total number of offences, the number of multiple convictions, as well as a longer interval between offences. Consumers (parents, youth, probation officers, etc.) were generally satisfied with the programme and provided detailed feedback about the programme's strengths and areas for improvement.
While 97% of the young offenders were predicted by probation officers at intake to the programme to require residential placements, only 42% were actually placed out-of-home in the 15 months of follow-up, averaging less than two months per placement. A cost analysis indicated that for every $1.00 spent on Community Support Services over 5 years, about $1.48 might have been saved in residential care costs. The data suggest that cost savings of about a quarter of a million dollars per year in residential dollars alone could be attributed to Community Support Services. It was in fact suggested that these modest savings are a conservative estimate of the benefits of the programme, for a number of reasons.
Hoge et al. (1993) reviewed Ontario-wide young offender programmes and found that they averaged a relatively low score of .29 on the sub-group of scales reflecting treatment on the CPAI (Gendreau and Andrews, 1996). The provincial average of .29 means that 29% of the 56 items were present across all Ontario programmes. Sector specific averages were: probation, .21; open custody, .26; secure custody, .29; and, the 9 community support teams (including Community Support Services), .51. In comparison, using the scoring guide provided by Leschied, Hoge and Andrews (1993), the Community Support Services programme scored above .70 on the scale -- meaning that more than 70% of the programme characteristics indicative of effective treatment were present.
The Teaching-Family Model is the "heart" of Community Support Services. From its original roots in a 1968 group home for delinquent adolescents called Achievement Place, the Teaching-Family Model has developed into an integrated service delivery system. Today there are over 300 [get current # ???] group homes serving over 2,500 children annually across the United States of America that serve not only delinquents but also abused, neglected, emotionally disturbed, autistic, and developmentally challenged children and young adults. In addition, the Teaching-Family Model has been recently adapted for youths in treatment foster care, independent living, and home-based services, serving over 1,200 annually. Over 100 [get current # ???] publications on the Model have researched its effectiveness and carefully evaluated its individual treatment components over the past 30 plus years. In fact, over $30M of the United States' government funding have supported the careful research and detailed development of the clinical, administrative, evaluation, and supervision systems which ensure that the Teaching-Family Model is delivered with integrity.
Appendix A details the Home-Based version of this Model, in terms of its goals as well as its treatment, programme, and treatment planning systems. All have been carefully integrated so as to support the systematic implementation of the Teaching-Family Model. Articles by Fixsen and Blase (1993; this volume) and Bernfeld, Blasé, and Fixsen (1990) present the conceptual and research basis for programme development and dissemination vis-à-vis the Teaching-Family Model.
The Teaching-Family Association oversees the quality assurance evaluations that hold organisations accountable for the programmes that utilise and disseminate this model. To become a member of this Association, a new organisation must be formally affiliated with an already certified site for 5 years, receive systematic help in developing Teaching-Family Programmes, and undergo a rigorous evaluation at the service or treatment level and the organisation level. The agency must meet all practice standards regarding the selection, training, supervision and evaluation of front-line staff, trainers, evaluators, and supervisors. The Association has established standardised procedures for how these functions are delivered in a certified organisation. A site must be re-certified annually.
In addition to maintaining records of staff supervision and training to assure the quality of the programmatic support, treatment implementation is ensured by the annual certification of individual staff. This involves a combination of consumer satisfaction data, in which staff must average a rating of 6 out of 7 on a scale that reflects satisfaction of consumers (parents, youths, case managers and others) with various aspects of service. As well, independent reviewers assess the actual in-home performance of staff and a 50-page report is completed which summarises both the consumer and on-site data and qualitative observations. Thus, the Teaching-Family Association provides a mechanism for assuring the consistency and quality of the implementation of the Teaching-Family Model internationally. Note that this is a non-profit organisation whose primary function is to disseminate the Model and ensure its planned, databased, evolution.
What is hopefully apparent from reading this Appendix, is that the Teaching-Family Model's vertically and horizontally integrated systems represent our "best practices" in how to translate knowledge about effective correctional treatment into practice with integrity, while ensuring quality assurance. The Model addresses the previously noted treatment integrity and systems factors in the literature by Gendreau, Andrews and others. As well, it represents the only systematic, published attempt in the area of Human Services to develop, disseminate, and evolve an integrated service delivery system.
Community Support Services worked towards site certification in the Teaching-Family Model from the early to the mid 1990s, when a change in the leadership of the provincial government drastically altered the funding arrangements that made this possible. The programme was about one year away from being certified, when the attempt had to be formally halted. Currently, Community Support Services is operating without formal implementation of all of the Teaching-Family Model systems. However, as it enters its second decade of service, it represents one of the oldest family preservation programmes in Canada, and the most experienced programme of its kind for young offenders.
In this section, some of the challenges of implementing an integrated treatment like Community Support Services within a fragmented children's services delivery system will be described. The intention is to do so from a "multilevel systems perspective", in which four levels of analyses are used to examine the delivery of human services: client, programme, organisation, and societal. Bernfeld, Blase and Fixsen (1990) originally delineated this perspective.
Clinical challenges occur at the interface of the treatment planning, programme and treatment systems reviewed in Appendix A. For example, given the intensive, home-based nature of the service, it is not surprising that the family's potential "resistance" represents a challenge to overcome. As reviewed by Serin and Kennedy (1997), internal treatment responsivity factors include client motivation, personality and cognitive deficits, while external factors reflect therapist, offender and setting characteristics. Treatment effectiveness depends on matching types of treatment and therapists to the types of clients.
Intensive supervision of staff in Community Support Services is critical to the "matching" process, so that the treatment "fits" the family. This is because the families of young offenders can sometimes be either difficult to engage, non-reinforcing to work with and/or reside in locations which are inaccessible or hazardous for staff. As well, the multiple problems of the target families and the intensity of the service impact directly on staff, who are immersed in the family often on a daily basis. Therefore, Specialists need support from a supervisor who is intimately familiar with their own professional issues, so that these do not interfere with the optimal delivery of services.
Also key to the "matching" of treatment to family is the expectation that there be a two-week long, in-home assessment of family issues before a contextually sensitive master treatment plan is designed. Finally, the weekly review of this plan allows for adjustments to the intervention, as the family's needs or outside circumstances evolve. This is especially important, as optimal matching must be a dynamic process. Moreover, in order for staff supervision to successfully impact on treatment effectiveness, it has to be intensive (occupy about 20% of staff time), multimodal (occur at individual, peer and group levels) and extensive. The latter reflects the range of supervisory activities (including regular field observations of staff interactions, and various meetings) and the various paperwork and time management systems which document programme implementation.
The above examples should not imply that issues at the client level operate in isolation to other levels to be discussed below. The fact that there are interactions between multiple levels of the service delivery system underscores the importance of this perspective on treatment integrity. For example, proposals by the labour union (at the Programme level), which had the potential to limit service intensity, accessibility and flexibility, had to be addressed to protect the integrity of the service. Also, shifting priorities of the juvenile justice system (at the Societal level) impact continually on the targeting of the service. Finally, the programme's preference for working at the "front end" of the juvenile justice system (in a family preservation mode) continually brought the programme in conflict with systemic pressures to serve youth currently in custodial care or those being released after spending long periods of time in out-of-home placements.
The family and community-centred nature of the service was at odds with the focus of the other programmes offered by the agency, which were residential. It often took the personal (and creative) intervention by the agency's Executive Director, Merice Walker Boswell, to solve any inter-programme misunderstandings or rivalries that undermined Community Support Services. Special meetings of all agency staff and the involvement of the Community Support Services supervisor in regular meetings with her colleagues helped share information and build informal interdepartmental "coalitions". Throughout, the Executive Director reminded employees of the superordinate goal that all agency staff shared-- to support, rehabilitate and advocate for children within the young offender system.
However, once again, the interactions between the multiple levels were critical to effective implementation of the service. For instance, at the Organisational level, the agency's structure facilitated efforts at the programme level to set up and evolve towards the Teaching-Family Model's treatment planning, programme and treatment systems. This is because the agency was relatively "loosely coupled", so that individual programmes like Community Support Services could set up independent systems to select, train and supervise staff, and evaluate employees individually and the programme as a whole. Moreover, the programme was autonomous enough to integrate these functions within its own operations and tie them directly to the Teaching-Family Model's protocols. This is quite unlike large bureaucratic organisations, like those in Corrections, which usually set up independent departments in these key areas, with differing mandates and procedures.
Moreover, because Community Support Services was positioned as a "leading edge" programme by the Executive Director (a key Organisational issue), it was supported in its efforts to pilot and refine new staff training and programme evaluation systems, which were later "exported" across the agency. The latter has been alluded to earlier when the development of a computerised Management Information system was discussed. Elements of this system were incorporated later in all other agency programmes.
The personnel selection process for Community Support Services was adapted from structured interviews used in the Teaching-Family Model. Applicant ratings on similar interviews have been shown to predict on-the-job performance (Maloney et al., 1983). Our interviews included an hour-long written package requiring, among other things, answers to a series of behavioural vignettes that assess the applicant's responses to hypothetical problem situations similar to those encountered on the job. The 90-minute interview itself assessed the applicant's answers to similar questions, along with performance on behavioural role plays. The latter assessed an applicant's teaching ability at baseline, after instructions in using the teaching interaction techniques of the Model, and after feedback. Three interviewers rated the applicant's performance in these and many other areas, and the comparative data was analysed in making hiring decisions.
The Executive Director of this small (50 employee) non-profit agency actively encouraged innovation. This is unlike the process in large organisations, like Corrections, as "the greatest strength of any bureaucracy is marshalling cogent arguments against change in the status quo" (p. 58, Zakheim, 1998). Due to her skilful efforts with staff and managers, the Board of Directors passed a motion committing the agency to the Teaching-Family Model. This supported the agency's work at achieving certification as a site. Resources were gathered to support this effort in a "low profile" manner, as the external zeitgeist shifted to a "punishing smarter" perspective. While Community Support Services was the focal point, the new secure custody programme was set up according to some key treatment aspects of this Model.
As well, the mentorship of the Community Support Services Director by the Executive Director (the present author) was essential in the development of his networking skills with other peers in the local and provincial juvenile justice system. These included the programme's key consumers--the local Supervisor of Probation and senior policymakers in the Provincial government. In fact, the programme's interactions with the referring agency (Probation) were critical to efforts to operate in a family preservation mode, and serve youths with appropriate risk levels. As Bernfeld, Fixsen and Blase (1990) noted, a Programme Director's personal relationship with key administrators and policymakers continues to be critical to programme survival and dissemination.
Finally, the management philosophy of the Executive Director was compatible with the Teaching-Family Model's view of administration as integral to effective treatment services. From this perspective, "there is no such thing as an administrative decision--every decision is a clinical decision" (D. L. Fixsen. Personal communication, December 12, 1988). This requires flexibility in the application of everything from accounting, budgeting, and human resources systems, so as to support short-term, community based, crisis services. For example, there was a need to develop a unique system to calculate "flex time" while ensuring staff accountability. As well, given the highly mobile and decentralised nature of the service and the fact that staff usually worked out of their own homes, new policies governing staff travel had to be established. These are just a few of the many procedures that needed to be developed or creatively interpreted to accommodate programme implementation.
Bernfeld, Cousins-Brame, and Knox (1995) comprehensively delineate the challenges of implementing an integrated treatment like Community Support Services within a fragmented children's services delivery system. The paper also describes a "sister" programme for child welfare clients, operated in collaboration with another agency. The authors demonstrate how the different structural and systemic supports for the other programme, were essential to mitigating the impact of challenges to integrity and enhancing its implementation and effectiveness. The major types of issues identified by Bernfeld, Cousins-Brame, and Knox are listed below, each with an example:
The above is a brief overview of some the challenges discussed by Bernfeld, Cousins-Brame, and Knox to implementing an integrated treatment like Community Support Services within a fragmented children's services delivery system. It is contended that these and other issues are critical to the integrity of any human service delivery programme. As well, a programme's ultimate effectiveness is a function of its ability to surmount "on the ground" challenges to its implementation--no matter how strong its empirical base or its success in limited demonstration projects.
An eclectic literature exists on the diffusion of innovations and the management of change in large-scale Human Service systems, some of which has been reviewed by Bernfeld, Blase and Fixsen (1990), Bernfeld, Cousins-Brame, and Knox (1995), and in the Introduction to this volume. This literature teaches us that the long-term survival of innovative human service programmes requires a unique confluence of supportive factors at the programme, organisational and societal levels. Some of these factors were already delineated in the previous section, which detailed the organisational structure of Community Support Services' host agency. Moreover, specific suggestions can be made for contextual supports for innovative programmes like Community Support Services. These include a detailed plan at the senior levels of government which:
Overall, in order to systematically disseminate innovation, government needs to develop a comprehensive, long-term implementation plan. For example, if such a plan was in place when Community Support Services began, it would have not have allowed the programme's referrals to be managed by the Probation Service. Problems arising from this situation discussed earlier were a result of "the fox being in charge of the Hen house". Instead, the referral structure should have resembled the system for Alberta Family Support Services.
There, an independent government committee at the Regional level made referrals if and only if they had already decided to remove youths from their homes. This committee was accountable to senior levels of government for reducing reliance on residential services, and could only refer youths to the family preservation programme if it had an opening. This structure preserved the autonomy and integrity of the programme and ensured a steady supply of referrals. In Alberta, the agency making referrals had a vested interest in the success of the innovative service and was not in competition with it. As discussed previously, the state of affairs was quite different for Community Support Services, which resulted in an ongoing struggle for integrity. This is just one way a comprehensive implementation plan by government could have worked .
It is contended that contextual supports are essential to innovative programmes like Community Support Services, so as to foster their effectiveness, longevity and key role as catalysts for systemic change in children's services.
Achenbach, T.M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 Profile. Burlington, Vermont: University of Vermont, Department of Psychiatry.
Andrews, D.A. (1995). The psychology of criminal conduct and effective treatment. In J. McGuire (Ed.), What works: Reducing re-offending (pp. 35-62). Chichester, UK: John Wiley & Sons.
Andrews, D.A., Gordon, D.A., Hill, J., Kurkowski, K.P., & Hoge, R.D. (1993). Program integrity, methodology, and treatment characteristics: A meta-analysis of effects of family intervention with young offenders. Manuscript submitted for publication, Carleton University.
Andrews, D.A., Leschied, A.W., & Hoge, R.D. (1992). Review of the profile, classification, and treatment literature with young offenders: A social-psychological approach. Ministry of Community Social Services, Toronto, Ontario.
Bernfeld, G.A., Blase, K.A., & Fixsen, D.L. (1990) Towards a unified perspective on human service delivery systems: Application of the Teaching-Family Model. In R.J. McMahon & R. DeV. Peters (Eds.), Behavior disorders of adolescents: Research, intervention and policy in clinical and school settings (pp. 191-205). N.Y.: Plenum.
Bernfeld, G., Bonnell, W., Cousins-Brame, M. L., Kippen, J., Knox, K., Kyte, D., Landon, B., Simmons, C., & Wright, P. (1995). Community Support Services: Annual Report. Kingston, Ontario: St. Lawrence Youth Association. [Reprints of the entire (90+pgs.) report: $10.00 to cover copying & postage]
Bernfeld, G., Cousins, M., Daniels, K., Hall, P., Knox, K., McNeil, H., & Morrison, W. (1990) Community Support Services: Annual Report. Kingston, Ontario: St. Lawrence Youth Association. [Reprints of the text portion of this report (up to pg. 32): $5.00 to cover copying & postage].
Bernfeld, G.A., Cousins-Brame, M.L., & Knox, K. (1995, May). Contextual challenges to family-centred services: Can integrated treatment operate in a "dis-integrated" service system? Workshop at the Growing '95 conference, Toronto.
Christensen, A., & Jacobson, N.S. Who (or what) can do psychotherapy: The status and challenge of nonprofessional therapies. Psychological Science, 5, 8-14.
Fixsen, D. L., & Blase, K.A. (1993) Creating new realities: Program development & dissemination. Journal of Applied Behavior Analysis, 26, 597-615.
Fixsen, D.L., Olivier, K.A., & Blase, K.A. (1990). Home-based, family-centred treatment for children. Unpublished manuscript. Hull Child & Family Services.
Gendreau, P. (1996). The principles of effective intervention with offenders. In A. T. Harland (Ed.), Choosing correctional options that work: Defining the demand and evaluating the supply (pp117-130). Thousand Oaks CA: Sage.
Gendreau, P., & Andrews, D.A. (1996). Correctional program assessment inventory (CPAI) (6th ed.) Saint John, NB: University of New Brunswick.
Gendreau, P., & Goggin, C. (1997). Correctional treatment: Accomplishments and realities. In P. Van Voorhis, D. Lester, & M. Braswell (Eds.), Correctional counselling (3rd ed.) (pp. XX- XX). Cincinnati Ohio: Anderson.
Gresham, F.M., & Elliott, S.N. (1990). Manual for the Social Skills Rating System. Toronto, Ontario: Psycan.
Henggeler, S.W. (1996). Multisystemic therapy using home-based services: A clinically effective and cost effective strategy for treating serious antisocial behavior in youth. Unpublished manuscript, Medical University of South Carolina, Charleston.
Hoge, R.D., Leschied, A.W., & Andrews, D.A. (1993). An investigation of young offender services in the Province of Ontario: A report of the repeat offender project. Ministry of Community and Social Services, Toronto, Ontario.
Leschied, A.W., Hoge, R.D., & Andrews, D.A. (1993). Evaluation of the Alternative to Custody Programmes in Ontario's Southwest Region. Ministry of Community and Social Services, Toronto, Ontario.
Levesque, R.J.R. (1996). Is there still a place for violent youth in juvenile justice? Aggression & Violent Behaviour, 1, 69-79.
Losel, F. (1995). The efficacy of correctional treatment: A review and synthesis of meta- evaluations. In J. McGuire (Ed.), What works: Reducing re-offending (pp. 79-114). Chichester, UK: John Wiley & Sons.
Maloney, D.M., Warfel, D.J., Blase, K.A., Timbers, G.D., Fixsen, D.L., & Phillips, E.L. (1983). A method for validating employment interviews for residential child care workers. Residential Group Care & Treatment, 1, 37-50.
Olivier, K.A., Oostenbrink, A., Benoit, G., Blase, K.A., & Fixsen, D.L. (1992). Alberta Family Support Services: Annual Report. Hull Child & Family Services, Calgary, Alberta.
Patterson, G.R., Reid, G.D., & Dishion, T.J. (1993). A social interactional approach to family intervention: Antisocial boys. Vol. 4. Eugene, OR: Castalia.
Pecora, P.J., Fraser, M.W., Nelson,K.E., McCroskey, J. & Meezan, W. (1995). Evaluating family-based services. N.Y.: Aldine de Gruyter
Robin, A.L., & Foster, S.L. (1989). Negotiating parent-adolescent conflict: A behavioral-family systems approach. N.Y.: Guilford. Serin, R. & Kennedy, S. (1997). Treatment readiness and responsivity: Contributing to effective correctional programming. Ottawa, Ontario: Correctional Services of Canada.
Tate, D.C., Reppucci, N.D., & Mulvey, E.P. (1995). Violent juvenile delinquents: Treatment effectiveness and implications for further action. American Psychologist, 50, 777-781. Whittaker, J.K., Kinney, J., Tracy, E.N., & Booth, C. (1990). Reaching high-risk families: Intensive family preservation in human services. N.Y.: Aldine de Gruyter.
Zakheim, D. S. (1998) Time to shake up America's peace team. The Jerusalem Report, 8, 58.
** REPRINT REQUESTS FOR CSS ANNUAL REPORTS. Please write: Ms. Mary Lynn Cousins- Brame Acting Program Director, Community Support Services, St. Lawrence Youth Association, 845 Division St. Kingston ON K7K 4C4 (613) 542-9634 Please make cheque payable to The St. Lawrence Youth Association
NOTE: The Figures in this Appendix (except for the last one on Treatment Planning) were adapted from those developed by Fixsen & Blase during their consultations with Community Support Services. They are also based on the 1994 TFA Standards for Home-Based Services. TFA can be contacted at: 910 Charles St., Fredericksburg, Virginia 22401; phone (540) 370-4439; or visit their web page at http://www.teaching-family.org/.