• Subject Areas
   – Energy & the Environment
   – First Nations
   – Health
   – Justice
   – Labour Market & Education
   – Regulatory Impacts
   – Research & Development
   – Safety
• Techniques
• Publications
• HBHC Evaluation

Human Resources Development Canada

The Service and Outcome Measurement System (SOMS) contains models that generate predicted effects of a range of interventions. At the time our firm undertook the development of the models, they had some weaknesses: the precision of estimating predicted effects was not known; single interventions tended to dominate the models' recommendations; and the models sometimes recommended interventions that were not plausible for certain clients. Our firm was hired to develop the models further, based on new, more comprehensive data, with the aim of overcoming the above weaknesses.

The work entailed considerable effort in developing the new data base into the form of regression variables suitable for use in the models. It also became feasible to define new variables for interventions that had previously occurred too infrequently to be included. We examined patterns of interventions to find whether different interventions occur in clusters. The models addressed four outcomes: amount of U.I. benefit received; occurrence of a job; number of weeks employed; and income from employment. Explanatory variables included several personal and environmental characteristics. Interventions (25) were measured as either counts of past occurrences or total weeks (for those with longer durations). In addition, the squares of these variables were included to account for some non-linearity in the relationship. A third kind of variable measured the number of weeks since the last occurrence of each intervention. Finally the basic set of intervention variables and the personal and environmental characteristics were cross-multiplied to create interaction variables, thereby allowing the model to estimate differential effects of the interventions according to client circumstances.

Effects of interventions were estimated and summarised in tables. The optimal assignment for each client was selected for each of the four outcome variables, based on the estimated effect of the intervention, disregarding cost and always had a positive effect. We examined consistency of the benefit of each intervention across the four outcome measures. Finally we made several recommendations for further research: increase the sample size; examine the effect of missing data for certain variables; analyse the distributions of occurrences and durations of interventions over time and by region and observable characteristics of clients; expand the models to include variables to measure non-linear effects of the time since the intervention; generate graphical displays of non-linear relationships; compute confidence intervals for estimated effects.