Summative Evaluation of the Women's Program - 2011-2012

Table of Contents

List of Acronyms

List of Tables

Executive Summary

1. Introduction and Description of the Women's Program

1.1 Program Description

2. Scope of the Evaluation and Methodological Approach

2.1 Evaluation Objectives and Issues
2.2 Lines of Evidence
2.3 Limitations

3. Findings

3.1 Relevance
3.2 Program Performance: Achievement of Intended Outcomes
3.3 Design and Delivery
3.4 Efficiency and Economy

4. Conclusions

4.1 Relevance
4.2 Performance

5. Recommendations

Appendix A: Evaluation Framework

Appendix B: Bibliography

Under Separate Cover

Appendices C-G: Data Collection Instruments
Appendix H: Case Study Summaries

List of Acronyms

AAND
Aboriginal Affairs and Northern Development
FaHCSIA
Department of Families, Housing, Community Services and Indigenous Affairs (Australia)
GBA
Gender-based Analysis
NCPC
National Crime Prevention Centre
NGO
Non-governmental organization
PAA
Program Activity Architecture
RBM
Results-based Management
SWC
Status of Women Canada
WCF
Women's Community Fund
WEF
World Economic Forum
WP
Women's Program
WPF
Women's Partnership Fund

List of Tables

Executive Summary

Overview

In support of the federal agenda for advancing the status of women, the Women's Program (WP) provides funding and professional assistance to organizations to carry out projects that advance equality for women across Canada through improvement of their economic and social conditions and their participation in democratic life. Status of Women Canada (SWC), the responsible agency for the program, has commissioned a summative evaluation of the WP.

Founded in 1973, the WP is a permanent program, with Terms and Conditions in effect for five years. The Terms and Conditions that are the focus of this evaluation came into effect in 2006-2007. The WP's current mandate is to advance equality for women across Canada through improvement of their economic and social conditions and their participation in democratic life. Its objective is to achieve the full participation of women in the economic, social and democratic life of Canada.

The WP provides both funding and technical assistance to funded organizations at the local, regional and national levels. The program supports a range of community-based projects that address women's economic security and prosperity, violence against women, and leadership and democratic participation. The program is delivered regionally through four offices, and an office that funds national projects. Program funding is $19 million annually. The program has two funding components:

Scope and Methodology

The evaluation covers the Terms and Conditions for the program in place between 2006-07 and 2010-11. Evaluation issues related to relevance and program performance were addressed, as well as design and delivery questions.

Multiple lines of evidence were used to increase the reliability and robustness of the evaluation of the WP. The evaluation included: a review of documentation, program data and literature; file review; key informant interviews; survey of program applicants; case studies; comparative program analysis; and cost-effectiveness analysis. Each of these methods is described below.

Document Review: Secondary sources were reviewed to develop a detailed profile of the WP and to contribute to addressing several of the evaluation questions including relevance, design and delivery, and program performance. A document review template was developed to summarize findings in the documents pertaining to each of the evaluation questions.

Program Data Review: SWC provided EKOS with a list of all projects approved for funding since September 2006 up to 2009-10 (n=398) for analysis. Coding was undertaken to transform some of the data for the purposes of analysis.

Literature Review: Recent literature was reviewed to contribute to the analysis of program relevance (need) and alternative delivery models. Sources included relevant literature and research on gender equality including research studies, position and policy papers, and domestic and international publications. Documentation and sources provided by SWC were complemented by an Internet search of academic, NGO research/ policy/advocacy organizations and government documents.

Review of Project Files: A stratified random sample of project files from SWC headquarters and regional offices was reviewed. In total, 40 files were reviewed, roughly 10 per cent of the projects funded during the period under study. The selection of files focused on closed projects and further considered the funding component, region and year of the agreement. An electronic data input form was used to capture file information in a consistent manner.

Key Informant Interviews: In total, 16 key informant interviews were completed with program managers and staff, partners, and external experts.

Online Survey of Program Applicants: The evaluation included an online survey of organizations that applied to the WP for funding during the period under study. The survey included a census of organizations that were funded by the WP between 2006-07 and 2009-10. For non-funded applicants, a census of eligible organizations from the third and fourth funding calls was surveyed.

Case Studies: Seven case studies of funded projects were conducted to illustrate impacts and sustainability of different representative approaches within the WP's funding activity. Each of the case studies was based on a review of the project file, a review of other project or organizational documentation; and key informant interviews with project proponents, partners and key stakeholders (the precise number of key informants was determined on a case-by-case basis).

Comparative Program Analysis: A review of program materials from federal grants and contribution programs with similar characteristics to the WP was conducted. Six programs were identified. For each of these programs, publicly available information was reviewed (e.g., program web-site, departmental planning and performance documents, program evaluation/audits) to compare programming aspects such as proposal solicitation and eligibility criteria, and to identify areas where the WP complements and/or overlaps with other federal programs. Interviews were also conducted with program staff for the comparative programs.

Cost-effectiveness Analysis: An analysis of the efficiency of the program was undertaken, including an examination of utilization of resources (including leveraged resources) and the administrative to program dollar ratio of WP (annually). This analysis was supplemented by qualitative feedback obtained in the key informant interviews about efficiency and economy (and where there are opportunities for efficiency gains).

Findings

Relevance

Performance

Effectiveness
Design and Delivery
Efficiency and Economy

Recommendations

Recommendations to improve the Women's Program are based on the findings and conclusions of the evaluation. The following recommendations focus on program improvements in program performance and program design and delivery.

Program Performance

1) In order to improve the overall effectiveness of the program in achieving its ultimate intended outcome of equality for women, the program should identify any necessary program design changes to fund activities that the evaluation found to have greater potential reach and sustainable impact. These evaluation results and the program's own internal analyses indicate that funded projects that include components that foster change at the level of policy or institutional practices have greater reach and sustainability. Funded projects that create resources or tools, or leverage proponents' existing regional or national networks/membership also have advantages in terms of reach and sustainability. While projects led by women's groups were also associated with greater reach and sustainability, any program design changes to the eligibility of organizations must be balanced against the benefits of introducing gender equality programming into mainstream organizations.

Design and Delivery

2) The program should clarify its theory of change from activities and outputs through to immediate, intermediate and ultimate outcomes. The program's current logic model does not adequately capture all activities carried out by the program, and the results chain does not reflect the program's evolving understanding of social change – i.e., based on a multi-dimensional approach that supports social change at multiple levels, including the participation of individual women, capacity and connections among organizations or networks of organizations, and change within communities and at the level of policy.

3) The WP should continue to initiate and strengthen relationship-building with provinces and other federal departments and agencies to seek input on program funding priorities. WP project activity in areas such as immigration and skills development to remove barriers to women's participation is diverting finite program funding to areas where other federal or provincial programs are in place and could address the participation of women. While other government departments are aware of the WP and have a favourable impression of the program, efforts moving forward would benefit from consultations with federal partners to ensure that WP's limited funding is deployed in areas where there is an absence of other levers to address barriers to women's participation.

4) The program should review deployment of its internal organizational resources to address significant capacity issues within the program, particularly a thinly-spread regional delivery structure. The program should continue its exploration of alternative delivery models or innovations with a view to addressing particular constraints in underserved regions (the West where applicant satisfaction levels tend to be lower and the North where there are few projects funded) (e.g., leveraging of provincial capacities, utilization of service delivery nodes in other federal departments, use of third party delivery for selected program focus areas).

5) The program should further leverage the project-level investments that it makes with enhanced mechanisms for knowledge building/sharing with and among stakeholders. Encouraging projects to share their results and embedding opportunities for dissemination and exchange at the program level would enhance the value of products developed by funded projects and keep the program funding focused on innovation. To support this:

  1. Consideration should be given to adjusting the current funding envelope (e.g., O&M dollars made available) to permit activities to support dissemination/exchange activities, including via web-site postings, workshops/meetings or other mechanisms.

  2. Review the program's Terms and Conditions to remove any restrictive eligibility criteria related to organization eligibility (e.g., academic organizations, cooperatives) and activity eligibility that inhibit undertaking knowledge building/sharing activities.

1. Introduction and Description of the Women's Program

In support of the federal agenda for advancing the status of women, the Women's Program (WP) provides funding and professional assistance to organizations to carry out projects that advance equality for women across Canada through improvement of their economic and social conditions and their participation in democratic life. Status of Women Canada (SWC), the responsible agency for the program, has commissioned a summative evaluation of the WP. The purpose of this Final Report is to present the findings of the evaluation of the WP.

The report is organized into the following sections: Chapter One provides a description of the WP; Chapter Two discusses the objectives of the evaluation and the methodological approach; Chapter Three presents the evaluation's findings related to relevance, design and delivery, and performance (effectiveness, efficiency and economy); conclusions are included in Chapter Four; and recommendations are presented in Chapter Five.

1.1 Program Description

a) Context

Since it was established in 1971, the Office of the Coordinator, Status of Women, (to be referred to as the "Agency") has played an important role in advancing equality for women in Canada. The Agency's strategic outcome is "Equality for women and their full participation in the economic, social and democratic life of Canada." SWC focuses its efforts on three key priority areas: ending violence against women and girls, improving women's economic security and prosperity, and encouraging women and girls in leadership and decision-making roles.

The Agency pursues its intended strategic outcome by way of two key program activities. First, SWC fosters inclusive and effective policies that advance gender equality and address those issues that affect women by enhancing the capacity of federal departments/agencies to integrate gender-based analysis (GBA) into all aspects of their work. Second, SWC seeks to ensure that women are equipped with the information, knowledge, skills and tools to be able to identify and address barriers that impede their full participation in society. The WP falls within the second program activity.

b) Overview of the Women's Program

Founded in 1973, the WP is a grants and contributions program that was created in response to a recommendation of the 1970 Royal Commission on the Status of Women.Footnote 1 Originally administered by the former Department of the Secretary of State, the WP has been housed within SWC since 1995. The WP is a permanent program, with renewal of its Terms and Conditions occurring every five years. The Terms and Conditions that are the focus of this evaluation came into effect in 2006-07. The WP's current mandate is to advance equality for women across Canada through improvement of their economic and social conditions and their participation in democratic life. Its objective is to achieve the full participation of women in the economic, social and democratic life of Canada.

The WP provides both funding and technical assistance to funded organizations at the local, regional and national levels. The program has two funding components:

Funded projects vary considerably in nature and scope and are undertaken by a variety of different proponents, address a range of issues and apply a variety of strategies to achieve their objectives. As noted above, the projects are designed to support women in their efforts to increase their awareness, knowledge and skills to identify and address barriers to their participation in society. The program does not provide core funding to organizations, nor does it cover ongoing administration costs. The maximum financial support that can be provided to an eligible recipient is $500,000 per fiscal year. The majority of the projects are funded through grants. Project duration can be up to a maximum of 36 months.

Those eligible for funding from the WP are incorporated not-for-profit and for-profit Canadian organizations whose mandates are consistent with the objective of the WP. The WP does not provide funding to individuals, unions, governments or government agencies, educational institutions, cooperatives or organizations mandated by governments (e.g., school boards, hospitals).

In 2006-07, applications for funding to WP were received through a process of continuous intake of proposals. This shifted to periodic general calls for proposals for the WCF in 2007-2008 (there were two calls during this year and annual calls in subsequent years). The WPF continued to be delivered through the continuous intake model. Note that in 2010-2011, the program revived the continuous intake model for the WCF.

The WP sets priorities for funding annually.Footnote 2 For some calls for proposals, the Program has also identified specific target populations in addition to funding priorities. In 2009-2010 and 2010-11, the Program's priority issues included: 1) ending violence against women and girls; 2) improving women's and girls' economic security and prosperity; and 3) encouraging women and girls in leadership and decision-making roles.

In 2007-08, the target populations were: Aboriginal women; immigrant women; visible minority women; and senior women. From 2008-2009 to 2009-2010, the target population was all women and girls. In 2010-11, for projects addressing the issue of violence against women and girls, the target populations were: 1) women and girls living in remote areas; 2) Aboriginal women and girls; and 3) immigrant women and girls.

c) Program Activity

During the first four years of the current program cycle, 398 projects were funded. Projects funded in 2006-2007 were granted under the Women's Program (WP) – prior to the creation of the WCF and WPF. Projects between 2007-2008 and 2009-2010 were funded primarily by the WCF (335 projects). There were 31 projects funded under the WPF. The average agreement value during this time was approximately $180,000.

Table 1.1: Number of Funded Projects by Year
  2006-2007 2007-2008 2008-2009 2009-2010 Total
WP 32 - - - 32
WCF - 166 91 78 335
WPF - 14 12 5 31
Total 32 180 103 83 398

With respect to the program's key priority issues, 45 per cent of projects addressed violence against women and girls, 40 per cent of projects were funded under the economic security priority and leadership was addressed by 10 per cent of projects (the remaining 5 per cent of projects addressed the WP priority area of health (2006-2007) that was named as a priority area for one of the Calls. Nineteen per cent of WCF funded projects were targeted to immigrant and refugee women, while thirteen per cent of projects were targeted to Aboriginal women. One in ten funded projects each targeted women who have experienced abuse, marginalized women or young women/girls.

d) Structure and Governance

Overall accountability for Program implementation resides with the SWC Coordinator. Reporting to the SWC Coordinator, the Director General of the WP is responsible for the delivery, performance and accountability of the Program.

Delivery of the WP is decentralized. Since 2007, the program has been delivered via a national office and four regional offices: Atlantic; Quebec/Nunavut; Ontario; and West/NWT/Yukon. Proposals for projects that are local, regional or provincial/territorial in nature are assessed for eligibility by the regional offices, while proposals for projects that are national in scope are assessed by the national office. All eligible project proposals are then considered against a comprehensive list of factors included in the project assessment guidelines. Final approval of applications rests with the Minister of Status of Women Canada. Of the 398 projects funded between 2006-2007 and 2009-2010, one-third were funded in the West/NWT/Yukon (though only a handful of these were led by organizations in the North). Approximately one-fifth were funded in Ontario and Quebec/Nunavut, respectively. Just over one-sixth were funded in the Atlantic region, and just under one in ten projects were national in scope.

e) Program Resources

Since 2008-09, the salary and operating costs of the program have remained stable at about $2.7 to $2.9 million per year. The WP annual budget is $18.95 million (grants and contributions), however the annual budget fluctuated during this period between approximately $17 and $24 million annually, due to various reprofiling exercises.

Table 1.2: WP Total Annual Budget
In $ millions 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
Salary and operating costs 2.0 2.9 2.9 2.8 2.7
Grants 13.45 17.55 14.75 14.75 14.75
Contributions 3.7 6.6 5.2 5.2 4.2
Total 19.15 27.05 22.85 22.75 21.65
Source: 2007-08 to 2010-11 Public Accounts and 2011-12 Main Estimates

f) Logic Model

A logic model is a visual representation of a program that identifies the linkages between a program's activities and the achievement of its outcomes. The Women's Program's logic model, presented in Figure 1, provides a graphical depiction of how the activities and outputs of the program relate to immediate, intermediate, and ultimate outcomes. Specifically, the intended immediate outcomes of the Program include:

The intermediate outcome for the Program is increased participation of women in their communities. The ultimate outcome of the program is equality for women and their full participation in the economic, social and democratic life of Canadian society.

Logic Model - Women's Program

Logic Model - Women's Program
Long description

2. Scope of the Evaluation and Methodological Approach

2.1 Evaluation Objectives and Issues

In the 2010–2011 Agency Report on Plans and Priorities, completing a summative evaluation of the Women's Program was identified as one of three management priorities for the SWC. The evaluation assesses the relevance of the program, program performance, as well as whether WP is delivered in an efficient and economical manner. Potential program improvements or alternative models that could enhance the achievement of outcomes are also examined. Furthermore, since the last evaluation of the program carried out in 2005, the WP has undergone significant changes and this evaluation will seek to assess whether the Program, in its current form, is meeting policy objectives. The evaluation covers the time period between 2006-07 and 2010-11. The Evaluation Framework is included in Appendix A.

2.2 Lines of Evidence

Multiple lines of evidence were used to increase the reliability and robustness of the evaluation of the WP. The evaluation included: a review of documentation, program data and literature; file review; key informant interviews; survey of program applicants; case studies; comparative program analysis; and cost-effectiveness analysis. Each of these methods is described below.

a) Document, Program Data and Literature Review

Document Review: Secondary sources were reviewed to develop a detailed profile of the WP and to contribute to addressing several of the evaluation questions including relevance, design and delivery, and program performance. SWC provided documents such as: program Terms and Conditions, program management documents (e.g., calls for proposals, application guides, and proposal assessment tools), internal analyses and performance reports and corporate/policy documents). A document review template was developed to summarize findings in the documents pertaining to each of the evaluation questions.

Program Data Review: SWC provided EKOS with a list of all projects approved for funding since September 2006 up to 2009-10. The list contained 398 funded projects (once amendments and supplements were removed). Coding was undertaken to transform some of the data for the purposes of analysis. Contents of the data file included: administrative information (e.g., call number, fiscal year, agreement number); proponent profile information (e.g., type of organization); financial data (e.g., funding component, amount of funding); and project information (e.g., priority area, project activities).

Literature Review: Recent literature was reviewed to contribute to the analysis of program relevance (need) and alternative delivery models. Sources included relevant literature and research on gender equality including research studies, position and policy papers, and domestic and international publications. Documentation and sources provided by SWC were complemented by an Internet search of academic, NGO research/ policy/advocacy organizations and government documents.

b) Review of Project Files

A stratified random sample of project files from SWC headquarters and regional offices was reviewed. In total, 40 files were reviewed, roughly 10 per cent of the projects funded during the period under study. Projects were sampled from the program's "master list" of funded projects, with a focus on completed projects (78 per cent) and active multi-year projects (22 per cent) in order to best document project outcomes. The selection of files further considered the funding component, region and year of the agreement. Selected files were profiled to ensure representativeness with the population (according to variables such as issue focus, dollar value of the agreement, primary target group). A profile of the characteristics of the files that were reviewed is included in Appendix C (under separate cover).

The project files were reviewed to obtain information on program design and delivery elements such as project profile data, project outputs and outcomes, leveraging of resources and sustainability. An electronic data input form, presented in Appendix D under separate cover, was used to capture file information in a consistent manner.

c) Key Informant Interviews

In total, 16 key informant interviews were completed with program managers and staff (6), partners (7) and external experts (3). The master key informant interview guide is included in Appendix E (under separate cover). This guide was further customized for each respondent group. All key informants were notified by SWC of the evaluation and invited to participate in the study. Key informants were then contacted by the consultant to schedule an interview and provided a copy of the interview guide prior to the interview. While most interviews were conducted by telephone, some interviews with respondents in the National Capital Region were conducted in-person; all interviews were conducted in the key informant's official language of choice.

d) Online Survey of Program Applicants

The evaluation included an online survey of organizations that applied to the WP for funding during the period under study. The survey included a census of organizations that were funded by the WP between 2006-07 and 2009-10 (n=335). For non-funded applicants, a census of eligible organizations that have never been funded, or not been funded by the WP since 2006, was surveyed (n=131), focusing on the third and fourth calls for proposals. Non-funded applicants that were deemed ineligible or who submitted proposals outside the funding parameters of the WP were screened out of the sample. Some organizations had submitted multiple applications for funding over several proposal calls. These duplicates were removed and the most recent application for funding retained. A portion of the sample – 11 per cent – had outdated or invalid contact information that could not be updated.

The survey instrument was designed for administration with both the funded and non-funded groups. Skip logic and branching routed respondents to the modules of the survey appropriate to their level of experience with the program. The survey obtained the perspective of applicants on evaluation issues related to relevance, design and project-level outcomes. The survey instrument is included in Appendix F under separate cover.

The survey duration was from June 6 to July 21, 2011. In total, 150 funded applicants and 51 unfunded applicants completed the survey. The response rate to the survey was 49 per cent. A comparison of the survey sample and population of funded applicants on key indicators (e.g., region, year of funding) indicated representativeness of the sample.

e) Case Studies

Seven case studies of projects funded by the WP were conducted to obtain detailed qualitative and quantitative data on the extent to which funded projects achieved their expected outcomes and contributed to the intended immediate and intermediate outcomes of the WP. The case studies bring together and analyze data collected at the project level.

Each of the case studies was based on a review of the project file, a review of other project or organizational documentation; and key informant interviews with project proponents, partners and key stakeholders (the precise number of key informants determined on a case-by-case basis). The case study interview guides are provided, under separate cover, in Appendix G.

WP management, in consultation with evaluation committee members and the consultant, selected projects to illustrate different approaches that are represented in WP's funding activity (as well as region and pillar of activity). Case study summaries are included in Appendix H under separate cover.

f) Comparative Program Analysis

In addition to Agency and program-related documents, the document review component also included a review of program materials from federal grants and contribution programs with similar characteristics to the WP. Six programs were identified.Footnote 3 For each of these programs, publicly available information was reviewed (e.g., program web-site, departmental planning and performance documents, program evaluation/audits) to compare programming aspects such as proposal solicitation and eligibility criteria, and to identify areas where the WP complements and/or overlaps with other federal programs. Interviews were also conducted with program staff from comparative programs.

g) Cost-effectiveness Analysis

An analysis of the efficiency of the program was undertaken, including an examination of utilization of resources (including leveraged resources) and the administrative to program dollar ratio of WP (annually). This analysis was supplemented by qualitative feedback obtained in the key informant interviews about efficiency (and where there are opportunities for efficiency gains). The subjective assessments of key informants were also considered with respect to the issue of economy. Efficiency and economy were also examined at the project-level by SWC using a case study approach focused on a cluster of projects to address women in non-traditional occupations. The results of this internal exercise have been incorporated in this report.

2.3 Limitations

A number of methodological caveats should be noted for this evaluation. First, much of the information that was gathered for this review is drawn from internal sources – program documentation and administrative data, as well as interviews or surveys with program managers and project proponents who have a clear stake in the program. This potential bias was mitigated to some extent by including external experts as interview respondents and unfunded applicants in the online survey.

There was limited information available directly from the ultimate client group for the program – women themselves. The WP funds external organizations to undertake project-based work, and reporting on results for program participants is the responsibility of the funding recipient. Contact information for project participants is not supplied to the program. Thus, impacts for individual women are limited to those indicated by project proponents (in project reports or the survey).

Determining the performance of the program in achieving its intended intermediate outcomes – increased participation of women in their community – was difficult. The survey and case studies provide some evidence in this area; however, the vast number of highly disparate funded projects made it difficult to achieve a succinct roll-up to the program level.

Other limitations of the individual lines of evidence include:

The limitations identified above are common ones for program evaluations of social programs. Like other studies, this evaluation uses multiple lines of evidence to address the weaknesses of individual methodologies. This process of triangulation strengthens the evidence to answer the evaluation questions. However, it should be noted that the determination of progress in achieving broad social objectives such as gender equality is difficult. This is due, in part, to the complex and long-term nature of the social change process, and the significant contribution required in other arenas (legislation, policy) and by other jurisdictions and sectors to achieve these objectives.

3. Findings

This chapter presents the findings of this evaluation of the WP, presented by evaluation issue (relevance and performance) and by the related evaluation questions.

3.1 Relevance

The evaluation examined the continued need for the WP, the alignment of the program with federal and departmental priorities, and the extent to which the WP is consistent with federal roles and responsibilities.

a) Ongoing need for the WP

Evaluation Question:

1. Is there an ongoing need for the WP to address the status and equality of women in Canada?

Overall findings:

The evaluation evidence indicates that there is an ongoing need to address gender equality. While there have been significant strides toward gender equality in Canada, inequalities remain in a number of areas. The program's priority areas – economic security and prosperity, violence and leadership – are supported by national data and are also consistent with those of other jurisdictions and governments internationally. Over the program cycle, demand for the program has been high, owing in part to a dearth of funding alternatives for programs targeted to women/gender equality.

The evaluation evidence supports the continued need to address the status and equality of women in Canada. National data and international measures indicate that while there has been progress on many indicators (e.g., women's participation in the workforce and post-secondary education), gender equality has not been achieved in Canada. The World Economic Forum (WEF), for example, tracks the gender gap in over a 100 countries using four key measures of gender parity: education, health, economics and politics. In its 2010 Global Gender Gap ReportFootnote 4, gender parity in Canada was found to exist in the areas of education and health. However, economics and politics represent two areas in which gaps in gender parity continue to exist in Canada. Based on these indicators, Canada ranked 20th out of 134 countries - a marked improvement from the 2008 Report in which Canada was ranked 31st out 130 countries, but a regression compared to Canada's 2006 ranking of 14th out of 115 countries.

With respect to the WP's three pillar areas of economic security, violence and leadership, the literature indicates:

Often referencing these statistics, evaluation key informants across all respondent categories agreed that there is continued need for interventions to advance gender equality. This sentiment was echoed by surveyed program applicants – 96 per cent indicated that "there is an ongoing need for a federal initiative such as the Women's Program to fund projects to promote gender equality and the full participation of women in the economic, social, cultural and political life of the country" to a great extent (responded 5 on a 5-point scale). Like key informants, program applicants pointed to persistent barriers to equality (e.g., violence, poverty, underrepresentation in political office, business and the media) experienced by women and the need for a long-term commitment to solutions.

A second indicator of ongoing need is demand for and utilization of the program. Based on the number of applications for funding received and projects approved, demand for WP funding has been strong, and grants and contribution funding for the five-year program cycle has been fully committed.Footnote 12. Since 2007-2008 (and the implementation of the program's revised Terms and Conditions and call for proposals processFootnote 13), the number of applications received annually increased dramatically, while the proportion of applications funded declined substantially. In between 2007-08 (the first full year of the program's new Terms and Conditions) and 2009-10, 26 per cent of applications received were approved. Despite the relatively low success rate, 80 per cent of surveyed applicants indicated that they plan to apply again for WP funding.

Table 3.1: Total WP Applications and Projects Funded — 2007-2008 to 2009-2010

Total WP Applications and Projects Funded - 2007-2008 to 2009-2010
Long Description

Program data. List of Applications Received and Projects Approved; compiled March 2010
EKOS Research Associates Inc.

b) Consistency with Agency Strategic Outcomes and Federal Priorities

Evaluation Questions:

2. Are the mandate, objectives, and programming elements of the WP consistent with SWC strategic outcomes and federal government priorities?

Overall findings:

The program is consistent with SWC strategic priorities (the program and the Agency share the same strategic/ultimate outcome of equality for women). The WP aligns with federal priorities, particularly those related to economic prosperity, and the program also supports complementary federal strategies (e.g., the Family Violence Initiative) and international commitments related to gender equality.

In addition to addressing persistent areas of gender inequality such as violence and economic security, other emerging needs were identified in the evaluation. Examples include: trafficking, sexual exploitation/hypersexualization of women and girls, immigrant and Aboriginal women. Given the breadth of the program's objectives and finite funds, a more definitive niche for the program was perceived to be important to drive more focused funding efforts for greater impact. This program niche would leverage federal strength and fund in areas where levers in other federal departments are absent.

Consistency with Agency Strategic Outcomes

The Agency's Program Activity Architecture (PAA) serves to provide an overview of how the programs and activities of SWC are linked and how their expected results are organized to contribute to achieving the outcomes and mandate of the SWC. SWC identifies two program activities in its PAA. The WP contributes to the second program activity which is women's participation in Canadian society and its expected result is "increased participation of women in their communities." In the immediate-term, the results of this program activity are measured by increased awareness of women to identify and remove barriers to their full participation in their communities.

The overall mandate and objectives of the WP are well-aligned with the SWC strategic outcome. The strategic outcome of SWC is "Equality for women and their full participation in the economic, social and democratic life of Canada", which is also the ultimate outcome of the WP.

Consistency with Federal Priorities

Program documents and key informant respondents indicate that the WP is aligned with federal government priorities in the following ways:

Gaps/Niche of the WP

The priority areas that have been identified for the program (i.e., the three pillars – violence, economic security and prosperity, and leadership) are widely believed by key informants across all respondent groups to continue to be "logical", "coherent", "a way to frame the dialogue". The priorities are also consistent with the way other organizations and jurisdictions provincially and internationally organize their understanding of gender equality. However, WP managers and staff key informants also noted some challenges with the priorities –they do not convey the extent of interconnectedness or intersectionality of the issues (e.g., that issues of violence are inextricably linked to women's economic security), and tend to represent the symptoms rather than root causes of gender inequality. The current areas of focus are also seen by program staff as being extremely broad. In recent targeted calls for proposals (Blueprint projects) efforts were made to focus on areas where more work needs to be done.

At the programming level, emerging priorities where the WP could focus are identified by the program in two ways: from the experience of the program itself (i.e., "organically, from the ground up" based on proposals that are received, networking and partnerships with community-based organizations); and through exchanges with the policy area within SWC – a relationship that program key informants note as having steadily improved.

Trafficking, sexual exploitation/sexualization of women and hypersexualization of girls are examples of emerging areas that are currently receiving attention within the program. Other emerging or pressing needs that could be addressed by the program that were mentioned by key informants include: work-family balance, violence against Aboriginal women, stereotyping, women and the environment, and women in rural or remote locations. Program applicants were also asked in the survey to identify any emerging issues or gaps in activities that the WP is particularly well-positioned to address in the next five years. Applicants identified a wide variety of issues and target groups, many reflecting areas where the WP is currently funding - e.g., economic security, violence, initiatives for immigrant and Aboriginal women.

Table 3.2: Emerging Issues/Gaps: Program Applicants
Are there any emerging issues or gaps in activities that you feel the Women's Program is particularly well–positioned to address in the next five years? Top 10 Responses
Per cent of Applicants (n=201)
Women's economic autonomy 17
Violence against women 16
Issues of gender equality (general) 13
Women's leadership and empowerment 11
Immigrant women
11
Women's participation in politics and public policy 11
Aboriginal women 10
Women who need mental health or addictions support 8
Employment and training for women
8
Research and policy development on women's issues 8
Source: Survey of Program Applicants, 2011

Program managers and staff were of the opinion that there is work to be done in better defining the niche of the WP, given its current breadth and finite funds. The program's previous Terms and Conditions focused on work with equality seeking organizations at the systemic level. The emphasis in the new Terms and Conditions for projects to have a direct impact on women was viewed as broadening the scope of the program, both in terms of potential proponent organizations and types of projects. A potential niche for the program was commonly described by program managers and staff as areas of programming where there is a void or vacuum. For example, according to these key informants, the program should be funding in areas where the provinces do not have jurisdiction or authority, or where there is an absence of federal levers in other departments. Areas such as women's leadership, gender-based analysis, awareness-raising around gender equality issues (being "the voice of equality", "women's voices") are examples of areas mentioned by program key informants where the program could play a 'value added' role.

c) Consistency with Federal Role and Responsibilities

Evaluation Questions:

3. Is the WP consistent with federal role and responsibilities?

Overall findings:

The federal level makes a relevant and important contribution to gender equality, though the federal government and the WP do not bear this responsibility solely. Canada's involvement in advancing gender equality is consistent with practices internationally. A caution is that program data indicate that the program funds many service delivery projects, an area that is traditionally the role of provinces.

While projects access complementary support from other funding sources (e.g., provinces, donations/fundraising), there were few concerns related to overlap or duplication.

Federal role and responsibilities

Key informants were uniformly of the opinion that the federal government has a role to play in gender equality. Advancing gender equality and achieving the full participation of women in Canadian society is perceived to fall within federal roles and responsibilities, though it is not solely the responsibility of SWC or the federal government. Stakeholders such as federal departments/agencies, other levels of government, non-governmental organizations and the private sector are also seen as having a key role to play in addressing the economic, social and cultural situation of women.

Canada's federal level involvement in gender equality programming is consistent with practices internationally. A review of programs and policies in selected countries (e.g., US, Australia, Finland) consistently indicated federal commitment to gender equality expressed in a variety of ways (e.g., National action plans, funding to women's organizations, federal government coordinating councils).

A caution is that the program data indicate that many projects funded under the program's Terms and Conditions focus on direct interventions with women (for example, skills development, and employment supports). Though many of these projects also include other activities such as increasing awareness, access to or improvements to services, this funding emphasis has frequently led the program into supporting service delivery projects, an area that is more typically within provincial roles and responsibilities. Program key informants raised this issue and supported a refocusing of programming to better emphasize the strength and opportunities offered by federal level engagement in the gender equality issue (e.g., multi-jurisdictional initiatives, awareness-raising, and knowledge building/knowledge transfer).

Overlap/duplication

All provinces and territories in Canada have women's directorates or women's offices. However, attention to the issue of gender equality at the provincial level was widely thought to be uneven across the provinces and territories and waning by key informants who are familiar with activities in these jurisdictions. It was noted among key informants and program applicants that there is generally a dearth of available funding in Canada to support projects that address the participation of women and gender equality specifically. However, there are a variety of funding sources (that may or may not have gender equality as their raison d'être) that are complementary to the WP. These programs provide funding for a specific target group (e.g., Aboriginal people) or issue (e.g., criminal justice).

Funded applicants most commonly identified the provincial government (31 per cent), donations/ fundraising (30 per cent) and non-governmental/philanthropic foundations or organizations (e.g., the Canadian Women's Foundation, the United Way) as complementary sources of funding for their project.

While there are other federal departments offering programming that touches on WP priorities (such as Canadian Heritage, Citizenship and Immigration Canada, Human Resources and Skills Development Canada and Aboriginal Affairs and Northern Development (AAND)), few project proponents indicated that they had received other federal funding for their project (15 per cent).

There are limited formal mechanisms for coordination of funding across federal departments or jurisdictions. At the regional level, there is some informal coordination between regional WP staff and provincial officers. Some regions have a more developed culture of collaboration than others, however, according to program key informants (and verified in the comments of provincial respondents) this contact has diminished over time due to regional capacity constraints.

3.2 Program Performance: Achievement of Intended Outcomes

This section presents the evaluation findings related to the achievement of the WP's intended outcomes. The emergence of promising best practices, as well as unintended outcomes are also discussed. WP performance was explored in key informant interviews, survey of applicants, and case studies, as well as the review of program documents and file review.

a) Immediate Outcomes

Evaluation Questions:

4. To what extent has the WP achieved or made progress toward achieving:

  1. its planned immediate outcomes (Increased awareness among women to identify and/or remove barriers to their participation in their communities; and increased partnerships with other federal departments, levels of government, NGOs and the private sector)
  2. its planned intermediate outcomes (Increased participation of women in their communities.)
Overall findings:

The program's immediate outcome of increasing women's awareness and removal of barriers to participation is measurable only by proxy through participation of women in the program's funded projects. Almost 400 projects were funded in the first four years of the program. The median number of women directly benefiting from projects is 125. There is good evidence that the program has been successful in stimulating partnerships with NGOs and other levels of government, while less so with other federal departments and the private sector. Projects leverage funding from partners – WP funds, on average about 60 per cent of project costs, with the proponent organization, and their partners funding the balance (including financial and in-kind contributions).

With respect to the program's intermediate program outcome – participation of women in their communities – there is some evidence of this impact, though it is difficult to "count" or provide a roll up of these impacts across the great variety of projects that are funded. The evaluation noted many examples of impacts across a spectrum of indicators including impacts on the individual women who participated, as well as projects that involved more far-reaching impacts on policy, institutional practices and services.

Awareness of Barriers

Awareness among women to identify and remove barriers to participation is expected to flow from the participation of women in WP funded projects. However, "awareness" is not generally measured at the project level and, therefore participation (i.e., number of women directly impacted by the project) is used as a proxy measure. As reported previously, 398 projects were funded under the WCF and WPF funding streams during the first four years of the program. According to funded applicants, nine in ten (91 per cent) projects included an awareness and outreach component (such as engagement of women in issues and identification of barriers), and the same proportion (91 per cent) included an intervention component directly with participants (involving, for example, education and training, workshops, community-based activities, or development of relationships and networks).

The nature of the barriers to women's participation that projects were designed to address was varied, and projects often addressed multiple barriers (i.e., intersectional in nature). According to funded applicants, projects most commonly addressed women's leadership or other skills, financial security/autonomy and lack of awareness/understanding of problems within the community (e.g., violence) (each barrier mentioned by one in four applicants).

Table 3.3: Nature of Barriers Addressed by Projects
What was the nature of the barriers to women's participation in their community that your project was designed to address? Per cent of Funded Applicants (n=150) (multiple mentions possible)
Leadership/other skills among women to bring about change 25
Financial security/autonomy 23
Lack of awareness regarding problems within the community 23
Addressing family/multigenerational violence and ensuring proper supports for victims 19
Barriers to basic needs (food, housing, transportation, child care, health care, etc.) 14
Access to services, capacity, and availability 13
Discrimination/marginalisation/oppression 13
Lack of access to training, education, work experience, etc. 13
Cultural barriers 12
Community isolation 11
Other 25

Based on the survey data, the median number of women that projects' directly affected was 125.Footnote 14 Across projects, there is a great deal of variation in reach, with some projects impacting small numbers of women (e.g., more intensive intervention projects with multi-barriered women) to other projects benefitting 3,000 women or more (e.g., projects that improve practices, service, or policy). This number was slightly higher among WPF projects, projects led by women's organizations or with prior program funding, projects with institutional level impacts, and projects that included development of a resource or tool. In the case studies, there were several examples of projects that obtained significant reach in terms of their impact. This occurred when projects were led by national or regional umbrella organizations or coalitions that were able to influence the practices or services of their member organizations and other networks for greater impact. Similarly, a monitoring exercise undertaken by the program in 2010 analyzed the outcomes of 13 WP projects and found that projects that impact individual women by addressing multiple facets of the barriers they are facing over the long term (addressing individual and systemic barriers) and projects that achieve structural improvements which, in turn, directly affect individual women have the most far-reaching impact (in terms of number of women impacted)Footnote 15.

About half of funded applicants (49 per cent) reported that the number of women directly affected by their project was higher than expected at the time of their funding application. One in ten (10 per cent) project proponents indicated that the number of women directly affected by their project was lower than expected.

Funded applicants were asked to rate their project's contribution to WP immediate outcomes. Most (89 per cent) felt their project contributed to increased awareness among women in identifying and/or removing barriers to their participation in their communities to a great extent. This was confirmed in a review of a sub-set of WP project files where nine out of ten projects were judged to have increased awareness among women in identifying and/or removing barriers to their participation in their communities.Footnote 16

Increased Partnerships

The development of partnerships was identified as an immediate intended outcome of the WPF only. However, because partnerships represent a key activity of the WCF also, and the WCF represents the majority of activity under the program, this immediate outcome is discussed for both funding components. Nine out of ten funded applicants (and all WPF project proponents) reported forming partnerships as part of their WP funded project. Of those projects that formed partnerships, nine in ten project proponents reported that all (15 per cent) or some (75 per cent) of the partnerships established for their project were with new partners. The vast majority of funded applicants felt that partnerships with other organizations have been important to the implementation of their project and achievement of project objectives (to a "great extent" by 82 per cent of WCF projects and 100 per cent of WPF projects).

Project partners were drawn from the community-based sector (88 per cent) more than from any other sector. Partnerships with other levels of government, particularly the provincial government, were common (42 per cent), while partnerships with other federal departments and the private sector were less frequent (15 and 11 per cent of funded applicants respectively). WPF projects were more apt to have partners from all these sectors (with the exception of the private sector) compared to WCF projects. Examples of other partners included Aboriginal organizations, educational institutions and health institutions.

The most common type of contribution from project partners, as reported by project proponents, was promoting awareness of the project (including recruitment of participants or providing participant referrals) (85 per cent). Projects also formed partnerships to share resources and materials, and to seek advice and expertise.

With respect to leveraging of resources, 91 per cent of funded applicants received other financial or in-kind assistance for their WP funded project. Of these, eight in ten project proponents (79 per cent) said that their own organization had contributed funding to the project. Six in ten surveyed funded applicants indicated that they had received funding from partners (financial or in-kind). Just over half of projects (54 per cent) leveraged support from volunteers and one-third from donations or fundraising (30 per cent).

The review of a sub-set of project files indicated that, on average, WP funding represented 61 per cent of the total cost of the project. While the number of WPF projects available for this file-based analysis was too small (n=5) to conduct a separate analysis, internal analyses conducted by the program indicates that, among WPF projects closing in 2008-09 and 2009-10 (n=11), financial and in-kind contributions were leveraged from partners in a better than 1:1 ratio with WP funds.Footnote 17

Funded applicants were asked to rate their project's contribution to WP's immediate outcome of increased partnerships. Two-thirds of funded applicants (66 per cent) indicated that their project increased partnerships with other federal departments, levels of government, NGOs or the private sector to "a great extent". Again, the review of project files was generally congruent – based on projects' final reports, six in ten projects were judged to have met objectives in terms of increased partnerships.

Almost all funded applicants reported that they will continue to work with project partners on other initiatives or build on results achieved in their WP project. In fact, six in ten (61 per cent) indicated that they are already working with partners on opportunities beyond the funded project. Another almost two in five (37 per cent) anticipated that there will be opportunities to work with the newly established partners in the future.

b) Intermediate Outcomes

The intended intermediate outcome of the WP is the increased participation of women in their communities. "Participation in communities" implies a broad spectrum of measures given the diversity of the projects that are funded by the program. Final reports that were examined as part of the file review provide a variety of indicators of community participation. Examples include:

Key informants also commented on the variety of forms of participation resulting from funded projects, noting that many projects resulted in very profound impacts on the women participating (e.g., leaving a violent relationship, obtaining positions of leadership in their community, securing additional education or long-term employment). Other projects were described as having far-reaching impacts for many women (e.g., implementation of pay equity provisions on a province-wide basis, providing access to culturally-relevant resources and tools to communities of women that had limited access to these resources, improving skills/practice guidelines among service delivery staff in shelters).

Funded applicants were asked to rate their project's contribution to WP intermediate outcomes in the survey. Most (82 per cent) felt their project contributed to increased participation of women in their communities to "a great extent" (responded 4 or 5 on a 5-point scale), again, consistent with the evidence in the review of project files.

c) Ultimate Outcome

Evaluation Questions:

4. To what extent has the WP achieved or made progress toward achieving:

Overall findings:

Key informants and project proponents were both cautious about the program's achievements in this area. Evaluation evidence to measure the program's performance on this measure was limited. There are also limitations on the program's potential impact given the finite amount of project-based funding available and the program's focus on a wide range of projects that are supported at the individual level. Still, there is evidence that many projects funded by the WP (six in ten) are at least moderately sustainable.

Progress toward outcomes

The program's planned strategic outcome is 'Equality for women and their full participation in the economic, social and democratic life of Canadian society'. By definition, the program's ultimate outcome is a longer-term endeavour, and one that is not the sole responsibility of WP. As noted earlier (Section 3.1(a)), though there has been progress on some indicators, gender equality in Canada has not been achieved. On many indicators (such as wage parity and representation in business and leadership positions) there has been a plateau.Footnote 19 In addition, new issues emerge over time (e.g., difficulties resulting from the economic downturn, honour-based violence).

In general, both funded applicants and key informants felt less confident in speaking to the impact of the program on gender equality. Funded applicants were cautious when asked in the survey to rate their project's contribution to the WP strategic or ultimate outcome. Six in ten (58 per cent) felt their project has contributed to the WP planned strategic outcome of equality for women and their full participation in the economic, social, and democratic life of Canada. Similarly, according to key informants, while the program can show concrete results in terms of participation of women in their communities, progress on gender equality was described as "slow" and in "bite sizes". Projects that are sustainable and/or lead to changes within communities, organizations/institutions or policies were perceived to have greater reach and importance in contributing to gender equality.

According to program key informants, the challenge in achieving the program's ultimate outcome is also symptomatic of a disconnect in the program's causal chain which does not move seamlessly from the awareness and participation of individual women to the achievement of the broader social objective of gender equality. Program managers and staff were of the opinion that while increasing the participation of women is one way of affecting social change, the program could be more effective in achieving its ultimate intended outcome by focusing on more sustainable projects with greater reach.

Sustainability of Effort

Two-thirds of funded applicants (65 per cent) reported that their project was completed at the time of the survey. For those projects no longer receiving WP funding, more than half of projects have been sustainable in some manner – that is, the project continued after WP funding ended (nine per cent) or the project is continuing on a limited scale (48 per cent). Projects that included impacts at the level of institutional practice were more likely to report sustainability. This effect was illustrated in several of the case studies of projects that created tools or resources to influence the way organizations deliver services. Gender-based analysis was a common technique to encourage organizations to review and change their practices or delivery, to better meet the needs of women.

Funded applicants with projects that continued in some form after WP funding ended were asked to identify the factors that support their project's sustainability. Just over half of these respondents indicated the project efforts will be sustained because their project activities have been incorporated or influenced their organization's operations (55 per cent) (e.g., project activities or aspects of them integrated into other programming, visibility of the organization/partnerships leading to other opportunities, proponent making investments in other program areas as a result of lessons learned from WP project). A similar proportion indicated that individuals trained by the project will continue to conduct activities with clients and communities (53 per cent) or sustainability will be supported through the continued dissemination of the resources and tools that were developed (52 per cent). Four in ten said their project activities have affected their partner organization's operations (38 per cent), additional funding has been received (38 per cent), or resources/tools are being used in other settings (36 per cent). These findings are very similar to those that were reported in the final reports – sustainability of the project most often occurring through integrating the project into the operations of the proponent organization or initiating knowledge-sharing activities to inform others about lessons learned or practices developed from the project.

Projects that have limited sustainability often are more resource-intensive projects that deliver a service to women. In the case studies that illustrated this approach, sustainability of these projects was a challenge. The project proponents in these instances were relatively small organizations, dependent on project funding with limited ability to continue the intervention (e.g., workshops, discussion sessions) in the absence of ongoing funding (a type of funding WP does not have the authority to provide).

Impact of External factors

Interview respondents in all categories were invited to identify external factors that may have a positive or negative influence on the effectiveness of the WP. Positive factors tend to be attributed to unpredictable opportunities or serendipity that will open a window of opportunity for collaboration or a change in the broader environment that creates demand for solution (e.g., labour shortage in Alberta providing opportunities for women to enter non-traditional occupations). Factors that were identified as negatively affecting WP's efforts include, for example, some feeling in the broader community that gender equality has already been achieved (the "job is already done"), thus undermining the perceived legitimacy of program investments. Several key informants noted that the economic climate (downturn) can negatively influence the amount of public funding that is available to address social issues, or the effectiveness of some projects, particularly in the skills development/employment area (i.e., limited employment opportunities available to project graduates).

At the project level, the survey of program applicants and the review of project files indicate a number of other, more practical challenges in project delivery. These challenges were often related to the recruitment of the target group. For example, one in five funded applicants identified difficulties with reaching their target groups (19 per cent) as a challenge for their project.

One in five funded applicants also identified financial constraints (19 per cent) as a challenge (e.g., budgetary shortfall, for example, due to higher than expected demand or unanticipated costs, need for ongoing funding) and one in ten experienced difficulties in finding partners or establishing networks (13 per cent). Also commonly cited were logistical challenges, such as securing childcare or transportation for participants.

d) Delivery Models/Best Practices to Achieve WP Intended Outcomes

Evaluation Questions:

5. What models have emerged as best practices? How are they contributing to the achievement of WP outcomes? How can these models be used to inform future program design and delivery?

Overall findings:

There were a number of observations from the evaluation (e.g., from the literature, views of key informants and funded applicants) on promising practices. Examples such as partnerships, involvement of participants in delivery, culturally relevant programming, mentoring, and wraparound supports are being used by many projects already. However, the knowledge-building and knowledge sharing capacity within the program, including the identification and sharing of promising practices/proven resources and tools, is limited and identified as a gap by the program and stakeholders.

The WP funds a wide array of projects utilizing various common activities or approaches. Examples of these that have been used by projects include: culturally adapted workshops; peer mentoring. In addition, several models have been used: asset-based model (Sustainable Livelihoods); co-operative model; 'stages of change' model; 'theraplay' model and a 'grassroots models of service delivery'. The evaluation evidence could not definitively determine among the various approaches and models which would be considered best or promising practices to inform future program/project design and delivery.

Program managers and staff were uniformly of the opinion that identifying 'best' or 'promising' practices among the interventions that it funds and disseminating these broadly is a time-consuming endeavour and is currently limited due to capacity issues. The recent Blueprints call for proposals (February 2011) is an example of the program's efforts to build this knowledge base and knowledge transfer capacity. Blueprints is funding projects to develop and pilot tools or approaches to address various priority areas identified by the program (e.g., women in non-traditional occupations). The projects include an external evaluation of the effectiveness of the model used and sharing of results.

However, based on the experience of funded applicants (survey and case study results), the program's own internal analyses and the literature, some overarching programmatic best practices could be identified.

Table 3.4: Best Practices: Funded Applicants
What best practices or lessons learned can be identified based on your organization's experience to improve the design or delivery of projects to address barriers to women's participation in their communities? Per cent of Funded Applicants (n=150)
Ownership/involvement of participants in the project 24
Partners, networks, champions are helpful to expand the project's reach across the community 19
Programs need to respond to community needs/community-driven 19
Ensuring adequate supports are in place for participation (e.g., transportation, child care) 19
Collaboration with partners and clients can strengthen the project's impact 13
Long-term, sustainable programs are needed to address some issues 9
Other 23

e) Unintended Outcomes

Evaluation Questions:

6. What unintended outcomes has the WP had? What are the lessons learned?

Overall findings:

Unintended outcomes of the program were generally few, but largely positive. The experience at the project level has shown that many projects have garnered greater than anticipated participation and interest in their initiative by women, partners and the community.

Four in ten funded applicants did not mention any unintended outcomes of their project, while the remainder reported largely positive unintended outcomes of their project. One-quarter (25 per cent) reported greater than expected interest in the project, while one in five (20 per cent) noted new or improved partnerships. One in seven identified organizational improvements (15 per cent) or improved capacity and additional funding initiatives (14 per cent) as an unintended (or unexpected) outcome of their program.

Of the minority of projects that mentioned negative unintended or unexpected outcomes, the risks or difficulties of participating in interventions to the women themselves was raised in some project contexts (e.g., that family or community members may feel threatened by the project, or women's participation may raise troubling issues for them – reliving trauma associated with abuse, sexual exploitation).

3.3 Design and Delivery

Evaluation Questions:

7. Are Program design, delivery mechanisms and structures now in place appropriate and effective? What has been the overall impact of the ongoing transitional changes? What opportunities exist to improve the program design?

8. Is the level of access to program funds by potential recipients sufficient to achieve program outcomes?

9. Does the performance measurement strategy/framework allow for the capture of adequate and useful information for assessing Program impact? How could performance reporting be improved?

Overall findings:

Design and delivery mechanisms are generally appropriate, and the program has made adaptive changes to improve delivery. Satisfaction with the application process was only moderate among funded applicants who identified the potential to streamline, simplify and clarify the application process and improve timeliness. Most unfunded applicants did not feel adequately informed as to why their application was unsuccessful.

The changes in the program's Terms and Conditions, as well as operational funding reductions have had both positive and negative impacts. The current Terms and Conditions have served to ensure a more concrete focus on the tangible impacts of programming on women, and introduced many mainstream organizations to gender-based programming. However, operational funding reductions have created capacity constraints and placed increasing and changing demands on staff, who have limited access to support.

The key themes that emerged in terms of design improvements had to do with: improving the knowledge building/knowledge transfer capacity of the program; extending/reinforcing collaborations with partners and stakeholders; streamlining applications processes; and support to staff.

Some of the program's funding eligibility criteria for organizations and for eligible activities were seen to be overly restrictive and limiting access to program funding to achieve objectives. Indeed, the criteria were found to be quite variable across a number of federal grants and contributions programs that were examined. Incorporating an emphasis on sustainability of projects was raised as a way to direct funds for maximum impact.

Performance measurement has improved over the current program cycle, with new tools and processes introduced to better capture results. Like other programs with broad and longer-term social objectives, the program faces challenges in measuring and attributing impact. Funded applicants are generally satisfied with the expectations and their requirements for monitoring and reporting (their level of effort, performance measures, accessibility of staff). While there are undoubtedly improvements that could be made to the tools (e.g., better reporting on outcomes as opposed to outputs), performance measurement is more seriously hampered by human resource constraints to undertake the time consuming analysis and synthesis of information to inform program decision-making and strategic planning.

a) Appropriateness/Effectiveness of Program Design

Program Design

Key informants were asked to comment on a number of aspects of the program's design, notably: the structure of the program's funding streams; proposal solicitation and assessment; roles and responsibilities; and program capacity. With respect to the WCF/WPF funding streams, interviewed program key informants uniformly indicated that the two programming streams did not unfold the way they were intended. According to these respondents:

With respect to processes for promotion, proposal solicitation and assessment, program key informants noted that the program has limited capacity to conduct formal promotion of the WP to potential grantees. Surveyed program applicants were most often aware of project funding opportunities available through the WP because they had previously received funding (38 per cent). Other common sources of awareness included word of mouth, the WP website or through contact with the WP.

Satisfaction with the program application process among applicants was moderate: half of the applicants surveyed were satisfied with the overall process of applying for project funding. Considering various aspects of the application process, satisfaction with the guidance or support received from WP staff during the proposal process (among those who received it) and with the amount of funding awarded in relation to the amount requested (for most projects, amount received was similar to amount requested) was comparatively high (over seven in ten expressed satisfaction on these aspects). Weaker elements of the application process were level of effort required to prepare an application and the timeliness of learning the result of the funding decision which had lower satisfaction ratings.

Table 3.5: Applicants' Experience Applying to the WP for Funding
How satisfied were you with...? Per cent of Applicants Table Note * (n=201)
The guidance or support received from WP staff during the proposal process (n=140) Table Note ** 77
The amount of funding awarded in relation to amount requested 73
The clarity of the requirements to complete application 58
The eligibility criteria for funding 58
The ease of learning about funding opportunities under the WP 54
The level of effort involved to prepare application 42
The way the program defines and assesses risk as a criterion 38 Table Note ***
The timeliness of learning the result of the funding decision 38
The overall process of applying for project funding under the WP 51

Table Notes

Table note *

Represents per cent satisfied (rated 4 or 5 on a 5 point scale)

Return to table note *  referrer

Table note **

Those who received guidance or support

Return to table note **  referrer

Table note ***

17 per cent of applicants responded "don't know" to this question

Return to table note ***  referrer

There is a gap in program communication with respect to unfunded applicants – most unfunded applicants surveyed felt that they were not adequately informed as to why their application for funding was not successful (only one in four felt adequately informed).

A key issue for program managers and staff with respect to proposal solicitation and application was the shift in the program's approach to proposal solicitation for WCF which began as a continuous intake process, shifted to call for proposals and now more recently has included a revival of the continuous intake model. In a Comparative Review of Funding Delivery Models conducted internally, a reported strength of the call for proposals model was the transparency of the approach and the consistency in the review and approval process. However, the lack of mechanism to pre-screen potential proposals or applicants resulted in a large numbers of applications and a lengthy review process (e.g., 4-6 month process to review and approve projects). The call for proposals model was also believed to represent a higher level of risk exposure given the volume of applicants and limited intelligence gathering/interaction between program staff and applicant organizations during the proposal process. Conversely, the strength of the continuous intake model was the short 4-6 week review and approval process. As well, workload – receipt of proposals, approvals, review of reports – was spread more evenly throughout the year.

Findings from the comparative program analysis conducted as part of this evaluation indicates that slightly more of the federal grants and contribution programs reviewed use a call for proposals or request for letters of interest approach than a continuous intake model (although one program reviewed uses both). Having experience with both types of processes, the current consensus view of program managers and staff interviewed is to favour a dual model – continuous intake of proposals to enable more proactive work with proponent organizations, together with strategic periodic calls for proposals.

With respect to roles and responsibilities and program capacity, program key informants raised a number of challenges, stemming largely from the changes in the program's Terms and Conditions and operational funding. These are described in more detail in the following section.

Program Transitions and their Impact

The WP Terms and Conditions and program administration changed significantly with program renewal in 2006-2007.Footnote 20 These changes were informed by a number of processes (the 2005-2006 Summative Evaluation of the WP, consultations with stakeholders and the recommendations of the Standing Committee on the Status of Women) and were influenced by reductions in SWC's operating budget in 2006-2007.

According to program managers and staff, the impact of changes to the program's Terms and Conditions has been mixed. On the positive side, the increased focus of programming to directly impact women has encouraged project proponents to think more concretely and to more effectively link their project activities to tangible results for women. Also positive, the extension of the eligibility criteria (beyond equality-seeking organizations) has broadened the pool of organizations with which the WP has contact to include mainstream organizations. Though these organizations may not have a formal mandate with respect to gender equality, their participation was perceived by the majority to extend the awareness of and attention to gender equality beyond women's groups (although the evaluation data indicate generally less reach and fewer sustainable impacts for projects led by mainstream groups).

Negative impacts of the program transitions identified by staff and management included:

Surveyed applicants were divided in their assessment of changes to the WP since 2006. The most positive change was seen to be the shift in the focus of the program from funding projects that foster institutional change to ones that have a direct impact on women; which was viewed as positive by about half of the applicants. Only one-quarter favoured replacing the continuous application process with an annual call for proposals (four in ten rated this change as negative). Similarly, the change in funding eligibility criteria was viewed positively by only one in five applicants (and viewed as negative by a similar number). Those who had previously received funding from the WP were more likely to see the change in eligibility criteria as negative.

The program's transitional changes appear to have had a moderate, though largely negative, impact on the level of service provided by WP staff. Among program applicants who had also applied to the program before 2006, two in five applicants noted a difference in the information, guidance or support they received from WP staff. Of those who noted a difference (n=80), the nature of the change was largely negative - nearly half noticed a decrease in ability to contact or receive support from staff (particularly among proponents located in the West), one in six noticed changes in funding and the application process (e.g. less funding available, more stringent eligibility criteria, tighter competition for funds), and one in ten commented on a deficit in the quality of informed or well-connected staff.

Design Improvements

A number of themes emerged with respect to suggested design changes to improve the effectiveness of the Program:

b) Level of Access to Program Funds

As reported previously, the WP received 1400 applications between 2007-08 and 2009-10, and funding was approved for 26 per cent. A detailed application form must be completed by all applicants. The application form collects information on the organization, the project, budget, and anticipated impact and reach. The application form must be accompanied by a project proposal (five pages or less) and a project work plan, results framework and performance measurement plan. The proposal must also provide information on the project context, a justification for the project; detail the population to be reached and anticipated results of the project. All proposals (to both the WCF and WPF) are assessed following a multi-stage process that considers general eligibility requirements of the WP; the quality of the proposal; and level of risk.

Projects are not selected for funding for a variety of reasons: they fall below the program's rating threshold; the proposed project is judged to be outside the scope or mandate of WP, the organization or proposed activities are ineligible for funding by the WP; or deficiencies are evident in the proposal (e.g., duplicates other work, unable to secure other partners, unable to demonstrate need or justify funding request).

For projects that are approved, incrementality of WP funding appears strong. Very few funded applicants surveyed believe that their project would have proceeded in the absence of WP funding. More than three-quarters of surveyed funded applicants believe that their project would not have proceeded; while one in five believe it would have proceeded on a reduced scale. Consistent with this finding, close to half of the unfunded applicants surveyed indicated that their project did not proceed. Only one in six unfunded applicants surveyed indicated that their project proceeded although on a reduced scale (e.g., shorter duration, fewer participants or fewer activities), while only one in ten proceeded as planned.

Program managers and staff had two concerns about proponents' access to funds. First, program managers and staff felt that certain restrictions with respect to organization eligibility criteria (e.g., exclusion of academic/educational organizations and cooperatives) and activity eligibility criteria (e.g., awareness campaigns, research) were limiting the potential effectiveness of the WP, particularly in the area of knowledge sharing. The comparative analysis of other federal programs indicates that federal grants and contributions programs are not consistent in defining these eligibility criteria (i.e., other programs allow applications from academic institutions, cooperatives and other government agencies, and/or include broader categories of activities).

Second, some program key informants favoured the notion of sustainability of projects as a criterion for access to project funds; that is, moving away from projects that are 'one-off' or have very small numbers of participants, to focus more strategically on projects that have greater reach and the potential for ongoing or longer-term impact. As noted previously, projects that report impacts both at the level of individual women, as well as at the level of communities and institutional practice/policy, are more likely to be sustainable.

c) Performance Measurement

Following the 2005 Summative Evaluation, the WP logic model was revised and the program strengthened its commitment to results-based management (RBM). At the project level, for contribution funding, payments are normally based on achievement of performance objectives and/or reimbursement of costs incurred or expenditures made by the recipient. Grant funding typically entails fewer reporting requirements, however, WCF projects are also required to submit interim and final reports which demonstrate progress in project implementation. The recent Blueprints call for proposals (February 2011) – particularly because it involves testing models – incorporates a strong evaluation component (there is dedicated funding to external evaluation and sharing of results).

Performance measurement templates and tools available to the program include:

Based on these project-level data sources, the program has produced some higher level analyses based on the closeout assessment forms. For example, the 2009-2010 Departmental Performance Report examined short-term outcomes for 22 projects organized according to the Program's three priority areas.

Most funded applicants provided positive ratings of the WP's performance measurement requirements and tools. Eight in 10 agreed that the level of project reporting required is reasonable, the required reporting format adequately reflects project activities, that WP staff were available to answer questions and provide expertise in addressing performance reporting questions, and that reporting requirements allow for the opportunity to demonstrate project impact. Of those who did not agree with the statements (n=12), the most common complaints were that reports were too complicated or required too much detail, or the reports took too much time or effort to prepare.

The consensus view of program managers and staff is that while monitoring and performance reporting have improved over time, there continue to be challenges in assessing the impacts of the program:

Funded applicants offered few suggestions to improve performance measurement and reporting. A minority of applicants – about one in ten or less – suggested more flexibility in reporting, or allowing for more customized reports, more contact with staff to help clarify requirements, or a more simplified and less repetitive reporting format. Key informants made suggestions for improvement to performance measurement that included: improve project-level reporting to obtain better information on how funded projects lead to longer-term or organizational change (i.e., emphasize reporting on outcomes as opposed to outputs); and analysis leading to more information on best practices and model approaches (e.g., who WP funds, for what models, and to what effect).

3.4 Efficiency and Economy

a) Program Costs

Evaluation Questions:

9. How does the cost of the WP delivery compare to other federal grants and contributions programs?

Overall findings:

The program's administrative efficiency ratio is reasonable – for every dollar of funding that is granted or awarded by the program, $0.13 is spent on program administration. Program comparators could not be identified.

A common indicator of program efficiency is administrative efficiency (the ratio of operational costs to program dollars), which can be computed for the WP grants and contribution program.Footnote 24 Note that program leveraging is often identified as an indicator of efficiency. This analysis may be found in Section 3.2(a).

The average annual administrative cost for the WP (administrative costs as a proportion of total program budget) has varied between 10.4 per cent and 12.7 per cent between 2007-2008 and 2011-2012 (average of 11.7 per cent).

Table 3.6: Administrative and Program Costs, Administrative Efficiency Ratio
In $ millions 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 Total
Salary and operating costs 2.0 2.9 2.9 2.8 2.7 13.3
Grants 13.45 17.55 14.75 14.75 14.75 75.3
Contributions 3.7 6.6 5.2 5.2 4.2 24.9
Total 19.15 27.05 22.85 22.75 21.65 113.45
Administrative costs as a proportion of total program costs (salary and operating costs/total program costs) 10.4% 10.7% 12.7% 12.3% 12.5% 11.7%
Administrative efficiency ratio (salary and operating costs as a proportion of grants and contribution dollars awarded) $0.12 $0.12 $0.15 $0.14 $0.14 $0.13
Source: 2007-08 to 2010-11 Public Accounts and 2011-12 Main Estimates

With respect to an administrative efficiency ratio, on average, the administrative cost for program delivery is $0.13 for every program dollar allocated. The administrative efficiency ratio for the WP compared to other grants and contribution programs of a similar nature in government is difficult to ascertain. For example, of those programs selected as "comparator programs" for the comparison review component of the evaluation, none of these programs publish administrative efficiency ratios, nor were they presented in evaluations of these programs (where the reports were provided or could be located).

b) Program Efficiency and Economy

Evaluation Questions:

10. To what extent has economy and efficiency been demonstrated by the WP?

Overall findings:

A number of factors support the program's efficiency and economy: the program's lean operations and leveraging of resources (including expertise, networks, volunteers) from the community-based sector. The capacity issues of the program were perceived to be a drag on program efficiency – that is, the limited staff complement and supporting automated systems undermine the program's efficiency and economy.

Few alternative models or agencies are available to deliver on the WP's mandate. The WP is distinguished as a national program that addresses equality through engagement of multiple stakeholders in multi-jurisdictional contexts. Potential alternative program vehicles such as provincial or third party delivery were generally seen to have limited capacity or authority to deliver the program, and carry some additional risks and potential disadvantages (e.g., related to accountability and federal visibility).

Factors that promote/hinder efficiency and economy

Program managers and staff key informants were asked to indicate the extent to which WP program activities have been delivered in an efficient and cost-effective manner. The views of key informants on the efficiency of delivery at the program-level were generally positive. Among the key factors that were seen to be contributing to the program's efficient delivery were the program's lean operations. As well, key informants noted the leveraged investment that occurs at the community level through project-based funding. Community-based proponent organizations typically have limited administrative costs and can leverage resources from local partners, as well as contribute their own expertise, networks and volunteers.

At the project level, projects that were included in the case studies also indicated efficient delivery. One of the projects that was examined noted their successful leveraging of significant resources in addition to WP funding from sources such as the province, and foundation, industry association and employer partners. Other case studies projects pointed to their ability to obtain other (non-monetary) contributions such as the expertise of their organization and networks at a low cost in the community sector. Also, utilizing proponents' existing dissemination channels and reach was noted as a cost-effective way to raise awareness and broaden project results.

It should also be noted that some clusters of projects funded by the WP potentially derive a social return on investment. Internal analyses conducted by SWC using a cluster of projects that support women's participation in non-traditional occupations found a substantial return on project investments for this cluster. This analysis examined the program's investment in women's employment in terms of social returns derived from increased wages for women and returns to employers, which yielded estimates of returns in the order of 1:1.5 to 1:2.Footnote 25

The lean operation of the program, while providing low program costs, was noted by most program key informants as also a hindrance to the effective delivery of the program. Limited human resources and technological support are persistent challenges to deliver the program in a way that is strategic and derives full value from the experience, resources and tools developed by projects through knowledge sharing activities.

Alternative potential delivery agents/alignment with other program delivery models or best practices

The international literature indicates a wide variety of approaches in other jurisdictions to gender equality. In the US, for example, federal funding of programming in support of gender equality tends to be dispersed among government agencies and departments. However, since March 2009, these efforts are coordinated by the White House Council on Women and Girls. Australia, like Canada, has a Women's Office, housed within the federal Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) that is mandated to advance the status of women through programs, services, grants and funding. Like SWC, the Women's Office is the principal agency responsible for delivering women's programs and for consultations between the Australian government and women's organizations. These programs tend to deal with specific areas such as representation and policy-making, leadership and development, sport and violence against women. Australia (and other countries such as Finland) have funded national groups to take the lead in some areas. In both countries, strategic direction is provided by national action plans (e.g., Time for Action: The National Council's Plan for Australia to Reduce Violence against Women and their Children, 2009-2022, Australia). There is little evidence internationally, however, on the relative merits of these various approaches or their transferability to the Canadian context.

Alternative potential delivery agents or alternative models to the WP that were suggested by key informants included program delivery by provinces or by a third party organization. However, program alternatives were generally not seen to be equivalent in terms of infrastructure or programming breadth to the WP. As mentioned, provincial women's directorates/offices were not seen by key informants, including the provincial representatives themselves, as being in a position to undertake national initiatives, given the limited capacity of these offices and the absence of levers outside their own province. Delivery of the program by a third party organization (such as a community-based organization) was identified as a second potential alternative, though again, limited capacity and infrastructure of voluntary organizations represents a disadvantage of this approach. The program has also explored delivery through regional service delivery locations of other federal departments. SWC control and visibility, as well as accountability would be more limited for these alternatives (in comparison to an internally-delivered program).

Another possibility, suggested by some program managers and staff, would be to provide funding to a third party organization(s) with specific expertise to deliver a portion of WP grants in their area of expertise (and within a set of broader priorities and guidelines outlined by the program). This approach could ease internal capacity issues within the program and potentially contribute to program efficiency and effectiveness.

4. Conclusions

SWC's Women's Program was renewed in 2006-07 with revised program Terms and Conditions and a mandate to advance the equality of women and their participation in social, economic and democratic life. The program's funding cycle was completed in September, 2011 and, therefore, a summative evaluation of the program was identified as a priority in the SWC's 2010–2011 Agency Report on Plans and Priorities. The findings of the evaluation lead to the following broad conclusions about the relevance, design and delivery, and performance (effectiveness, efficiency and economy) of the Women's Program.

4.1 Relevance

4.2 Performance

Effectiveness

Design and Delivery

Efficiency and Economy

5. Recommendations

Recommendations to improve the Women's Program are based on the findings and conclusions of the evaluation. The following recommendations focus on program improvements in program performance and program design and delivery.

Program Performance

Design and Delivery

Appendix A Evaluation Framework

Summative Evaluation of the Women's Program 2010 - Evaluation Framework
Evaluation Issues and Questions Indicators Literature Review Document Review File/Data Review KI Interviews Targeted Survey Case study
Relevance
1. Is there an ongoing need for the WP to address the status and equality of women in Canada?
  • Changes or trends over 2006/07 to 2010/11 in indicators of gender equality
  • Support for addressing equality issues through an instrument such as WP
  • Evidence of program demand (e.g., applications, program funding spent)
X X X X    
2. Are the mandate, objectives, and programming elements of the WP consistent with SWC strategic outcomes and federal government priorities?
  • Degree of correspondence between mandate, objectives, and programming elements of the WP with SWC strategic outcomes
  • Assessment of alignment between mandate, objectives, and programming elements of the WP with federal government priorities.
  • Identification and analysis of gaps that could be addressed by the WP
X X   X    
3. Is the WP consistent with federal role and responsibilities?
  • Degree of correspondence between program mandate and federal jurisdiction, roles and responsibilities
  • Degree or nature of overlap of federal role and responsibilities with roles/responsibilities of provincial orders of government.
  • Analysis of other potential delivery agents/ alternative model, their advantages and disadvantages identified
X X   X    
Performance
4. To what extent has the WP achieved or made progress toward achieving:
  • a. its planned immediate outcomesFootnote 26 (Increased awareness among women to identify and/or remove barriers to their participation in their communities. AND increased partnerships with other federal departments, levels of government, NGOs and the private sector)
  • Number and nature (including activities) of funded projects
  • Reach of funded projects (# and profile of women participating)
  • Types of barriers identified by projects
  • Types of resources (outputs) produced that identify and address barriers to participation; perceived quality and usefulness
  • Number and nature of funded projects that demonstrate increased awareness and knowledge of barriers to participation
  • Assessment of leveraging of WP funds
  • Number and nature of partnerships created:
    • through Partnership Fund
    • at the project level through the Community Fund
  X X X X X
  • b. its planned intermediate outcomesFootnote 27 (Increased participation of women in their communities.)
  • Extent to which results achieved at project level with respect to women's participation have been achieved
  • Changes in capacity to identify and address barriers
  • Number and nature of partnerships created:
    • through Partnership Fund
    • at the project level through the Community Fund
  X X X X X
  • c. its planned strategic outcomeFootnote 28 (Equality for women and their full participation in the economic, social and democratic life of Canadian society.
  • Extent of contribution of Program to equality/ participation
  • Assessment of sustainability of effort
  • Extent of perceived impact of external factors (mitigating/enhancing)
  X X X X X
5. What models have emerged as best practices? How are they contributing to the achievement of WP outcomes? How can these models be used to inform future program design and delivery?
  • Identification of best practices supported by evidence from peer- based and relevant government literature
  • Analysis of factors of success (nature and reach)
  • Cost benefit analysis of selected model
X X X X X X
6. What unintended outcomes has the WP had? What are the lessons learned?
  • Identification and analysis of unintended outcomes and effects
  • Analysis of impact on the WP and/or how dealt with
  X X X X X
Design and Delivery
7. Are Program design, delivery mechanisms and structures now in place appropriate and effective? What has been the overall impact of the ongoing transitional changes? What opportunities exist to improve the program design?
  • Extent of change in impact / effectiveness of Program design and delivery on achievement of results
  • Analysis of perceived impact of various factors related program design and delivery (all actors)
  • Analysis of processes for promotion, proposal solicitation and assessment
  X X X X  
8. Is the level of access to program funds by potential recipients sufficient to achieve program outcomes?
  • Analysis of project funding decisions (what is and what is not)
  • Extent to which program design supports project/activity sustainability
    • Degree to which eligibility criteria are appropriate measured against applicant
  • Extent to which program delivery approach contributes to project/activity sustainability
    • Degree to which internal selection criteria are appropriate measured against applicant
    • Changes in recipient perspectives on project / program information, processes and mechanisms (e.g., quality, usefulness of program material; project performance measures, selection criteria, selection of funding instrument, project performance reporting requirements, program response times, etc.)
    • Analysis of processes for promotion, proposal solicitation and assessment
  • Assessment of ability of project applicants to secure other funding
  X X X X  
9. Does the performance measurement strategy/
framework allow for the capture of adequate and useful information for assessing Program impact? How could performance reporting be improved?
  • Analysis of project recipients' capacity to respond to performance reporting requirements (performance measures, templates, interim and final activity reports and evaluations)
  • Assessment of quality of program administration, information and reporting tools and processes
  • Extent to which current performance mechanism supports program analysis, policy and management (decision making)
  X X X X  
Efficiency and Economy
10. How does the cost of the WP delivery compare to other federal grants and contributions programs?
  • Analysis of WP administrative costs using those of similar federal programs including an assessment of mitigating factors (e.g., size, delivery of program, orientation)
X X X X    
11. To what extent has economy and efficiency been demonstrated by the WP?
  • Assessment of alignment with other program delivery models or best practices identified from the literature
  • Analysis of perceptions of factors that promote/ hinder efficiency and economy
X X X X    

Appendix B Bibliography

Aruna Papp, "Culturally Driven Violence Against Women in Canada's Immigrant Communities", Frontier Centre for Public Policy Series No. 92, July 2010.

Bramham, Daphne. 2010. "Someday There Will Be No More "Firsts." Ottawa Citizen, Sept. 9. http://www.ottawacitizen.com/life/Someday+there+will+more+firsts/3786640/story.html

Burghardt, J., & Gordon, A. (1990). More jobs and higher pay: How an integrated program compares with traditional programs. New York: Rockefeller Foundation.

Canadian Centre for Justice Statistics. "Victimization and offending among the Aboriginal population in Canada", Jodi-Anne Brzozowski, Andrea Taylor-Butts and Sara Johnson. Juristat. Vol. 26, no. 3, 2006.

Canadian Centre for Policy Alternatives. "Women's Poverty and the Recession." Monica Townson. Sept. 2009
http://www.policyalternatives.ca/sites/default/files/uploads/publications/National_Office_Pubs/2009/Womens_Poverty_in_the_Recession.pdf

Carter, Ph.D., Nancy M., and Christine Silva. Pipeline's Broken Promise. The Promise of Future Leadership: Highly Talented Employees in the Pipeline, Catalyst Inc.
http://www.catalyst.org/file/340/pipeline%27s_broken_promise_final_021710.pdf

------. 2009. Opportunity or Setback. High Potential Women and Men During Economic Crisis. Catalyst Inc.
http://www.catalyst.org/file/305/opportunity_or_setback_final_081209.pdf

Castaño, C. et. al. Policy towards Gender Equity in Science and Research, Seventh Framework Programme, October 2010.

Catalyst Research and Communications, Monitoring Mission. Final Report. March 31, 2010

Centre for Policy Alternatives. "Women's Poverty and the Recession", Townson, Monica. Ottawa: 2009.

CIBC Small Business. "Women Entrepreneurs: Leading the Charge, 2005" http://www.cibc.com/ca/pdf/women-entrepreneurs-en.pdf

Clark, M., & Huston, T. (1993). Assisting the smallest business: Assisting microenterprise development as a strategy for boosting poor communities. An interim report. Washington, DC: Self-Employment Learning Project, Aspen Institute for Humanistic Studies.

Cooper, M. et. al. Review Of Best Practices In Family And Sexual Violence Programming, Family and Sexual Violence Review Advisory Committee, City of Calgary, 2004.

de la Rey, C. C., Jankelowitz, G. G., & Suffla, S. S. (2003). Women's leadership programs in South Africa: a strategy for community intervention. Journal of Prevention & Intervention in the Community, 25(1), 49-64.

European Social Fund, Equality and Diversity Good Practice Guide: Gender, 2007

Ibarra, Herminia, Nancy M. Carter and Christine Silva. 2010. "Why Men Still Get More promotions than Women, Your high-potential females need more than just well-meaning mentors." Boston, Mass: Harvard Business School Publishing Corp. https://hbr.org/2010/09/why-men-still-get-more-promotions-than-women

Indian and Northern Affairs Canada. "Aboriginal Women: A Demographic, Social and Economic Profile". Ottawa: Summer 1996.

Industry Canada. "Key Small Business Statistics," July 2010
http://www.ic.gc.ca/eic/site/061.nsf/eng/h_rd02488.html

Justice Canada. Research and Statistics Division. "Examination of Declining Intimate Partner Homicide Rates: A Literature Review." Dawson, M. 2001.
http://www.violenceresearch.ca/sites/default/files/reports/DAWSON%20%282001%29%20EXAMINATION%20OF%20DECLINING%20INTIMATE%20PARTNER%20HOMICIDE%20RATES_0_0.pdf

Livingstone, Anne-Marie, Lucie Chagnon et al. From Poverty to Empowerment: A Research Report on Women and Community Development (CED) in Canada.
http://www.cdnwomen.org/PDFs/EN/CWF-frompoverty-report.pdf

Lugo, Nancy Rudner. Empowerment education: A case study of the Resource Sisters/Compañeras Program. Health Education Quarterly. Vol 23(3), Aug 1996, pp. 281-289

McNish, Jacquie and Janet McFarland. 2010. "Why the executive suite is the final frontier for women." Globe and Mail, Friday, Oct. 08, 2:43PM EDT.

Morris, Marika, et al. 2007. "Integrating the voices of low-income women into policy discussions on the Canada Social Transfer (CST)." Canadian Research Institute for the Advancement of Women. Fact Sheet #9.

Murray, Janet, et al. Women in Transition Out of Poverty: A guide to effective practice in promoting sustainable livelihoods. Women and Economic Development Consortium, January 2, 2002.
http://www.cdnwomen.org/PDFs/EN/CWF-WIT-guide.pdf

National Council on Welfare, Presentation to the Standing Committee on the Status of Women, June, 2006.

National Council on Welfare, release of 2006-2007 report, December, 2008.

Native Women's Association of Canada. 2009. Voices of Our Sisters In Spirit: A Report to Families and Communities. http://ywcacanada.ca/data/research_docs/00000022.pdf

Okagaki, A. (1989). Women and self-sufficiency: Programs that work, policy that might. Washington, DC: Corporation for Enterprise Development.

Raheim, S. S., & Bolden, J. J. (1995). Economic empowerment of low-income women through self-employment programs. AFFILIA: Journal of Women and Social Work, 10(2), 138-154.

Sanders, C. K., Weaver, T. L., & Schnabel, M. M. (2007). Economic education for battered women. AFFILIA: Journal of Women and Social Work, 22(3), 240-254.

Smith, E., Nowhere to Turn? Responding to Partner Violence Against Immigrant and Visible Minority Women, Canadian Council on Social Development , 2004.

Statistics Canada. "Women in Canada, A Gender-based Statistical Report." http://www.statcan.gc.ca/pub/89-503-x/89-503-x2010001-eng.htm

Statistics Canada, "Family Violence in Canada: A Statistical Profile, 2009".
http://www.statcan.gc.ca/pub/85-224-x/85-224-x2007000-eng.pdf

Statistics Canada. "National Trends in Intimate Partner Homicides, 1974-2000", Pottie-Bunge, V. 2002. Juristat. Vol. 22, no. 5. Catalogue no. 85-002-XPE. Ottawa.

Statistics Canada. Canadian Centre for Justice Statistics. "Family Violence in Canada: A Statistical Report, 2007". http://publications.gc.ca/collections/collection_2007/statcan/85-224-X/85-224-XIE2007000.pdf

Statistics Canada. 2003. "Canadians and their non-voting political activity." The Daily. June 19. http://www.statcan.gc.ca/daily-quotidien/070619/tdq070619-eng.htm

Status of Women Canada. 2010. "2010-2011 Report on Plans and Priorities", http://www.tbs-sct.gc.ca/rpp/2010-2011/inst/csw/csw-eng.pdf

Status of Women Canada. The Honourable Rona Ambrose, Minister of Public Works and Government Services and Minister for Status of Women. "2010, November 25". Government of Canada calls for an end to violence against women [Press release].
https://www.canada.ca/en/news/archive/2010/11/government-canada-calls-an-end-violence-against-women.html

Townson, Monica, "Women's Poverty and the Recession." Canadian Centre for Policy Alternatives, Sept. 2009, isbn 978-1-897569-72-6, p. 15

Trimble, Linda, and Jane Arscott. Still Counting: Women in Politics Across Canada, 2003. Alberta, Canada: Athabasca University. http://stillcounting.athabascau.ca/index.php

Tutty, L. et al., An Environmental Scan of Programs that Address Domestic and Sexual Violence: What's Working in Alberta?, 2005 Online: http://www.ucalgary.ca/resolve-static/reports/2005/2005-02.pdf

Weiss, C. (1990). The role of intermediaries in strengthening women's self employment activities. In Local initiative for job creation: Enterprising women (pp. 59-74). Paris: Organisation for Economic Co-operation and Development.

World Economic Forum, The Global Gender Gap Report 2010, Switzerland, 2010.

YWCA Canada, Turning Point For Women Effective Practices In Sheltering Women Leaving Violence In Intimate Relationships, Phase II Report, 2006

Date modified: