Archived information

This content is archived because Status of Women Canada no longer exists. Please visit the Women and Gender Equality Canada.

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Archived

This site has been archived on the Web.

Planning and Reporting

Evaluation of the Women's Program

Volume I - Final Report

---

4.0 Findings

This section of the report summarizes the findings from all data collection methods.

4.1 Design and delivery

The overall design and delivery of the WP meet with the general approval of stakeholders. Stakeholders perceive the main strengths of the WP to be:

• Its flexibility - The parameters of the WP, including its eligibility criteria, objectives and priority areas, are sufficiently general to accommodate a wide diversity of groups, strategies, and issues. This flexibility enables the WP to respond to new and emerging issues and to support newer, smaller, non-registered charitable organizations, or organizations that may not have access to other sources of funding.
  
  
• Its social development approach - The WP supports long-term social change by supporting the capacity of women's groups and other equality-seeking organizations to participate more fully in Canadian society. In keeping with its social development approach, the WP assists organizations in developing their proposals for funding and provides them with various other forms of non-financial or technical assistance.
  
  
• Its presence in communities - As a result of the decentralized structure of the WP, Program Officers are physically located in communities across the country and can work directly with organizations in those communities. This enables the WP to respond to differing realities across the country.
  
  
• Its specific focus on women's equality - Perhaps most importantly from the perspective of stakeholders, the WP is the only funding program at the federal level that focuses specifically on advancing women's equality.

Nevertheless, the WP is also perceived as having several weaknesses, related mainly to its funding mechanisms, performance measurement system, and program management approach. The WP's strengths and weaknesses will be discussed in detail throughout the rest of this report.

4.1.1 Technical assistance

Through the technical assistance component, the WP provides various forms of non-financial assistance to women's groups and other equality-seeking organizations. As the primary vehicle by which the WP pursues its social development philosophy, the technical assistance component is widely seen by stakeholders as one of its major strengths.

Of the large majority of survey respondents who had received technical assistance other than assistance with proposal development and preparation (which is treated separately below), 65% characterized this assistance as very useful, explaining that:

• Staff were always available and/or approachable.
  
  
• The information and resources provided were useful.
  
  
• The assistance helped them to conceptualize the work of their organization.
  
  
• They received valuable referrals to other funding sources.
  
  
• The assistance facilitated the development of linkages and partnerships between their organization and others.
  
  
• Staff understand and support women's issues and/or the organization's work.

We're really fortunate in having the Women's Program to support our work. The staff person we work with is very helpful and provides lots of assistance.  It's very easy to access her when we need to talk to her. She also has a good understanding of the issues (…).  (Case study participant)

An additional 29% of these survey respondents characterized the technical assistance that their organization received as somewhat useful.

Similarly, applicants who were interviewed agreed that these types of assistance are helpful and have enabled them to either broaden their perspective, access funds from other sources, or operate more effectively and efficiently (i.e., saving them time and effort by connecting them to relevant information or to organizations doing similar work).  The most significant form of technical assistance provided by the WP is, without question, the assistance provided by program staff and managers during proposal development. The proposal development and approval process is discussed in detail below.

4.1.2 Proposal development and approval

Within the WP's social development framework, program staff and managers provide considerable direct assistance to organizations in developing their proposals for funding. Before embarking on proposal writing, prospective applicants are encouraged to discuss their ideas with staff to determine whether they fit within the parameters of the WP (sometimes, program staff proactively approach organizations with ideas for initiatives that fit these parameters). If initiatives fit the parameters of the Program, staff invite the organization to prepare a formal application.  Program staff and managers provide extensive assistance in preparing proposals for submission. A key consequence of this approach, as several applicants pointed out in interviews, is that formal, written applications for funding are usually approved.  Among respondents to the survey of applicants, 21% reported having ever submitted a written request for funding that was not approved.

Program staff and managers believe that this approach to proposal development is an effective risk management strategy, since it enables the WP to avoid funding poorly thought-out initiatives or organizations that lack capacity. However, they also observed that proposal development can be difficult and time-consuming, depending on the level of sophistication of the applicant organization. Moreover, they said, the process has become more protracted since the introduction of the outcome-based approach, with its emphasis on articulating a strong rationale, defensible objectives, and clear outcomes for all funded initiatives.16

Applicants who were interviewed expressed similar views. A minority characterized the process as onerous, arguing that the outcome-based approach is an abstract and demanding conceptual model that can be difficult for many organizations, particularly small or inexperienced ones, to grasp.17 While some acknowledged the value of the model, they also maintained that its complexity favours the more established and sophisticated organizations. However, most said that proposal development, although time-consuming, is relatively straightforward, which they attributed to the assistance provided by staff and to the knowledge they have gained from their own past experiences in developing proposals for the Program.

The staff at SWC were very helpful (…) and assisted us in understanding the process and the information we needed to include in the proposal so that we could frame the logic model in a way that met the needs of the association and SWC, so that we could better plan, evaluate and monitor the project. (Case study participant)

Results from the survey of applicants likewise indicate that most respondents have positive views of the proposal development process. About 80% agreed that during proposal development, their organization was required to demonstrate that it had involved a full diversity of women affected by the issue in the initiative; that the eligibility criteria for funding are clear; and that WP staff were clear in communicating expectations about proposal requirements. Almost three-quarters agreed that the proposal development process helped their organization to clarify the objectives of their initiative, while just over two-thirds agreed that they understand the criteria used to assess proposals for funding and that the proposal development process helped their organization to identify appropriate strategies for their initiative. However, 61% said they understand how funding requests are approved.  See Table 4.

able 4: Survey respondents' views on proposal development and approval process (n=215)

Table 4: Survey respondents' views on proposal development and approval process (n=215)

	Percent in agreement
The eligibility criteria for funding are clear.	79%
The proposal development process helped our organization to clarify the objectives of our initiative.	73%
The proposal development process helped our organization to identify appropriate strategies for our initiative.	68%
During proposal development, our organization was required to demonstrate that we had involved a full diversity of women affected by the issue in our initiative.	80%
Women's Program staff were clear in communicating expectations about proposal requirements.	78%
I understand the criteria used to assess proposals for funding.	69%
I understand how funding requests are approved.	61%

Source: Survey of women's groups and other equality-seeking organizations.

Of the 44 survey respondents who had submitted a written request for funding that was ultimately declined, two-thirds (64%) agreed that program staff made it clear why their organization's request for funding was not successful, while about 40% agreed that staff provided information on alternate sources of funding for their initiative, and only one-quarter agreed that they were informed of how to have the decision on their funding request reviewed. These findings suggest a need for greater transparency and increased communication with organizations during and after the assessment process, particularly when it comes to proposals that are declined.

Across all groups that participated in the evaluation, stakeholders' suggestions for improving the proposal development and approval process include:

• providing training in proposal development and the outcome-based approach
  
  
• providing more staff assistance
  
  
• simplifying and making the proposal development process more flexible
  
  
• clarifying the information available in program documentation and on the WP's web site
  
  
• expediting the grant approval process by, for example, introducing and adhering to a standard processing time for applications.18

4.1.3 Financial assistance

In 1998, following consultations with women's groups and other equality-seeking organizations, the WP made several major changes to its funding mechanisms, replacing program funding and project funding with initiative funding, and introducing 18-month and multi-year funding. The mechanism through which the WP disperses funds to women's groups and other equality-seeking organizations remains a controversial and widely debated subject. In fact, the new House of Commons Standing Committee on the Status of Women, created in November 2004, has recently concluded that the issue of gender equality has receded from the public policy agenda over the last decade and a half; that over the same period the WP's funding (in current dollars) has decreased; and that the absence of program funding from the WP has weakened the women's movement in Canada.19  The Committee's position includes a recommendation that funding to the WP be increased immediately by at least 25%20 and a recommendation that SWC introduce a mix of core funding and project funding for the WP.21

Given the ongoing attention that the subject is receiving in the public policy arena, the impact of the 1998 changes to the WP's funding mechanisms were a significant issue in this evaluation.

Termination of program funding

The termination of program funding was, and remains, the most controversial of the 1998 changes. On balance, most program staff and managers support this change, although they recognize that it may have had detrimental consequences for some organizations. Almost all observed that the termination of program funding has had the positive effect of increasing equity of access to WP funding, which was an important part of the original rationale for the change. Program staff and managers explained that whereas program funding benefited a relatively small number of organizations, its removal and the introduction of initiative funding have enabled organizations that previously were unable to access funds to do so, and some organizations have made significant gains under the current funding arrangements. Moreover, key informants said, all organizations are now on the same playing field, since under initiative funding, the requirements for proposal development and grant approval apply equally regardless of an organization's size, history, or mandate.  It is important to note that program funding was evidently not an equally important funding tool in all regions. Where this mechanism was used infrequently, the impact of its removal was minimal.

At the same time, program staff and managers also acknowledged that some organizations, especially national organizations with a broad mandate to advocate for women's equality, felt weakened and destabilized as a result of the termination of program funding.22  Program funding, unlike initiative funding, could be used by these organizations to cover their operational costs, thereby affording them the necessary stability to pursue their advocacy work as new issues emerged in a context of environmental and political change. Many program staff and managers reported that the termination of program funding was interpreted by many of its prior recipients as symbolizing a lack of ongoing federal commitment to equality work. They said that it has taken a significant amount of time for the WP to rebuild its relationship with these organizations, and that some organizations have not adjusted to or accepted the change.

Survey results corroborate the impressions of program staff and management, to the extent that they demonstrate a lack of consensus among applicants regarding the termination of program funding.  Overall, among funded respondents (n=193) 13% experienced the termination of program funding as a positive change and 22% experienced it as negative. However, 26% thought the change had no impact on their organization and 39% either did not know or gave no response. Together, the latter two categories represent fully two-thirds of all funded respondents (see Table 5).

Table 5:   Survey respondents - Impact of termination of program funding by prior program funding status (n=193)

	All funded respondents (n=193)	Previously program- funded (n=66)	Never program- funded (n=88)	Don't know/no response* (n=39)
Positive change	13%	14%	11%	15%
Neutral	26%	33%	28%	8%
Negative change	22%	32%	14%	26%
Don't know/no response	39%	21%	47%	51%
Total	100%	100%	100%	100%

Source: Survey of women's groups and other equality-seeking organizations.
*In response to a series of questions on prior funding history, a total of 39 funded respondents either indicated that they did not know whether they were previously program funded or did not answer the question. As a result, we do not know whether they ever received program funds.

What is perhaps more unexpected is the degree of ambivalence that exists even among prior recipients of program funding. As Table 5 shows, previously program-funded applicants were more likely than others to see the termination of program funding as a negative change - 32%, compared to 14% of those who never received program funding and 26% of those whose prior funding history is unknown. However, prior recipients of program funds are far from united in viewing the termination of program funding as detrimental. In fact, while one-third (32%) said the change was negative for their organization, the same proportion (33%) said it had essentially no impact, and one-fifth (21%) did not know or did not respond. Furthermore, there was little difference in the proportion of respondents in the three groups who thought the change was positive.

In both interviews and their survey responses, applicants who see the termination of program funding as a negative change argued that women's organizations need stable funding in order to function effectively. From their perspective, its absence means difficulties in retaining permanent staff, difficulties in maintaining office infrastructure, discontinuation of certain organizational activities, inability to plan ahead, inability to respond to new issues as they arise, and "mission drift."  Some also objected to the more burdensome application process and the need to tailor their activities to meet the priorities of the WP.  On the other hand, applicants who believe the termination of core funding was a positive change explained that change-oriented strategies are positive and/or more responsive to women's needs; that the result was increased visibility and capacity for their organization; that they experienced increased cooperation and interaction with other groups; and that they benefited from a wider perspective or vision. 

Taken together, these results show a lack of consensus within the WP's constituency on the question of program funding. Evidently, program funding is seen as crucially important by a portion of this constituency, the nature of whose work may not be easily accommodated by initiative funding, but is seen as less important by others. Indeed, in response to an open-ended survey question on how the WP could be improved, more applicants recommended increasing the overall budget for the Program (24%) than recommended restoring program funding (19%).  These were the two most frequently mentioned suggestions for improvement.  All others were mentioned by 8% or less of survey respondents.  In interviews, several applicants emphasized a need for both types of funding, arguing that program funding is necessary for organizations with permanent employees and a broad advocacy mandate, while initiative funding is well-suited to organizations without permanent staff and those with a narrower mandate or focus.

Among program staff and managers, there was little support for restoring program funding, although many identified a need for longer-term funding that would enable organizations to plan several years in advance, to respond to their needs with respect to organizational development, and to devote less time and fewer resources to the application process, while still requiring them to define precise objectives and undertake rigorous evaluation. It was suggested, for example, that five-year funding be available for more experienced, established groups, and those with a longer relationship with the WP.

Introduction of initiative funding

Many key informants consider the termination of program funding and the introduction of initiative funding as two sides of the same coin, and after giving their opinion of the impact of the former, most did not offer any further commentary when asked about the latter. Program staff and managers reiterated the view that equity has improved as the number of organizations that have access to WP funding has increased, although some previously program-funded organizations have had difficulty adapting to the current funding arrangements.

Several program staff and managers also observed that initiative funding is better able than program funding to assist organizations in the longer term, since it is more focused and clearer in its expectations and therefore more likely to produce and assist in identifying results. From their perspective, accountability was poor under program funding, since recipients were not required to justify their receipt of government funds, to submit any kind of strategic plan, or to demonstrate progress toward or success in achieving some goal.  These areas have improved substantially with the introduction of initiative funding in 1998.  However, others argued that the introduction of initiative funding has not necessarily resolved the WP's accountability issues, and that the Program might have had more success at improving accountability had it identified organizations that were not performing under program funding and developed a strategy for dealing specifically with them.  

Table 6 shows how funded respondents regard the introduction of initiative funding.  Overall, 34% of these respondents believe that initiative funding was a positive change, while 13% said it was negative. Almost one-quarter (24%) were neutral, and 30% did not know or gave no response.  Although previously program-funded respondents were more likely than others to characterize the introduction of initiative funding as a negative change or to be neutral on this question, this can be explained to a considerable extent by the much larger proportion of respondents in the other two categories who either did not know or did not provide a response. Indeed, as was also the case with respect to the termination of program funding, similar proportions of respondents in all three categories said the change was positive.

Table 6:   Survey respondents - Impact of introduction of initiative funding by prior program funding status (n=193)

	All funded respondents (n=193)	Previously program- funded (n=66)	Never program- funded (n=88)	Don't know/no response* (n=39)
Positive change	34%	30%	35%	36%
Neutral	24%	36%	22%	10%
Negative change	13%	24%	5%	10%
Don't know/no response	30%	9%	39%	44%
Total	101%	99%	101%	100%

Source: Survey of women's groups and other equality-seeking organizations.
Note: Totals may not sum to 100% due to rounding.
*In response to a series of questions on prior funding history, a total of 39 funded respondents either indicated that they did not know whether they were previously program funded or did not answer the question. As a result, we do not know whether they ever received program funds.

In interviews and their survey responses, applicants who see the introduction of initiative funding as a negative change explained that the new approach entails a heavier administrative burden; that they are unable to do long-term planning; that initiative funding is too inflexible and/or required them to change the nature of their work; and that it is difficult to continuously develop new and innovative initiatives.  Those who see initiative funding in a positive light most often argued that it results in more focused, coherent, or complex projects, and that it enabled their organization to be funded.

Introduction of 18-month and multi-year funding

Virtually all program staff and managers consider the introduction of 18-month and multi-year funding to be positive developments. They said that for staff, 18-month funding entails less work, although the same is not necessarily true of multi-year funding, which requires more attention to risk management and more upfront research to assess proposals.  For organizations, both 18-month and multi-year funding permit greater flexibility, require less paperwork, and allow more time to achieve outcomes. To a certain extent, according to program staff and managers, 18-month and multi-year funding compensate for the termination of program funding.

Survey results show applicants to be generally supportive of these changes, although it is also clear that some lacked the experience with these types of funding that would enable them to form an opinion.  As shown in Table 7, 61% of respondents said the availability of 18-month funding is a positive change, while 45% said the same about multi-year funding. In both cases, organizations that had received these types of funding were more likely than others to view them positively.  In addition, significant proportions of survey respondents could not comment on the impact of the changes, likely because they had never received these types of funding themselves.

Table 7: Survey respondents - Impact of availability of 18-month and multi-year funding (n=193)

	18-month funding %	Multi-year funding %
Positive change	61%	45%
Neutral	15%	18%
Negative change	1%	1%
Don't know/no response	23%	36%
Total	100%	100%

Source: Survey of women's groups and other equality-seeking organizations.

In interviews and their survey responses, applicants echoed program staff and managers in identifying advantages of 18-month and multi-year funding. They argued that these longer-term types of funding are beneficial because they permit more time to develop and execute initiatives; make it easier to achieve outcomes; permit more complex and substantial initiatives; promote greater continuity and stability within their organization; and entail less administration and paperwork.

Overall, stakeholders identified very few drawbacks to either of these funding mechanisms. A small number of applicants cited the difficulty of predicting what issues will be relevant to the women's movement three years in advance and the possibility of being "stuck" working on outdated issues should they gauge the issues incorrectly, and argued that organizations have to balance the need for longer-term funding against the need to remain relevant.23 Program staff and managers said that multi-year funding is limited by its eligibility criteria, which exclude organizations without a history of funding with the WP, and by its inconsistent use across the country. While some regions use multi-year funding quite often, others use it only minimally and one region, not at all; the latter reportedly because of the difficulty of predicting the environmental and political context three years in advance. Suggestions for addressing these limitations included making multi-year funding available to less-experienced organizations and ensuring that it is equally available in all regions.  In relation to the need for multi-year funding and the difficulty associated with predicting issues, a key informant who participated in a case study indicated that:

Modifying public perceptions and influencing policy-making requires long-term efforts, as visible results in terms of influencing public policy can take a long time to materialize.   (The organization) would like to see funding mechanisms better adapted to the reality of the work required to influence policy.  (Case study participant)

4.1.4 Outcome-based approach

The most recent change to the WP occurred in April 2003, when the Program implemented a new outcome-based approach with an emphasis on highlighting outcomes achieved. Important elements of the new approach included an accountability framework, a logic model, a performance measurement strategy, and a new suite of instruments and forms. Training was provided to managers and staff on the new approach three months prior to its implementation, and workshops were held a year later to familiarize them with a new tool kit to support the implementation of the approach.  A key issue for this evaluation was to determine the extent to which the outcome-based approach has been implemented and to identify areas where improvements are necessary. Overall, the evaluation found that while stakeholders generally see the outcome-based approach as a positive development and some progress has been made toward its implementation, it is not without shortcomings and would benefit from a number of improvements.

Program staff and managers observed that the new approach resulted in a difficult period of adjustment for all involved as program staff, managers, and applicants became familiar with the concepts and language, and despite a more complex and protracted proposal development process, has produced more focused, strategic, and effective initiatives. Similarly, a majority of applicants who were interviewed reported that implementation of the outcome-based approach either had a positive impact on or made little difference to their organization. Several specifically praised the outcome-based approach for helping to keep their initiatives focused on objectives and concrete outcomes.

As table 8 indicates, the survey of applicants shows that despite some success at implementing the outcome-based approach, there is clearly room for improvement. Two-thirds of funded survey respondents reported that WP staff have clearly explained the outcome-based approach to their organization, while the same proportion said that assistance from WP staff has helped them to identify realistic outcomes and indicators. Almost 60% agreed that the new Application Form and Guide are easy to understand and that the Interim and Final Reports are easy to use. However, less than half agreed that they are able to plan more effectively as a result of the outcome-based approach and that the outcome-based approach has helped them to demonstrate the results of their work. This suggests that applicants may not be fully at ease with the outcome-based approach nor convinced of its benefits as an aid to strategic planning and reporting.

Table 8: Survey respondents' views on outcome-based approach (n=193)

	Percent in agreement
Women's Program staff have clearly explained the outcome-based approach to our group.	67%
Assistance from Women's Program staff has helped our group to identify realistic outcomes and indicators.	65%
As a result of the outcome-based approach, our group is able to plan more effectively.	49%
The outcome-based approach has helped our group to demonstrate the results of our work to our members, to the community at large, and/or to other funders.	49%
The Application Form and Guide are easy to understand.	56%
The Interim and Final Reports are easy to use.	57%

Source: Survey of women's groups and other equality-seeking organizations.

More broadly, it is evident from the results of this evaluation in general that successful implementation of the outcome-based approach is hampered by a number of significant obstacles.

1) Lack of understanding of the outcome-based approach and related concepts among both program staff and applicants

Perhaps most importantly, neither applicants nor all program staff and managers have fully grasped and embraced the outcome-based approach and its related concepts, terminology, and requirements. Many key informants observed that the outcome-based approach is an abstract and demanding conceptual model that has been challenging for some applicants, especially small organizations that lack resources and experience, to understand and apply. Moreover, some applicants and external stakeholders also believe that the WP itself does not fully understand the outcome-based approach, an observation that was corroborated by program staff and managers themselves, who acknowledged that there continues to be substantial debate within the Program regarding the concepts and terminology.

I like this approach but am uncomfortable with how WP has understood it. I have to change my results to fit what they believe are results.  It would be helpful if there were more consistency among the departments in the government on what is result-based management. (Participant in key informant interviews)

Results from the file review lend credence to these impressions. The file review found that crucial terms associated with the outcome-based approach are often used inconsistently, and indeed interchangeably, on the part of both applicants and program staff. As a result, the activities, objectives, and expected outcomes of a given initiative are seldom readily apparent. For example, the review found that despite some progress since implementation of the outcome-based approach in 2003-2004, it is not uncommon for applicants to identify different objectives and expected outcomes in their initial applications and final reports.

The problem, moreover, is not limited to applicants. The file review found that recommendations for approval frequently contain different objectives and expected outcomes than original proposals.24 It is not clear whether this pattern is indicative of misunderstanding on the part of program staff, or an attempt to clarify the logic of the proposed initiative.  However, it seems obvious that notwithstanding the social development approach, proposals should not proceed to the recommendation for approval stage until they articulate objectives and expected outcomes in a satisfactory way, and that once a proposal is deemed acceptable, all documents pertaining to a given initiative should articulate the same objectives and expected outcomes for that initiative.

All of these findings suggest that the outcome-based approach continues to be problematic for both applicants and program staff and managers. When asked what the WP could do to make it easier for them to apply the outcome-based approach, survey respondents most commonly suggested in-person workshops, training, or seminars in the outcome-based approach, and additional examples, guidance, and one-on-one support and assistance during the proposal development process.

2) Lack of a strategy for analysing and reporting on program outcomes

Although the WP has developed an accountability framework, logic model and reporting forms, a second major obstacle to successful implementation of the outcome-based approach is the absence of any strategy or tool for analysing and reporting on program impact. While the WP collects a considerable amount of information via the various forms it introduced in 2003-2004, at present it does not "roll-up" or synthesize these data, at either the national or the regional level. As a result, the Program is unable to report in any systematic way to Parliament and to Canadians on the results it has achieved.

Key informants reported that an analysis and reporting strategy is currently under development and will hinge on information contained within the standard forms, particularly the Final Report Form and the Close-Out Assessment Form. While the evaluation did not have access to the details of this strategy, it is clear that its success will depend on the forms first, being completed consistently and second, containing information that can be synthesized and analysed with relative ease.

With respect to the first point, the file review found that all 15 of the 2003-2004 files reviewed contained a Close-Out Assessment Form, while Final Reports using the standard form were included in just over half of the files. Where the standard Final Report Form was not used, a final report in an alternate format was included, since substituting the standard form with a final report prepared for another funder is permissible. However, in these cases, the final reports tended to describe the initiative rather than respond directly to the outcome-based questions posed by the standard form, clearly complicating the task of synthesis and analysis.

It is important to acknowledge here that the file review component of this evaluation was limited in the extent to which it could meet its second objective, that is, to examine the extent to which the outcome-based approach to reporting was being implemented and to identify any emerging issues that might benefit from further investigation or ongoing monitoring.  This was due to the small number of "closed files" available for review from the 2003-2004 fiscal year, and was a consequence of the short period between the implementation of the outcome-based approach and this evaluation.

With respect to the second point, the information provided on the Final Report Forms seems to consist largely of self-assessments by the funded organizations. Generally speaking, this type of information is difficult to validate, synthesize, and analyse without risking a significant loss of meaning. Given the inherent difficulties of measuring social change, several program managers questioned whether the WP is collecting useful and relevant outcome information, and recommended that the WP diversify its performance measurement approach by focusing on more objective qualitative data collection methods, as well as "quantitative" methods using the standard forms.25

3) Inadequate program database to capture information collected and facilitate reporting on outcomes

A third obstacle is the inadequacy of the current program database. It is not accessible to regional managers and staff, and it does not permit analysis of and reporting on activities and outcomes achieved at the regional level and across the Program as a whole. Key informants reported that it is outdated, accessible only at the national level, difficult to manipulate, and incapable of producing necessary or useful information.26

Many program staff and managers pointed out that a functioning program database is absolutely essential if the WP is to meet its performance monitoring and accountability obligations, particularly given the near impossibility of performing the necessary analysis manually. The development of such a database is under discussion as part of a larger information management project at the departmental level.27 However, there is a perception among program staff that progress toward it has stalled, which they attributed to lack of financial resources, technical capacity, and management commitment.

At the same time, however, several cautioned against over-reliance on a database solution, arguing that information generated in this way may risk superficiality or irrelevance if not understood in its proper context or as part of a larger whole. Again, these key informants advocated a diversified performance measurement strategy that uses qualitative as well as quantitative approaches.

4) Uncertainty over appropriateness of outcome-based approach to social development work

Finally, there is continuing uncertainty about the appropriateness of the outcome-based approach to social development work. In interviews and the survey, many applicants expressed concern that the WP has unrealistic expectations when it comes to measuring and reporting on results. Applicants pointed out that social change, as the ultimate goal of their work, is not easily measured in the way the outcome-based approach requires it to be. They observed that the sought-after outcomes of social development work are affected (sometimes negatively) by external factors, are frequently difficult to measure because of their intangible nature, and often cannot be realized, much less measured, in the short term - yet they remain a worthy target and the work to achieve them must be done. For these applicants, the Program's reporting expectations are not consistent with the nature of social development work.

Given the long-term nature of social development work, the difficulty of measuring many expected outcomes, and the limited funds the Program makes available, many applicants recommended that the WP modify its reporting expectations. For example, it was suggested that for reporting purposes, the Program should place less emphasis on measuring outcomes and more emphasis on documenting activities undertaken with WP funding  - and accept the assumption that if planned activities have been carried out, some progress toward outcomes has likewise been achieved.

Not all systemic change is easily measured in the way that the reports require them to be.  (…) I think there needs to be room for some assumptions in the reporting strategy. (Participant in key informant interviews)

4.2 Impact

Because it is a summative evaluation, a primary goal of this study was to determine the extent to which the WP has achieved its four objectives; namely, promoting institutional change, influencing public policy, increasing public understanding in order to encourage action on women's equality issues, and building capacity among women's organizations.  Unfortunately, the evaluation was limited in its ability to discern the overall impact of the Program in these areas.

The evaluation had very little administrative data available to it for analysis, since prior to implementation of the outcome-based approach in 2003-2004, the WP did not systematically collect information on outcomes achieved. Thus, although the evaluation reviewed a sample of program files with the objective of collecting outcome information, it was ultimately impossible to perform this analysis due to inconsistent identification of activities, objectives, and outcomes across documents within the files; missing documents; failure of final reports to establish a clear link to original applications; and failure of final reports to provide evidence to support assertions that outcomes had been achieved. Moreover, although the WP introduced standardized forms in 2003-2004 to collect information on outcomes, to date this information remains paper-based and therefore difficult to synthesize and analyse (in any case, at the time of data collection, there were only 15 closed files from that year). As a result of the poor quality of the outcome information in the earlier files, and the relative infancy of the outcome-based approach, the evaluation relies primarily on anecdotal information on program impact gained through key informant interviews, case studies, and the survey of applicants.

4.2.1 Achievement of WP objectives

The evaluation found a broad consensus among stakeholders that the WP has had a positive impact. Most key informants are convinced that the WP has helped to advance women's equality in Canada, and, as Table 9 shows, funded survey respondents overwhelmingly agreed that WP funding has enabled their organization to undertake initiatives it otherwise would not have been able to pursue (93%), and that WP funding has helped their organization to achieve some of its objectives in advancing women's equality (91%).

Table 9: Survey respondents' views on impact of the Women's Program (n=193)

	Percent in agreement
Women's Program funding has enabled our organization to undertake initiatives it otherwise would not have been able to pursue.	93%
Women's Program funding has helped our organization to achieve some of its objectives in advancing women's equality.	91%
Women's Program funding has helped our organization to leverage other sources of funding.	63%

Source: Survey of women's groups and other equality-seeking organizations.

It is considerably more difficult to pinpoint the extent to which the WP has achieved each of its four objectives.  Based on the survey results, the WP has made its greatest contribution towards increased public understanding; 37% of funded survey respondents reported increased awareness or understanding of women's issues and needs as a result of their initiative. Another 25% reported some type of increased civic engagement by women. This category, which includes involvement in institutional and public policy decision-making, advocacy efforts and politics, encompasses the institutional change, public policy, and capacity-building objectives of the WP.  Table 10 provides a complete summary of responses.

Table 10: Survey respondents - Thinking about what your organization has accomplished with Women's Program funding, what has changed as a result of those initiatives? (n=193)

Changes	%
Increased awareness/understanding of women's issues/needs	37%
Increased civic engagement by women (includes involvement in institutional and public policy decision-making, advocacy, and politics)	25%
Establishment of partnerships/networks/collaboration	13%
Organization's resources/tools are being used by other parties	12%
More women involved in a particular economic/industrial sector	8%
Increased organizational capacity/expertise/positioning	6%
Increased availability/accessibility of services for women	5%
Public policy/legislative change	5%
Increased self-confidence/self-esteem on part of women	4%
More women leaving abusive relationships/seeking help	3%
Increased financial independence/reduced poverty for women	2%
Other	9%
Nothing - initiative not yet concluded	3%
	11%
Source: Survey of women's groups and other equality-seeking organizations. Note: Total does not sum to 100% due to multiple responses.

Key informants, for their part, believe that the WP has made the most progress toward its public understanding and public policy objectives, and that progress toward institutional change and increased capacity of women's organizations has been more difficult to achieve. Speaking from their own experiences and personal observations, they gave numerous examples of changes relating to each of the WP's four objectives.

Promoting institutional change

Key informants believe that promoting institutional change has been one of the more difficult of the WP's objectives to achieve. They also find it difficult to measure progress toward achieving this objective.  Several key informants indicate their awareness of significant changes in various institutional policies toward increasing equality, but also indicate that very little is known about the implementation and impact of these changes.

Influencing public policy

Key informants believe that considerable success has been realized on the public policy front, and they cited numerous examples of changes that have resulted in part from the efforts of WP-funded organizations.  These include the federal government's recent announcement of the National Child Care Agenda, changes to maternity benefits and leave, the halting of the Secure Care Act in British Columbia, the introduction of the Victims of Domestic Violence Act in Alberta, successful lobbying for provincial funding for transition houses in Ontario, incorporation of indigenous women's issues into the Beijing +10 process, and increasing participation of Aboriginal women in Aboriginal government and self-government structures in Yukon and Northwestern Ontario. They cited other successes such as changes to the justice system response to family violence in some provinces, and introduction of harassment policies in educational, health, and correctional institutions.  Despite the fact that key informants believe that considerable success has been realized, only 5% of funded survey respondents said that public policy or legislative change occurred as a result of their initiative, while 25% reported increased civic engagement by women.

Using data gathered through court, police, and corrections monitoring, JFG, together with law students, an articling student, and an incarcerated young woman, is writing a practice guide that provides instruction in how to effectively advise and represent young women charged with criminal offences under the Youth Criminal Justice Act. Ultimately, the purpose of the guide is to decrease the incarceration of teenage girls by giving lawyers the tools they need to advocate effectively on their behalf. The guide is to be published in the coming year.  In another example, JFG and other community organizations successfully mobilized in opposition to the Secure Care Act, which would have allowed for the detention of young women without arrest. The Province of British Columbia has never enforced the Act.  (More details as to other activities and outcomes of selected initiatives are available in the interim report on case studies, in Volume II.)

Increasing public understanding

Key informants reported increased public understanding on a diverse range of issues as a result of WP-funded initiatives, including pay equity, sex trafficking, violence against Aboriginal women, sharia law, female genital mutilation, child care, the differential impact of budgets on women, poverty and welfare, and legal aid and family law issues pertaining to women. As already noted, according to the survey of applicants, increased public understanding was the most common outcome resulting from WP-funded initiatives, with 37% of funded respondents identifying this outcome.

Building capacity of women's groups

Key informants believe that the WP has realized limited success in building the capacity of women's organizations. In fact, some argued that women's organizations have suffered a "backward spiral" in this regard, which they attributed to the elimination of program funding. On the other hand, others reported increased capacity of a diverse range of women's organizations, representing sex workers, Aboriginal women, women with mental illness, and immigrant/refugee women's groups.  Some gave specific examples of ways in which the capacity of these organizations has improved, including establishment of organizations as formal coalitions and their participation in the women's movement, increased visibility of organizations, and increased ability to support other organizations. Based on the survey of applicants, increased capacity was a relatively common outcome of funded initiatives, with 13% of funded respondents reporting increased partnerships, networks, and collaboration as a result of their funded initiative, 12% reporting that other organizations are using tools they developed, and 6% mentioning increased organizational capacity, positioning, or expertise.

Overall contribution toward achieving WP objectives

Key informants observed that it is virtually impossible to pinpoint the net contribution of the WP in these areas, due to the equally significant role of other factors in influencing social change. The most important of these, they believe, is the political climate. Thus, while the WP supports women's groups and other equality-seeking organizations and contributes to advances to the extent that it can, progress is ultimately a matter of political will. As a result, change is often slow in coming and, more importantly, can be quickly erased.  Other factors thought to influence social change include the dedication and hard work of women's organizations; demographic changes; the state of the economy; the impact of globalization; media exposure; the contribution of other funders, donors, and supporters; the attitudes and actions of provincial governments; the number of women in positions of political power; the support of key politicians; and the existence of a critical mass of public understanding leading to pressure for action.

4.2.2 Achievement of AGE objectives

Since 2000-2001, the WP has received $2 million in additional funding per year as part of the AGE initiative. Within the WP, AGE funding is intended primarily to broaden the scope of organizations receiving financial support and to increase the Program's ability to support action on new and emerging issues. 

Administrative data from the WP showed that AGE funds have enabled the WP to support a total of 122 new organizations and 123 files involving a wide range of new and emerging issues. New issues include women in the fisheries industry, trade and globalization, sexual exploitation and sex tourism-trafficking, date rape/dating violence, reproductive rights, AIDS and infectious diseases, Aboriginal self-government, violence against Aboriginal women, women in technology and e-economy, sharia law, genital mutilation, extreme urban poverty, rural issues, and federal maternity and parental benefits. New organizations include those working on these issues, such as, but not limited to, Aboriginal groups, immigrant and ethnocultural groups, organizations involved with trade, technology, and/or globalization, and rural groups. According to key informants, one region has reportedly used the AGE to fund "mixed" groups (groups with both male and female membership), while another used AGE funds to support region-wide issues, which has helped to improve networking and collaboration among groups across the region.

However, several program staff and managers also noted that they could have funded these issues and organizations even in the absence of the AGE. These key informants reported that the purpose of the AGE was not explicitly identified when this funding was made available. Therefore, any successes in achieving AGE objectives were realized more by accident than by design. The additional funds simply made it possible for the WP to invest more money overall. 

4.2.3 Achievement of SWC strategic objectives

In 2004-2005, the strategic objectives of SWC were strengthened and more equitable public policy, a broader range of informed stakeholders, and increased departmental effectiveness. While most program staff and managers believe that the WP has contributed, although to varying degrees, to achieving a more equitable public policy and a broader range of informed stakeholders, they were divided on the matter of its contribution to departmental effectiveness. Several said that the WP has contributed greatly in this regard since the introduction of the outcome-based approach and the accountability framework, and a few argued that SWC would not achieve any of its strategic objectives were it not for the Program.

However, many program staff and managers were more ambivalent, reporting that the WP has a poor relationship with the rest of SWC. According to them, the WP and the rest of SWC work in isolation from one another and do not communicate or collaborate effectively. In short, according to these key informants, the WP is not well integrated into the agency, communications with other directorates within the agency are poor, and the WP lacks influence on departmental decision-making - including influence on the development of the agency's strategic objectives. There is a widespread perception that the root of the problem is a fundamental philosophical difference regarding the best way to achieve women's equality. The WP uses a social development approach, but this is not, nor reportedly has ever been, the approach taken by SWC. These key informants emphasized the need for improved communications and collaboration among all of the directorates within SWC. 

4.2.4 Unexpected impacts

The evaluation did not find evidence of significant unexpected impacts resulting from the WP, either positive or negative. Among funded survey respondents, about half reported that their initiative had unintended positive consequences, such as the development of new or unexpected partnerships or allies; use of the organization's resources by other groups; increased visibility or credibility for the organization; increased membership in the organization; and unexpected political mobilization of women. Key informants gave similar examples. However, it should be noted that these impacts are not, strictly speaking, unexpected, given the objectives of the WP. Only 9% of funded survey respondents reported that their initiative had unintended negative consequences.  The latter were too few and divergent to be summarized.

4.3 Cost-effectiveness/alternatives

4.3.1 Efficiency of program delivery

The evaluation explored several aspects of program delivery to determine whether the WP is delivered efficiently, including the proposal development and approval process, the community's access to program staff, staff access to adequate technologies, and staff supports.

Proposal development and approval

In 2003-2004, the WP spent roughly $11 million on grants and contributions,28 $2.8 million on salaries, and $562,000 on overhead and management costs.29 This translates into an administrative efficiency ratio of approximately 31¢ per dollar awarded (i.e., the administrative cost to deliver one dollar of funding is 31¢).  Since it is not possible to distinguish among various administrative costs, the ratio does include costs likely associated, in large part, to the WP's social development framework. Within this framework, program staff provide considerable assistance to organizations in developing their proposals for funding as well as other forms of technical assistance, with the goal of building the capacity of women's organizations and ultimately strengthening the capacity of the women's movement as a whole.  In addition, we note:

• The complexity of women's equality issues and the social change process can contribute to making the proposal development process more time consuming.
  
  
• Marginalized organizations and/or new and emerging organizations can require more assistance.
  
  
• The decentralized delivery model of the WP, by providing greater accessibility, can prove more costly.
  
  
• Technical assistance can be provided to groups who never receive funding.
  
  
• Regional staff are also responsible for other SWC duties, in addition to the delivery of the WP, and it is not clear what portion of the costs associated with these other duties is included with the administrative costs of the WP.

Given the social development framework and, more recently, the additional requirements of the outcome-based approach, the need for efficiency in program delivery cannot easily be separated from the question of effectiveness. Many program staff and managers emphasized that while the WP's approach to proposal development is complex and time consuming, it is also an effective risk management strategy that enables the WP to avoid funding poorly thought-out initiatives and organizations that lack capacity. That being said, stakeholders gave two main suggestions for improving the efficiency of the proposal development process: simplifying the process so that it is less complex and time consuming, and providing workshops and training in proposal development and the outcome-based approach to applicants so that they are better positioned to prepare proposals without extensive staff assistance. 

With respect to grant approval, applicants expressed concern that it is often subject to long delays, which they said can have negative consequences for organizations that rely on WP funding. Applicants observed that, compared to other funding sources, the Program's proposal and reporting requirements are more detailed and rigid and the proposals and reports require more time to prepare; however, the amount of funding provided by WP is generally larger.  Program staff and managers acknowledged that the process is less expeditious than applicants would like and suggested that the Program introduce and adhere to a standard processing time for applications.

Staff access to adequate technologies

Lack of staff access to adequate technologies emerged as a major issue in this evaluation. As already discussed, program staff and managers emphasized the need for a program database, accessible to regional staff, to capture data and facilitate data analysis and reporting on outcomes. In addition, program staff had numerous other concerns regarding their access to appropriate technologies, including poor or merely adequate computers; lack of technical support at the regional level; lack of cellular telephones and laptops for staff use while travelling; poorly designed electronic forms; under-utilization of the WP's Intranet; and excessive reliance on in-person meetings at the management level when technology-based approaches would also work.

Community's access to staff

From the perspective of program staff and managers, WP staff are as responsive and accessible as possible given their numbers and financial resources. Staff from several regions stressed the importance of in-person contact with organizations in their own communities and said that they lack the personnel, as well as the travel budget, to serve all communities and groups within their region equally (although others emphasized other ways of ensuring accessibility through, for example, 1-800 lines and selective travel). Indeed, program staff and managers commonly cited insufficient staff resources as a weakness of the WP. However, survey results do not support the notion that the WP's constituency considers access to program staff to be problematic.

Staff supports

Program staff and managers reported that efficient program delivery is hampered by poor internal communications and limited information-sharing among the regions and the national office. To a great extent, they reported, WP staff "work in silos" and are unaware of the work being done by their colleagues in other regions and even the way the WP operates in other regions. There was widespread agreement that the Program needs a mechanism for effective, timely communications among staff and managers in the national and regional offices.

Finally, as already noted, there is a widespread perception among program staff and managers that senior management within SWC does not truly understand, value, and support the WP and the social development approach and that, as a result, the WP is not well integrated into the agency. Key informants identified the need for a more supportive and collaborative relationship among all of the directorates within SWC, arguing in particular that the grassroots work funded by the WP needs to be better connected with and "feed into" the agencies' policy development processes.

4.3.2 Duplication

There is a general consensus among stakeholders that the WP does not duplicate any other programs to advance women's equality in Canada, primarily because very few similar programs exist. Some examples include Canadian Heritage's Aboriginal Women's Program, Québec's Secrétariat à l'action communautaire autonome and À égalité pour décider, and, in some provinces, women's directorates and advisory councils. Key informants consider the WP to be unique in providing financial support for advocacy efforts, in providing a high level of technical assistance, in addressing the full range of equality issues, and in its specific mandate to advance women's equality. However, although key informants do not believe that the WP duplicates any other programs, several said that it can be perceived to work at cross-purposes with other federal programs because it funds organizations to influence government policy and legislation.

WP funding varies in importance for applicants who were interviewed, representing between 2% and almost 100% of their total funding. While some noted that WP funding is critical to the survival of their organizations, others said that its importance has declined, particularly since the termination of program funding.  Applicants observed that WP is the only funder focusing specifically on advancing women's equality. Like other key informants, applicants did not identify any duplication with other programs.

4.3.3 Alternative delivery models

Key informants had few suggestions for alternative delivery models, arguing that generally speaking, the model used by the WP is appropriate. Several pointed out that determining an appropriate delivery model is a very complicated matter, particularly given the diversity of groups that form the Program's constituency, the Program's limited resources, and the imperative, not likely to disappear in the foreseeable future, to demonstrate results to Parliament and to Canadians. Others said that none of the alternatives they knew of would constitute an improvement.  The most common recommendations were:

• Contribution funding - A few key informants recommended that the WP consider the use of contribution agreements in certain specific circumstances, and in particular, for longer-term and more complex initiatives. These key informants argued that while grants are appropriate for short-term initiatives, larger initiatives could benefit from the greater degree of collaboration and shared responsibility that contribution agreements entail. However, opposition to contribution agreements was also expressed on the grounds that they do not allow recipients sufficient flexibility.
  
  
• More sustained or longer-term funding - Key informants recommended various forms of longer-term funding, including the reintroduction of program funding and the extension of multi-year funding to a maximum of five years from the current three.  Among survey respondents, 19% recommended restoring program funding, while another 3% recommended making more multi-year funding available.
  
  
• Community-based delivery model - A few key informants suggested that the WP consider the use of a community-based model of program delivery. In such a model (key informants gave the model used by the National Homelessness Secretariat as an example), a committee or working group of community representatives makes recommendations for funding and administers the program. The perceived benefits of this approach include regular consultation with the community and an arm's-length relationship with the political process. On the other hand, some key informants were opposed to community-based delivery, observing that where this model has been used, it has been subject to conflicts of interest. These key informants believe that government delivery is more transparent and equitable.
  
  
• Social Development Canada's Social Development Partnership Program - Finally, a few key informants recommended that the WP consider a model similar to Social Development Canada's Social Development Partnership Program, which offers both program and project funding in relation to specific competitions.

4.3.4 Alternative delivery agents

The evaluation found little support for transferring the WP from SWC to another delivery agent.  Key informants emphasized the importance of maintaining the Program within the federal government, given its commitment to and responsibility for gender equality under the Canadian Charter of Rights and Freedoms. Moreover, although a few key informants speculated about the possibility of relocating the WP to another federal department such as Social Development Canada, on balance the vast majority was adamant that the Program be retained within SWC, primarily on the grounds that this is the only federal organization with a mandate to advance women's equality. In fact, several pointed out that the Program was transferred to SWC in the first place to consolidate within a single organization the various federal bodies that were pursuing women's equality and, ultimately, to encourage "synergies" and permit collaborative efforts to flourish. Nonetheless, as noted elsewhere in this report, key informants emphasized the need for the WP and SWC to resolve their outstanding differences so that the two may work effectively together. For these key informants, the synergies and collaboration anticipated to result from the relocation of the WP to SWC have not yet materialized.

4.3.5 Other options

Key informants emphasized the ongoing need for the WP, but also identified other strategies. Most often, they identified a need for SWC to have greater visibility and a stronger role within the federal government. Currently, the organization is small, has limited financial resources, and, as a federal agency under the Department of Canadian Heritage, lacks autonomy and influence.  As a result, SWC is limited in the extent to which it can support the WP  - be it in terms of funding, visibility or clout - and in the extent to which it can advance women's equality more generally.

These key informants said that ideally, a stand-alone ministry of women's equality should be established at the federal level with a coordinating role with respect to gender equality issues. Related to this point, many key informants argued that SWC should be responsible for monitoring the implementation of gender-based analysis across the federal government and for holding departments and ministries accountable for its implementation.

4.3.6 Value for money

Key informants were unanimous that the WP is giving Canadians value for their tax dollar, observing that the WP encourages community-based organizations to work on issues that are important to Canadian society "on a shoestring," while relying heavily on volunteer contributions of time and effort. In short, key informants characterized the WP as an under-funded program that has accomplished exceptional results despite a limited budget. That being said, it bears repeating that a significant portion of the WP's budget goes toward administrative costs, including the provision of technical assistance.

4.4 Relevance³⁰

Among the WP's stakeholders, there is widespread agreement that the WP is still relevant to advancing women's equality in Canada because, despite progress in some areas, women's equality has not yet been achieved. In fact, many key informants argued that the WP is now more relevant than ever, as new and complex issues emerge in a social and political context that some characterized as increasingly indifferent to women's equality.  Several key informants pointed out that the Program is particularly relevant to the most marginalized women in Canada, including Aboriginal women, women in the sex trade, immigrant and refugee women, and women in rural, northern, and isolated areas.

On the other hand, key informants also suggested that, for fear of controversy, the Program has become overly cautious in its funding decisions (i.e., in the types of organizations and issues that it funds) and therefore risks irrelevance. This impression was corroborated, to some extent, by results from the applicant survey. While 90% of respondents agreed that the WP is still relevant to advancing women's equality in Canada, considerably fewer believe that the Program responds well to new emerging issues (58%) and responds well to new and emerging groups (51%) (see Table 11).

Table 11: Survey respondents' views on relevance of the Women's Program (n=215)

	Percent in agreement
The Women's Program responds well to new and emerging groups.	51%
The Women's Program responds well to new and emerging issues.	58%
The Women's Program is still relevant to advancing women's equality in Canada.	90%
Source: Survey of women's groups and other equality-seeking organizations.

Survey respondents and key informants predicted detrimental consequences for women's organizations and women's equality in general if the WP did not exist. They argued that some women's organizations would disappear, while many others would curtail their activities or change their mandate (by, for example, becoming service providers only); the diversity of women's voices would be diminished; the federal government would lose the mechanism that connects it to the grassroots women's movement; the women's movement as a whole would lose coherence and effectiveness; and, ultimately, the advancement of women's equality in Canada would slow, stall, or possibly even regress. In short, given the absence of other significant players in the field, stakeholders consider the WP to be of critical importance to the health of the women's movement and to the advancement of women's equality in Canada.

4.4.1 Alignment

Program staff and managers were divided on the question of the WP's alignment with federal priorities. Some believe that the WP aligns well with these priorities, citing examples such as addressing issues involved in violence against Aboriginal women, child care, trafficking in women, social capital, and democratic processes. However, many questioned the implicit assumption that the WP should align with federal priorities. They pointed out that the Program's mandate is to promote women's equality and that, quite often, this means supporting groups to challenge the status quo - that is, in terms of public policy and institutional decision-making. For these key informants, the crucial question is the extent to which the federal government is aligned with the priorities of the WP and women's perspectives are integrated into federal policy - and not the reverse.

Similarly, program staff and managers had differing opinions on the extent to which the WP aligns with the strategic objectives of SWC. While a few said that the WP aligns well with the agency's strategic objectives, others said that the strategic objectives are vague and do not reflect the priorities of the grassroots women's movement. These key informants argued that the WP should be consulted in the development of the strategic objectives, and in departmental planning processes more broadly, to ensure that community, local, and regional concerns are appropriately reflected.

---

16 The outcome-based approach is discussed in detail in Section 4.1.4.

17 Some also acknowledged that this is not necessarily inappropriate or different from other government funders.

18 The timeliness of the grant approval process was an issue for some applicants, who said that long delays can have negative consequences for groups that rely on WP funding, and program managers and staff acknowledged that the process is often less expeditious than groups would like. Since it was beyond the scope of this evaluation to compare the WP's application processing time with that of other federal grant and contribution programs, the evaluation cannot comment on the reasonableness of the time the WP takes to process applications.

19 Increasing Funding to Equality-Seeking Organizations (February 2005) and Funding Through the Women's Program: Women's Groups Speak Out (May 2005), House of Commons Standing Committee on the Status of Women.

20 Increasing Funding to Equality Seeking Organizations, p.4, House of Commons Standing Committee on the Status of Women, February 2005.

21 Funding Through the Women's Program: Women's Groups Speak Out, p.8, House of Commons Standing Committee on the Status of Women, May 2005.

22 In one region, the main recipients of program funding were not national groups but remote and/or rural women's centres.

23 Note that multi-year funding is not necessarily three years in duration, but is the term used to describe funding that is between 18 and 36 months in duration.

24 The same could be said of Close-Out Assessment Forms, although these may legitimately contain different objectives and expected outcomes because of an approved revision to the original objectives.

25 Strictly speaking, the information collected via the standard forms is not quantitative data but rather qualitative assessments by organizations and program staff and managers.

26 These impressions were corroborated in various ways throughout the evaluation. For example, the WP provided the evaluation team with basic administrative data derived from their grant and contribution database, but more detailed requests for data were not put forward since the team was told that any data extracts would be difficult and time-consuming to produce.  Similarly, the WP could not produce an up-to-date electronic database or spreadsheet of all organizations that had applied for funding (whether funded or not) within a specified time period, along with their contact information, for the purpose of the survey sample.

27 In 2004, a Database Working Group was established to study the WPdatabase requirements and how to move from a system that collects mostly administrative data to one that will collect and analyse outcome information from the Close-Out Assessment Forms.  The Database Working Group recommended that SWC purchase the GCIMS database from Justice Canada (Grants and Contributions Database Analysis, Women's Program Database Working Group, May 6, 2005).

28 Women's Program: Accountability Framework, p.3.

29 Administrative data provided by Status of Women Canada (P. McInnis, June 6, 2005).

30 According to the Treasury Board Secretariat of Canada: "Relevance issues might include whether the policy, program or initiative is the most appropriate response to an identified need. There may also be issues around whether the identified need which led to the implementation of the policy, program or initiative has changed."  http://www.tbs-sct.gc.ca/eval/pubs/RMAF-CGRR/RMAF_Guide_e.pdf , p.23 (August 17, 2005).

---