Stronger Families and Communities Strategy National Evaluation

Questions and Answers

National Evaluation and Data Guidelines

Q: Will local evaluators have access to any of the national level data collected?
A: Aggregated data about each site will be made available to the site's Facilitating Partner.

Q: Are there any guidelines for sites and community partners to follow when collecting data that will contribute to the National Evaluation?
A: There are no specific guidelines as the National Evaluators will not be using data from the local evaluations. All participants from whom the National Evaluators collect data, including SFIA respondents and service providers, receive consent forms that explain the use of the data.

Q: Can you provide suggestions on how to use the Proposed Outcome Indicators and Draft Data Sources (National Evaluation Framework, Appendix A, p.61) in the process of developing program logic, reporting and evaluation guidelines with our community partners?
A: The outcomes indicators provide information about the overall population in the Communities for Children areas, and we don't expect much change in the first few years. However, you may want to supplement these data with more local and finely grained data sets which you may have access to, and which could provide better information about local outcomes. We are only using data that is available consistently across Australia.

Q: I need to further explore ethical questions, for example: What guidelines can Facilitating Partners, Community Partners and Local Evaluators use to ensure they are following or developing ethical processes that allow all parties to use instruments designed by someone else; to adopt correct and coordinated data storage and handling practices; and adopt ethical confidentiality and respondent access to data practices?
A: SPRC has obtained ethics clearance from UNSW for the evaluation as a whole. However, this ethics clearance will apply only to the specific procedures outlined in the National Evaluation Framework and the specific questionnaires that will be made available on the CAFCA website. If these questionnaires are adapted, the ethic clearance will not cover the adaptation.

Q: At this stage, the outcomes and process components of the National Evaluation of the Communities for Children Initiative are happening in two waves - 2006 and 2007. What will be the Local Evaluators' role beyond the second round of data collection in 2007 in relation to the National Evaluation?
A: In fact there will be three waves, in 2006, 07 and 08. The role of the Local Evaluator (or Facilitating Partner) in relation to the National Evaluation, is now simply to provide lists of interviewees for the National Evaluation. We will supply each Facilitating Partner with aggregate data on their site, which may be analysed by the Local Evaluator on behalf of the Facilitating Partner.

Service User Study - Communities for Children (CfC), Invest to Grow (ItG) and Local Answers (LA)

Q: Are the final versions of the Service User Study questionnaire available on the website?
A: Yes. The final Service User Study questionnaires are available in PDF and Word format.

Q: Who should complete the Service User Study questionnaires?
A: They can be completed by the service user themselves, or together with the service provider.

Q: Regarding the Service User Study questionnaires, what provisions are there for service users from Culturally and Linguistically Diverse (CALD) or Aboriginal backgrounds, and for those with literacy problems?
A: Service providers may deal with this as appropriate in their particular situation, e.g. a staff member might help the service user to fill in the questionnaire. In addition, the Service User Study may be adapted by the service to suit the local context.

Q: How many service users do we need to interview for the Service User Study?
A: We are not specifying numbers for the Service User Study, as this is a voluntary activity undertaken by the Local Evaluator. It is up to you to decide how many service users to interview based on your resource capacity and what you think you need to know in order to successfully evaluate your services.

Q: In our sites, very few service users will be actually taking part in up and running CfC activities by early 2006 - data will be very limited. How will this issue be handled?
A: This is similar in most CfC sites. The Service User Study questionnaire does not have to be administered in early 2006. It is meant to be used throughout the life of the Strategy, whenever new services start up or people finish using a service.

Q: Should the Service User Study be administered to all the users of a particular service or to a sample?
A: We have not specified the sample because we don't think that we can be prescriptive about that. It depends on the number of users and the availability of staff. Ideally the questionnaire would be administered to the whole population including people ceasing service use and those who did not take up the service but were offered it, but we realise that this is not feasible in many cases.

Q: We have been advised that the Service User Study questionnaires can be adapted to different services. What does this mean?
A: The Service User Study questionnaire consists of a core module and a number of optional modules, which are appropriate for different contexts - eg, different ages of children or types of services. It is made available to Local Evaluators and Project Holders/Facilitating Partners to use, but you are free to adapt it to your local circumstances. For example, if you want to know whether parents have used a particular local facility, you may add such a question to the study.

Comprehensive Community Profiles - Communities for Children (CfC)

Q: When will the demographic information from the CfC Comprehensive Community Profiles be available?
A: The demographic profiles are gradually being provided to Facilitating Partners through mid-2006.

Snapshot Questionnaire - Communities for Children (CfC)

Q: I have been sent a copy the 'Snapshot Questionnaire' - does this replace the 'Strategic' and 'Operational' Snapshot questionnaires previously provided?
A: Yes, the new Snapshot Questionnaire is an amalgamated and easier to use instrument that replaces the two existing questionnaires - the 'Strategic' and 'Operational' Snapshots. If you have completed the previous questionnaires you will not need to complete the new ones, as they are essentially the same.

Q: Are the data collected in the Strategic and Operational Snapshots compatible with that of the amalgamated, single version of the study - the 'Snapshot Questionnaire'?
A: The new questionnaire has been designed to ensure compatibility with the data collected from earlier versions so it doesn't matter whether services in your area have completed the two questionnaires or the single one - all data will be analysed.

Q: I have already distributed the earlier versions of the snapshot questionnaires. Do I now use the new, single version or do I continue to use the separate questionnaires?
A: Facilitating Partners who have distributed the two, separate questionnaires may wish to continue to do so to avoid confusion amongst agencies.