Use the Evaluation Toolkit to plan, analyze, report, and apply evaluation data. The Toolkit includes planning steps that can also be used as a checklist, and links to evaluation instruments.
How do we evaluate program effects?
STEP 1: Define Program and Evaluation Goals
What do we want to find out?
- Define measurable and attainable long- and short-term program goals (attitudinal, behavioral, academic).
- See Evaluation Plan Matrix Sample, Column 1 for examples.
- Use program goals to define measurable and attainable evaluation goals.
- See Evaluation Plan Matrix Sample, Column 2 for examples.
STEP 2: Choose Data Collection Methods
What data do we need to collect?
- Examine school data:
- Academic: GPAs, failure rates, retention, graduation, standardized test scores
- Behavioral: attendance, "tardies," discipline incidents (rate and type)
- Explore self-report survey options:
- Conduct focus groups and/or interviews with students, staff, and/or parents:
- See School-Connect Satisfaction Surveys above for sample questions.
- Design interview questions based on program and evaluation goals.
- Ask for anecdotal examples of students using School-Connect skills (e.g., effective communication, emotional management, study strategies, conflict resolution).
- See STEP 5 for implementation assessment options. Monitoring implementation is essential to deriving outcome findings and understanding cause and effects.
STEP 3: Chart Evaluation Procedures
How do we collect the data?
- Create an Evaluation Plan Matrix to align program and evaluation goals with data collection methods. (see Evaluation Plan Matrix Sample)
- Use program and evaluation goals from STEP 1 for Columns 1 and 2.
- Choose data collection methods for Column 3.
STEP 4: Plan Schedule for Collection and Analysis
When do we collect and analyze the data?
- Consider using the School-Connect SELF Survey, a 63-item self-assessment for students based on School-Connect objectives.
- Plan data analysis feasibility. Consider:
- Data entry or Scantron fees
- Data analysis services (in-house, at the district level, or through a consulting firm)
- Conduct pretests before implementing curriculum. Pretest options:
- End of the previous school year
- Beginning of the semester or school year
- Conduct teacher satisfaction surveys a minimum of four weeks after steady implementation.
- Conduct posttests at the end of the semester or school year.
- For more reliable results, implement self-report surveys:
- Mid-week (Tuesday, Wednesday, or Thursday)
- Away from holidays, school breaks, and exam periods
STEP 5 (ongoing): Assess Implementation Quality and Consistency
How do we know we are implementing the program as intended?
- See the Implementation Toolkit for implementation guidelines.
- Complete an S-C Implementation Plan at the beginning of the year.
- Check in with School-Connect teachers on a regular basis. Save time in staff meetings to discuss S-C success stories, challenges, and questions.
- Consider using the School-Connect Lesson Observation Form to provide feedback to teachers on lesson facilitation.
- Ask teachers about frequency of use and amount of preparation. (See School-Connect Teacher Satisfaction Survey, questions 1-4.)
- Assess what percent of the lessons and lesson outlines are being implemented relative to the initial implementation plan. The S-C Implementation Log can be a helpful tool for gathering teacher feedback.
STEP 6: Monitor Progress and Report Findings
How do we report our findings?
- Chart data collection and implementation feedback over time.
- Examine the correlation between implementation fidelity and outcome data.
- Share evaluation findings with staff and administration.
STEP 7: Discuss and Apply Evaluation Findings
What do we do as a result of our findings?
- Use data results to inform and improve implementation plan and quality.
- Consider additional training in social and emotional skill building.