School-Connect Evaluation Toolkit
Use the Evaluation Toolkit to plan, analyze, report, and apply evaluation data. The Toolkit includes planning steps that can also be used as a checklist, and links to evaluation instruments.
Evaluation Steps/Checklist
How do we evaluate program effects?
STEP 1: Define Program and Evaluation Goals
What do we want to find out?
- Define measurable and attainable long- and short-term program goals (attitudinal, behavioral, academic).
- See Evaluation Plan Matrix Sample, Column 1 for examples.
- Use program goals to define measurable and attainable evaluation goals.
- See Evaluation Plan Matrix Sample, Column 2 for examples.
STEP 2: Choose Data Collection Methods
What data do we need to collect?
- Examine school data:
- Academic: grades, pass rates, retention, graduation, standardized test scores
- Behavioral: attendance, tardies, discipline incidents (rate and type), suspensions
- Explore self-report survey options:
- Conduct focus groups and/or interviews with students, staff, and/or parents:
- See School-Connect Satisfaction Surveys above for sample questions.
- Design interview questions based on program and evaluation goals.
- Ask for anecdotal examples of students using School-Connect skills (e.g., effective communication, emotional management, study strategies, conflict resolution).
- See STEP 5 for implementation assessment options. Monitoring implementation is essential to deriving outcome findings and understanding cause and effects.
STEP 3: Chart Evaluation Procedures
How do we collect the data?
- Create an Evaluation Plan Matrix to align program and evaluation goals with data collection methods. (see Evaluation Plan Matrix Sample)
- Use program and evaluation goals from STEP 1 for Columns 1 and 2.
- Choose data collection methods for Column 3.
STEP 4: Plan Schedule for Collection and Analysis
When do we collect and analyze the data?
- Consider using the School-Connect Student Skills Assessment toward the end of the semester/year. This 28-item survey is designed to compare students using School-Connect to another group, class, or school that is not using School-Connect. To make it more reliable for students to objectively self-assess, it measures students' behaviors more so than their opinion of their SEL skills.
- Plan data analysis feasibility. Consider:
- Data entry or Scantron fees (if applicable)
- Data analysis services (in-house, at the district level, or through a consulting firm)
- Conduct pretests before implementing curriculum. Pretest options:
- End of the previous school year
- Beginning of the semester or school year
- Conduct teacher satisfaction surveys a minimum of four weeks after steady implementation.
- Conduct posttests at the end of the semester or school year. (NOTE: Student self assessments may not show pre-post effects. As students become more aware of SEL skills, they may rate themselves the same or lower because they are more aware of areas they need to improve upon.)
- For more reliable results, implement self-report surveys:
- Mid-week (Tuesday, Wednesday, or Thursday)
- Away from holidays, school breaks, and exam periods
STEP 5 (ongoing): Assess Implementation Quality and Consistency
How do we know we are implementing the program as intended?
- See the Implementation Toolkit for implementation guidelines.
- Complete a S-C Implementation Plan at the beginning of the year.
- Check in with School-Connect teachers on a regular basis. Save time in staff meetings to discuss S-C success stories, challenges, and questions.
- Consider using the School-Connect Lesson Observation Form to provide feedback to teachers on lesson facilitation.
- Ask teachers about frequency of use and amount of preparation. (See S-C Teacher Satisfaction Survey, questions 1-4.)
- Assess the percentage of lessons being implemented relative to the initial implementation plan. Consider averaging the percentage of lessons completed and the degree to which teachers follow the lesson outline (e.g., do teachers implement the lessons as intended?) The School-Connect Implementation Log can be a helpful tool for gathering teacher feedback.
STEP 6: Monitor Progress and Report Findings
How do we report our findings?
- Chart data collection and implementation feedback over time.
- Examine the correlation between implementation fidelity and outcome data.
- Share evaluation findings with staff and administration.
STEP 7: Discuss and Apply Evaluation Findings
What do we do as a result of our findings?
- Use data results to inform and improve implementation plan and quality.
- Consider additional training in social and emotional skill building.
