Top things people ask me (about evaluation)
Top things people ask me (about Evaluation)
#1 How do I write a good survey?
What do you want to learn from the survey? Why are you asking people to take this survey? What are you going to do with the data? How will asking these questions inform your work or help you meet your program goals? Write out your thoughts to each of these questions and put it in bold, colorful letters at the top of your survey draft.
Keep these tips in mind:
Brainstorm! White down every question that comes to mind, anything you want to ask your respondents. Don’t worry about wording or getting off-topic yet. Do this for at least 7 minutes – go ahead and set a timer.
Cut it down and stay focused: Okay now you can worry about length and staying on-topic. Look back at what you wrote for “What do you want to learn from the survey?” and keep only the questions that help you learn what you said you wanted to learn from the survey.
Simplify the questions: First check that every question asks only one question at a time (Ex. “Was the classroom welcoming and safe?” Changed to “How welcoming was the classroom?” and “How safe did you feel in the classroom?”) Secondly, for each question, ask yourself “What am I actually trying to ask?” then talk in plain language that’s often long and maybe you trip over your words a bit figuring out what you are actually trying to ask. Then take what you said in your regular language and form it into a question.
Use these questions, but is a survey the best method? People seem to be surveyed out these days. There are some other cool methods described below that will get you the same information and probably be more fun, in-depth, and with a better response rate. Still use these questions you developed though!
#2 We got barely any responses to our survey, how can we get a better response rate?
If you want to do the survey again, here are a couple of ways you might be able to get more people to respond:
Incentives – they can be gift cards, a raffle for something, food, swag, etc.
Meet people where they already are – use the last five minutes of programming, school pick-up, or directly during program time
Shorten and simplify the survey (for how to do this, see above) – people are much more likely to finish a survey if it’s short and sweet
Show them where the data is going and what it’s for – share the data from the last survey, tell them why their response is so important, explain how you are going to use what they say to make improvements or how it will benefit them, their children, staff, whoever
If you’re interested in trying a new method, here are some really cool data collection methods I love to use:
For open-ended questions (qualitative data, stories, etc.):
Sticky note wall with 2-3 prompts: Add prompts up onto a wall and ask folks to write their responses on sticky notes and add them under the prompt.
Short discussion with 2-3 different questions. Make sure on the first question, you get everyone to answer and it helps people ease into the discussion. This is like a focus group, but shorter.
In pairs, draw a picture and share out what you drew: Ask people to draw out an open-ended question you have, give folks at least 5 minutes and then ask everyone to talk about what they drew (the data is what they talk about not the drawing!).
In a group, build what it would look like using toys and paper. Gather many different children’s toys, paper, tape, markers, and whatever else people would need to build what your prompt is. Then, ask the prompt and have folks gather the materials and build what they envision! Then have them talk about what they built (again, the data is what they talk about and not necessarily the built thing).
For closed-ended questions (numbers, percentages, etc.)
Token survey: Display a question and then add jars with labels so that people can drop their response into the corresponding jar
Polls on Zoom: Zoom has a cool feature where you can add polls ahead of time and launch them during your virtual meeting
Heads down, thumbs up game: You maybe played this as a kid, everyone puts their heads down and then when you ask questions they put up their thumb for responses
Four/three corners: Label different corners of the room with different response options. Ask a question and have people go to the corner with their response. Then ask them why they decided to go to that corner.
#3 When do I need to bring someone in to help with evaluation?
People engage in evaluation for a bunch of different reasons: to learn what’s working well and where there is room for growth, to measure their progress and impact, to report to funders, to understand how their day-to-day activities will lead them to their larger goals, and more.
People hire evaluation consultants to help them do these things - maybe they want someone to take data and evaluation off their plate, maybe they have no idea where to start or how to collect, analyze, and or share data, or maybe they want someone to walk side-by-side with them helping them to learn the ropes. If any of those ring true for you, it might be time to bring someone in to help with evaluation.
