But often the moment is lost to a basic question: “What worked well, and what didn’t work as well?” We respond about workflow, user interface, and maybe that we didn’t like the shade of blue (which, as any designer will tell you, is really low on the priority list!). And now the moment has passed, even if we are really excited about the product. How could it have gone better? We’ve heard both edtech entrepreneurs AND edtech-enthusiastic teachers bemoan the challenges product developers face in getting meaningful feedback from potential users. The answer is simple, but challenging to master: ask a better question. You can look to similar professional strategies in the classroom to start.
One thing that makes a great teacher is an ability to ask probing questions to deeply understand students' thinking... Phrasing questions just right to get to the understanding we're looking for takes years of practice and honing. For example, Lindsey knows that asking students to diagram and explain why we cut the ends off cut flowers before placing them in a vase, why plant leaf stomata close on dry days, and why sugar snap pea vines wither from the ground up will all demonstrate their knowledge more clearly than a request to define botanical transpiration. This is part of our "pedagogy," and especially "pedagogical content knowledge" - techniques (in this case, questioning strategies) that work particularly well for specific content.
How can software developers use these educator skills to elicit the feedback they need to move their tools in directions that will really improve learning outcomes for our students?
You - edtech entrepreneurs seeking our feedback - can take advantage of our years of experience to start to change this paradigm by carefully crafting your prompts and questions to specifically target the feedback you need. Here are a few ideas to make magic happen:
To get deep with understanding how your product could improve learning activities in a classroom: Ask your teacher collaborator to consider a real potential use case in the classroom. Think of an existing lesson that could be modified using your product. Follow up with questions like:
- What are the learning goals of this use case?
- How would you normally conduct this lesson?
- What would this product allow you and your students to do that your previous style of conducting this lesson would not allow?
- What would this product NOT allow that your previous style of conducting this lesson WOULD?
To probe levels of engagement possible with your product: Ask your teacher collaborator to break down potential use cases for your product according to the SAMR model. If several teacher collaborators can’t imagine M or R level learning experiences with your product, that may be a red flag! Ask your teacher collaborator “What is one learning activity that you could imagine a teacher developing at the _________ level of technology integration, using this product?” Those levels are:
- “Substitution,” in which teachers directly replace paper-based activities with computer-based activities, with no change to the activity itself. (An important first step for many teachers!)
- “Augmentation,” in which the technology provides a functional improvement and the activity itself is essentially the same. (The technology may simplify peer editing processes, for example, or forums may support clearer book group discussions.)
- “Modification,” in which the technology affords different and perhaps more engaging individual learning activities than would otherwise be possible, and the greater overall pattern to the learning may be unchanged. (Student work products may include multimedia or interactive elements rather than static posters, but the topic or driving question to the learning is unchanged.)
- “Redefinition,” in which the technology fundamentally changes the learning opportunity in ways that were inconceivable without technology (For example, through a wider network of collaboration, student-driven content creation, and/or the application of concepts towards global problem solving.)
To get suggestions for tweaks and feature improvements that would lead to broad learning improvements: We know that teachers will suggest features left and right that they think would be spiffy, but do those meet a broad need? And are they necessary for the learning goals supported by the product? To narrow your feedback to features that genuinely meet your goals and the learning needs of kids, you again need to narrow your feedback prompts.
- What are two or three specific learning outcomes that this product supports well?
- What are two or three specific learning outcomes that this product *could* support, but currently doesn’t do well?
- What adjustments could be made to support a specific learning outcome? Please describe that learning outcome, and how it could be met through this product?
To get more detailed feedback on the issues you already know to ask about: You want to find bugs. You want users to hammer on your product, but keep in mind that most teachers are not professional software testers. Phrase even those simple questions well so that you’ll avoid “Well, I didn’t really like the blue color…”
- As you worked through your example lesson use case, did you encounter any opportunities for improving specific features or aspects of this product?
- How easy or challenging was it to move from one part of the learning process to the next?
- What features did you find useful within the teacher dashboard? What barriers prevented you from accessing the student data or crafting the learning activity from within the teacher dashboard?
Carefully phrasing your feedback prompts can definitely lead to much deeper and more meaningful feedback from your teacher collaborators, and it can also support your own deeper understanding of learning trajectories in the classroom as well as your teacher collaborators’ varying pedagogies. If you ask what each of us likes, that doesn’t give you a strong window into our classrooms or into our individual instructional designs. Why each of us makes instructional decisions can be quite different from why another teacher makes instructional decisions. The best edtech developers will have a heuristic understanding of overarching pedagogy to create the best products that will work for a range of teachers, supporting effective pedagogy and (big finish!) improving learning outcomes for all kids!
*jazz hands!*
- Collaboratively written by me and Stephanie Sandifer
- Thanks to Jay Goyal of ActivelyLearn for critical target-audience feedback on this article!