Lots of teachers hear "big data" and think of what's available now: test scores and other relatively meaningless data. Biz and Devs hear "big data" and think of what could be available in the future through revolutionary data mining technology. This disconnect is part of why many at SXSWedu wanted the whole "big data" term banned from the edtech lexicon.
Hearing this other perspective led to my first big "YES!" in the big data conversation while sitting on the panel... If we achieve this potential future the biz and dev folks envision, we'll be able to utilize technology to analyze the petabytes of meaningful student work products - essays, videos, games, blog posts, photo essays, and all sorts of other work - to build longitudinal views of each student's growth as well as snapshots of understanding across a whole school and across the whole country. (Watch the panel and tell me if you can actually see the lightbulb turn on over my head.)
With this understanding, I could envision a (distant: 10-15 years?) future where we have algorithms that can scan student work products looking, for example, for evidence of a student's ability to connect evidence to a scientific conclusion, but at a much deeper scale than simply looking for "if..., then..." sentence structures. (Such short-cuts in looking for understanding were part of the downfall of the Washington State "WASL" tests, that attempted to more deeply analyze student understanding than multiple choice tests, but the assessment rubrics had such low interrater reliability as to be relatively meaningless.) My previous knowledge of difficultly with human-performed text analytics and my lack of knowledge of the state of computer-performed text analytics led to my final assertion that we will definitely still have multiple choice standardized tests in 5 years.
Then came Friday night...
Friday night, I attended a SXSWi party with dear old friends from high school, including one who is now a sales engineer for Spredfast. In learning about Spredfast's social media analytics for corporations to track their online presence and user sentiment, my mind started reeling! Meaningful text analytics that can interpret misspellings, parse deeper meanings than just keywords, and deliver large-scale data about patterns among users - this is all much farther along than I thought! The progress is just in a field that we generally expect to have little relevance in the classroom... (Why would I care whether women in the midwest respond positively to Old Navy's new marketing campaign?)
There seems to be a lot of totally non-education related movement in a direction that education can benefit from enormously in a few years. I got home and, of course, my nerdy husband already knew about some of this kind of stuff going on, especially in stock market analytics.
- A group at Stanford looked at what kinds of algorithms could use sentiment on Twitter to predict stock trends. Of course, simple keywords weren't enough to pull out useful meaning, but they developed an algorithm that successfully tracked language in patterns of individual tweets to patterns in market changes.
- A group at the Technische Universitat Munchen also developed what appears to be a simpler mechanism that can also track market sentiment on Twitter and use that tracking to predict market changes.
- "Text mining" is a whole thing, especially used by Department of Defense to track chatrooms, blogs, Twitter, and more to identify potential threats and fraud. (This makes me very nervous, actually, from a personal perspective, but the technology developed for evil can also be used for good!)
I'm pretty darn smart, but I won't pretend to understand what is actually happening in these algorithms. The awesome thing is... they're working! As this kind of text analytics is perfected in money-rich fields like big corporate marketing and government communications, the trickle-down effects on education have the potential to be truly revolutionary:
- Imagine keeping a portfolio of a student's creative writing from kindergarten through 12th grade, and having software that will spit out that student's progress in theme, story arch, characterization, or setting over the course of their school career.
- Imagine a teacher collecting all 120 of her students' reflections following a major project, and - in only 2 minutes - seeing patterns in the growth of those kids ability to articulate their own metacognition over previous project reflections.
- Imagine seeing a whole district's portfolios over time, and being able to pinpoint spots in the timeline where many students' progress seem to stagnate, and using that data to support teachers in that spot on the timeline to improve their practice.
As Frank Catalano pointed out in his GeekWire post summing up Bill Gates's big SXSWedu keynote, we were all pretty overwhelmed with discussion of "big data," but without a clear view of what that data could look like (and definitely with fear of how that data will be used, or especially sold... again, technology for good or evil). But I definitely think my excitement for big data is growing as I more deeply understand the analytics happening in other fields that we can apply to education soon!
If my vision is correct:
- these analytics will interpret real student work products - not just multiple-choice tests - and perform that interpretation in a highly-reliable way.
- Teachers would no longer need to "teach to the test" to be successful in large-scale assessments, but could focus on teaching necessary skills in a context of real, meaningful, personal projects that would be quickly and flexibly interpreted by these complex text analysis algorithms.
- Teachers would be able to teach using their specific pedagogical strengths, create learning opportunities that best meet their immediate students' needs, and still produce student work products that demonstrate necessary content understanding and skills - all still able to be interpreted by this highly flexibly software!
We're not there yet, but the big money in corporate marketing and DOD will get the technology there, and when we apply it to education - THAT will be big data that truly disrupts and revolutionizes education for millions of kids and their teachers.
This is one of those crossroads moments for me as an educator and tech enthusiast - I see so much potential in how tech can be applied to actually increase student and teacher learning and, I believe, change how we learn (in terms of both how we learn and how we structure learning). And then my department tells me today that in two years, the infamous and almost existent PARCCs will be administered in NYS to 9th, 10th, AND 11th grade ELA. That, and they will be given in two sittings: a three-day and a two-day (not full days, but double the length-wish of the current Regents exam). I hear this, and I have to ask myself if this a best practice to collect this data. I say no, there has to be a better way, a way we can use tech to give standardized formative assessments often, with real and lasting feedback for both student and teacher. shift the funding away from the pockets of the test-maker and use it to provide the test-taker (I.e. the students) with a viable way to create their learning. The politicians are sorely misled here.
ReplyDeleteI love the concept of using "big data"in my practice, and I hope that the right people get their voices heard soon.
On a related tangent, I found out about this tool at the Harvard Future of Learning session in 2011, but haven't followed up on it yet. Has anyone had experience with Disco tests (https://discotest.lectica.org/index_discotest.html)? I remember being blown away by the idea of using multiple, small, frequent standardized tests that provided real time data.
I totally agree, Anthony! These new tests just increase anxiety, reduce classroom individualization, and simply don't give good data! When we can collect standardized data across the breadth of creative, personalized projects (administered in natural classroom settings rather than testing banks!) developed by innovative teachers, THAT will be useful data for a national conversation!
ReplyDeleteAwesome insight...as if my mind wasn't already "reeling" enough with inspiring classroom tech!
ReplyDeleteThinking of more future possibilities is overload for me, today. :)
(sidebar) I'm just starting to use google docs, seeing the plethora of positives in students helping each other that way.
That's for the blog.