Archive for the ‘Education’ Category

The Fallacy of the Right Answer

The Fallacy of the Right Answer is everywhere. With regards to education technology, it dates back at least to BF Skinner.

Skinner saw education as a series of definite, discrete, linear steps along a fixed, straight road; today this is called a curriculum. He referred to a child who guesses the password as “being right”. Khan Academy uses similar gatekeeping techniques in its exercises, limiting the context. Students must meet one criterion before proceeding to the next, being spoon-fed knowledge and seeing through a peephole not unlike Skinner’s machines. Furthermore, these steps are claimed to be objective, universal and emotionless. Paul Lockhart calls this the “ladder myth”, the conception of mathematics as a clear hierarchy of dependencies. But the learning hierarchy is tangled, replete with strange loops.

It is fallacious yet popular to think that a concept, once learned, is never forgotten. But most educated adults I know (including myself) find value in rereading old material, and make connections back to what they already have learned. What was once understood narrowly or mechanically can, when revisited, be understood in a larger or more abstract context, or with new cognitive tools. There are two words for “to know” in French. Savoir means to know a fact, while connaitre means to be familiar with, comfortable with, to know a person. The Right Answer loses sight of the importance, even the possibility, of knowing a piece of information like an old friend, to find pleasure in knowing, to know for knowing’s sake, because you want to. Linear teaching is workable for teaching competencies but not for teaching insights, things like why those mechanical methods work, how they can be extended, and how they can fail.

Symbol manipulation according to fixed rules is not cognition but computation. The learners take on the properties of the machines, and those who programmed them. As Papert observed, the computer programs the child, not the other way around (as he prefers). Much of this mechanical emphasis is driven by the SAT and other unreasonable standardized tests which are nothing more than timed high-stakes guessing games. They are gatekeepers to the promised land of College. Proponents of education reform frequently cite distinct age-based grades as legacy of the “factory line model” dating back to the industrial revolution. This model permeates not only how we raise children, but more importantly, what we raise them to do, what we consider necessary of an educated adult. Raising children to work machinery is the same as, or has given way to, raising them to work like machinery. Tests like the SAT emphasize that we should do reproducible de-individualized work, compared against a clear, ideal, unachievable standard. Putting this methodology online does not constitute a revolution or disruption.



Futurists have gone as far to see the brain itself as programmable, in some mysteriously objective sense. At some point, Nicholas Negroponte veered off his illustrious decades-long path. Despite collaborating with Seymour Papert at the Media Lab, his recent work has been dropping tablets into rural villages. Instant education, just add internet! It’s great that the kids are teaching themselves, and have some autonomy, but who designed the apps they play with? What sort of biases and fallacies do they harbor? Do African children learning the ABCs qualify as cultural imperialism? His prediction for the next thirty years is even more troublesome: that we’ll acquire knowledge by ingesting it. Shakespeare will be encoded into some nano-molecular device that works its way through the blood-brain barrier, and suddenly: “I know King Lear!”. Even if we could isolate the exact neurobiological processes that constitute reading the Bard, we all understand Shakespeare in different ways. All minds are unique, and therefore all brains are unique. Meanwhile, our eyes have spent a few hundred million years of evolutionary time adapting to carry information from the outside world into our mind at the speed of an ethernet connection. Knowledge intake is limited not by perception but by cognition.

Tufte says, to simplify, add context. Confusion is not a property of information but of how it is displayed. He said these things in the context of information graphics but they apply to education as well. We are so concerned with information overload that we forget information underload, where our brain is starved for detail and context. It is not any particular fact, but the connections between them, that constitute knowledge.  The fallacy of reductionism is to insist that every detail matters: learn these things and then you are educated! The fallacy of holism is to say that no details matter: let’s just export amorphous nebulous college-ness and call it universal education! Bret Victor imagines how we could use technology to move from a contrived, narrow problem into a deeper understanding about generalized, abstract notions, much as real mathematicians do. He also presents a mental model for working on a difficult problem:

I’m trying to build a jigsaw puzzle. I wish I could show you what it will be, but the picture isn’t on the box. But I can show you some of the pieces… If you are building a different puzzle, it’s possible these pieces won’t mean much to you. You might not have a spot for them to fit, or you might not yet. On the other hand, maybe some of these are just the pieces you’ve been looking for.

One concern with Skinner’s teaching machines and their modern-day counterparts is that they isolate each student and cut off human interaction. We learn from each other, and many of the things that we learn fall outside of the curriculum ladder. Learning to share becomes working on a team; show-and-tell becomes leadership. Years later, in college, many of the most valuable lessons are unplanned, a result of meeting a person with very different ideas, or hearing exactly what you needed to at that moment. I found that college exposed to me brilliant people, and I could watch them analyze and discuss a problem. The methodology was much more valuable than the answer it happened to yield.

The hallmark of an intellectual is do create daily what has never existed before. This can be an engineer’s workpiece, an programmer’s software, a writer’s novel, a researcher’s paper, or an artist’s sculpture. None of these can be evaluated by comparing them to a correct answer, because the correct answer is not known, or can’t even exist. The creative intellectual must have something to say and know how to say it; ideas and execution must both be present. The bits and pieces of a curriculum can make for a good technician (a term I’ve heard applied to a poet capable of choosing the exact word). It’s not so much that “schools kill creativity” so much as they replace the desire to create with the ability to create. Ideally schools would nurture and refine the former (assuming something-to-say is mostly innate) while instructing the latter (assuming saying-it-well is mostly taught).

What would a society look like in which everyone was this kind of intellectual? If everyone is writing and drawing, who will take out the trash, harvest food, etc? Huxley says all Alphas and no Epsilons doesn’t work. Like the American South adjusting to an economy without slaves, elevating human dignity leaves us with the question of who will do the undignified work. As much as we say that every child deserves an education, I think that the creative intellectual will remain in an elite minority for years to come, with society continuing to run on the physical labor of the uneducated. If civilization ever truly extends education to all, then either we will need to find some equitable way of sharing the dirty work (akin to utopian socialist communes), or we’ll invent highly advanced robots. Otherwise, we may need to ask ourselves a very unsettling question: can we really afford to extend education to all, given the importance of unskilled labor to keep society running?


If you liked this post, you should go read everything Audrey Watters has written. She has my thanks.

Brief Thoughts On Scratch

Previously, I’ve lambasted the children’s programming language Scratch for its cockpit’s worth of controls.  This encourages its users to try anything and see what works, rather than plan, predict, and understand exactly what each piece of code is doing. It’s instant gratification … and a tight feedback loop.

Scratch is not a tool to learn programming or metacognition; Scratch is a tool to create artistic displays that could not otherwise be created (by children). Scratch thus allows children to explore ideas not related to mathematics or programming. They have creative freedom, much like art class. And what elementary schooler produces anything particularly good, objectively speaking, in art class? So don’t judge the Scratch projects too harshly.

Scratch is a social platform, except that the socialization happens in real life. Get a few kids in a room using it, and they’ll share both creations and code,  motivate each other, and change goals on the fly. This differs from more mature programming, where one has a specific goal in mind. The other key difference is that most languages discourage straight up experimentation; one have to know what one is doing in order to do it. Scratch reverses this: a kid can learn what a command does through using it. This is because all the commands are displayed, ready to be used.

Not only displayed, but also labelled, unlike the Khan Academy programming language that drops down four numbers with no context. It’s a way to slyly introduce relative and absolute motion – move up by, move to – in a way that lets kids work out the rules. No, they won’t work out all the rules, but I think they’ll come to fewer incorrect conclusions (misconceptions) in a reactive medium than with marks on paper. They will figure it out later, much later.

Scratch is a way to put Lego bricks into the bucket. The kid will reassemble them into many different knowledge structures over the years before creating something strong and beautiful – an educated mind. It’s during that process, that struggle, that they can learn to program with planning and expressiveness, rather than tacking on bricks ad-hoc. It’s a stage everyone goes through, and Scratch can help a child make the most of it. But don’t confuse acquiring bricks with figuring out how to assemble them.

This isn’t to say that Scratch as it exists is perfect; far from it. We need to keep rethinking what tools are best to give to our children (and adults, for that matter). But I’m backing off my previous stance that guided minimalism is the answer. (Or is my new “wait and let them figure it out” view too fatalist?)

The Top 5 Things Done Wrong in Math Class

Sorry to jump on the top-n list bandwagon, as Vi Hart deliciously parodies, but that’s just how this one shakes out. Some of the reasons why these things are done wrong are pretty advanced, but if you’re a high school student who stumbled upon this blog, please stay and read. Know that it’s okay that you won’t get everything.

All of these gripes stem from the same source: they obfuscate what ought to be clear and profound ideas. They’re why math is hard. Like a smudge on a telescope lens, these practices impair the tool used to explore the world beyond us.

EDIT: This list focuses on notation and naming. There are other “things” done wrong in math class that any good teacher will agonize over with far more subtlety and care than this or any listicle.

5. Function Composition Notation

Specifically f \circ g, which is the same as g(f(x)). No wait, f(g(x)). Probably. This notation comes with a built-in “gotcha”, which requires mechanical memorization apart from the concept of function composition itself. The only challenge is to translate between conventions. In this case, nested parentheses are ready-made to represent composition without requiring any new mechanistic knowledge. They exploit the overloading of parentheses for both order of operations and function arguments; just work outwards as you’ve always done. We should not invent new symbols to describe something adequately described by the old ones.

Nested parentheses lend themselves to function iteration, f(f(x)). These functions are described using exponents, which play nice with the parens to make the critical distinction between f^2(x) = f(f(x)) and f(x)^2 = (f(x))^2 = f(x)f(x). This distinction becomes critical when we say arcsine aka \sin^{-1} and cosecant aka \frac {1}{\sin} are both the inverses of sine. Of course, things get confusing again when we drop the parens and get \sin^2x = (\sin x)^2 because \sin x^2 = \sin (x^2). This notation also supports  first-class functions: once we define a doubling function d(x) = 2x, what is meant by d(f)? I’d much rather explore this idea, which is “integral” to calculus (and functional programming), than quibble over a symbol.

4. The Word “Quadratic”

I’m putting “quadratic” where it belongs: number four. The prefix quadri- means four in every other context, dating back to Latin. (The synonym tetra- is Greek.) So why is x^2 called “quadratic”? Because of a quadrilateral, literally a four-sided figure. But the point isn’t the number of sides, it’s the number of dimensions. And dimensionality is tightly coupled with the notion of the right angle. And since x equals itself, then we’re dealing with not just an arbitrary quadrilateral but a right-angled one with equal sides, otherwise known as a square. So just as x^3 is cubic growth, x^2 is should be called squared growth. No need for any fancy new adjectives like “biatic”, just start using “square”. (Adverb: squarely.) It’s really easy to stop saying four when you mean two.

3.14 Pi

Unfortunately, there is a case when we have to invent a new term and get people to use it. We need to replace pi, because pi isn’t the circle constant. It’s the semicircle constant.

The thrust of the argument is that circles are defined by their radius, not their diameter, so the circle constant should be defined off the radius as well. Enter tau, \tau = \frac{C}{r}. Measuring radians in tau simplifies the unit circle tremendously. A fraction time tau is just the fraction of the total distance traveled around the circle. This wasn’t obvious with pi because the factor of 2 canceled half the time, producing \frac{5}{4}\pi instead of \frac{5}{8}\tau.

If you’ve never heard of tau before, I highly recommend you read Michael Hartl’s Tau Manifesto. But my personal favorite argument comes from integrating in spherical space. Just looking at the integral bounds for a sphere radius R:

\int_{\theta=0}^{2\pi} \int_{\phi=0}^{\pi} \int_{\rho=0}^{R}

It’s immediately clear that getting rid of the factor of two for the \theta (theta) bound will introduce a factor of one-half for the \phi (phi) bound:

\int_{\theta=0}^{\tau} \int_{\phi=0}^{\frac{\tau}{2}} \int_{\rho=0}^{R}

However, theta goes all the way around the circle (think of a complete loop on the equator). Phi only goes halfway (think north pole to south pole). The half emphasizes that phi, not theta, is the weird one. It’s not about reducing the number of operations, it’s about hiding the meaningless and showing the meaningful.

2. Complex Numbers

This is a big one. My high school teacher introduced imaginary numbers as, well, imaginary. “Let’s just pretend negative one has a square root and see what happens.” This method is backwards. If you’re working with polar vectors, you’re working with complex numbers, whether you know it or not.

Complex addition is exactly the the same as adding vectors in the xy plane. It’s also the same as just adding two numbers and then another two numbers, and then writing i afterwards. In this case, you might as well just work in R^2. (Oh hey, another use of exponents.) You can use the unit vectors \hat{x} and \hat{y}, rather than i and j which will get mixed up with the imaginary unit, and besides, you defined that hat to mean a unit vector. Use the notation you define, or don’t define it.

Complex numbers are natively polar. Every high school student (and teacher) should read and play through Steven Witten’s jaw-dropping exploration of rotating vectors. (Again students, the point isn’t to understand it all, the point is to have your mind blown.) Once we’ve defined complex multiplication – angles add, lengths multiply – then 1 \angle 90^{\circ} falls out as the square root of 1 \angle 180^{\circ} completely naturally. You can’t help but define it. And moreover, (1 \angle -90^{\circ})^2 goes around the other way, and its alternate representation (1 \angle 270^{\circ})^2 goes around twice, but they all wind up at negative one. Complex numbers aren’t arbitrary and forced; they’re a natural consequence of simple rules.

Even complex conjugates work better with angles. Instead of an algebraic argument and a formula to memorize, we can geometrically see that we we need to add an angle that brings us back to horizontal, which is just the negative of the angle we already have. This is mathematically equivalent to changing the sign on the imaginary component of the vector, but cognitively it’s very different. You can, with clarity and precision, see what you are doing in a way numerals can never express.

1. Boxplots

Boxplots make the top of the list because they’re taught at a young age and never challenged. They are brought up as a standard way to visualize data, when the boxplot was a relatively recent invention of one statistician, John Tukey. Edward Tufte has proposed variants which dramatically reduce the ink on the page. They are much easier to draw, which is important when you want to convince children that math isn’t about meticulous marks on the page. They have no horizontal component, so in addition to being more compact, they also do not encode non-information in their width.


Boxplots infuriate me because they indoctrinate the idea that there is one way to do it, and that it is not up for discussion. More time is spent on where to draw the lines than why quartiles are important, or how to read what a boxplot says about that data. Boxplots epitomize math as a recipebook, where your ideas are invalid by default and improvisation is prohibited. Nothing could be further from the truth. Moreover, boxplots slap a one-size-fits-all visualization on the data without bothering to ask what other things we could do with them. Tukey’s plots don’t just obscure the data, they obscure data science.

Media for Thinking the Eminently Thinkable

Bret Victor has done it again. His latest talk, Media for Thinking the Unthinkable, gives some clues as to how scientists and engineers can better explore their systems, such as circuits or differential equations. He’s previously shown an artistic flair, demoing systems that allow artists to interact directly with their work, without the use of language. In both cases, the metric for success is subjective. What the novel user interface is doing is not determining the “right” outcome but allowing a human to better see and select an outcome. Victor is trying to help users develop intuition about these systems.

That strategy looks promising for a sufficiently complex systems, but when it comes to pre-college education, the goal is not to instill semiconscious, unarticulatable hunches. This relatively simple material demands full, clear, and lingual understanding. This decidedly different goal will, or at least should, guide the design of educational technology in a different direction than Victor’s demos.

What is an educational technology? In today’s world. you’re probably thinking of a game, video, recorded demonstration, or something else with a screen. Those all qualify, but so do manipulatives (teacher-speak for blocks and other objects for kids to play with the grok a concept) and pencil and paper. For the simplest of problems, say 2+2, there’s no need to get out the iPad, just grab four blocks. For advanced math, a computer is necessary. So where is the meeting point, where the see-saw of objects and devices balances?

What follows is a case study of educational technology applied to a specific mathematical idea, multiplication of negative numbers, which is pretty close to that balance point. We’re trying to explain why a negative times a negative equals a positive, a classic grade school trouble spot. (Instead of fixing a specific misconception, we could have to open-ended exploration of a slightly larger topic. Touch Mathematics has some interesting material on that front.)

The MIND Research Institute has developed a series of visual math puzzles that all involve helping a penguin traverse the screen. In their multiplication unit, the penguin stands on a platform in front of a gap. Multiplying by a number stretches the platform proportionally. A negative number will rotate the platform 180 degrees, so two rotations cancel each other. When a student interacts with this system, they gain a useful metaphor for thinking about negative numbers: go left on the number line. However, many of the more advanced puzzles seem contrived, and are difficult to do without resorting to pencil and paper because they don’t provide a cognitively useful metaphor. Moreover, adopting the unit, setting up computers for the students, and setting aside time has significant overhead. Can we go simpler?

Let’s take the rotation metaphor and find physical objects for it. We could use plastic spinners with telescoping arms that allow them to stretch. The arm should lock into place at regular intervals, and the rotation in two opposite directions. I don’t think this will work because the system no longer reacts, and the student has to provide the ideas, and it’s susceptible to mechanical breakage. We could have pentagonal pieces that look like squares with a slight arrow on one side that allows us to indicate direction. Then we place groups of them together, three groups of two for 2×3, and rotate them when we have a negative. But then we have to rotate all of them, which is time consuming. We could put blocks into “boats”, some way of grouping them and assigning a rotation, but that’s even more cumbersome. All of these methods require special manipulative to be purchased, organized, stored, and cleaned. To summarize, I can’t think of a good way to adopt the rotation metaphor into physical objects.

At even less fidelity, we can use square blocks. These are generic enough to justify keeping around for other lessons, and we can have magnetic squares for the teacher to put on the board and actual blocks for the kids to play with at their desks. We can use the grouping idea, and different colors for positive and negative blocks. Here’s how I envision it working:


So the blue squares are positive numbers, and the red squares are negative ones, except in the second two examples when we’re on the other side of the number lines, in Negative Land. In Negative Land, the blue squares are negative! So that means that the red squares that were negative are now positive.

That doesn’t make much sense. We’ve created an abstract visualization, so that even as the kids grasp the squares they won’t grasp the concept. The Negative Land mumbo jumbo winds up hand waving away the crux of the issue: why is a negative times a negative positive? We’ve illustrated the idea of reversing something twice but haven’t done much to justify it. Even worse, we’ve created two representations of the same number. The middle two lines are both -2, and the outer two lines are +2, but they don’t look the same.

Even more low tech: repeat the same exercise on paper with Xs and Os standing in for red and blue squares. This is eminently practical but the students now lose the ability to hold something concrete in their hands. We’ve pared the metaphor of flipping twice down to its essence, but lost a lot of its effectiveness in the process.

To quote Edison, “I haven’t failed. I’ve found ten thousand ways that don’t work.” Well, not quite so many, but same idea. Hopefully I’ve illustrated some of the pitfalls in making educational technology both practical and effective. And maybe you learned something about negative numbers in the pro — wait a sec. If explaining different ideas for explanations can actually work, then we may not have to come away empty handed after all.

The computer is assisted imagination. We can take the metaphors expressed most clearly in software and give them to the kids directly. Tell them to imagine themselves on a stretching, rotating platform. Better yet, line up groups of students and have them all rotate together, sensing the math with their own bodies.

The hard part of crafting a lesson plan, whether in person or over technology, is devising the metaphor, a new way to think about a topic so that it makes sense. Once we see negative numbers as rotation, systems of inequalities as unbalanced mobiles, complex numbers as spinners, then the students can explore them. That can be in spoken or written word, symbols, movement, sculpture, drawing, or on the computer. That’s the easy part.

This doesn’t bode well for educational technology companies. If their products are effective, their central metaphors can be easily expatriated to classroom exercises. At best, the metaphor is wrapped in unnecessary packaging; at worst, the packaging is all there is, hawked by cargo cults worshipping motion and interactivity as if these things only exist on a screen.

* * *

An addendum, back to Bret Victor. In a note about the talk, he defines its subject matter as “a way of thinking. In particular — a way of using representations to think powerfully about systems.” (Emphasis his.) He is striving to create the next generation of tools that make unthinkable thoughts thinkable. Only these “powerful, general, technology-independent ways of thinking” will still be relevant a hundred years from now. It’s a daunting, open-ended task, especially considering how much trouble we got into just with arithmetic.

With the announcement of iOS 7, John Maeda criticized not just the OS but the debate it engendered. By phrasing interface design as a binary, of photorealism vs. flat abstraction, “To Skeu or Not To Skeu”, we lose sight of the possibilities that lie before us. Maeda writes, “What we need now is to move beyond the superficial conversation about styles and incremental adjustments to boldly invent the next frontier of interface design.” What do those new designs look like? “Something we haven’t even dreamed of yet.”

Reading visionaries like Victor and Maeda, it’s tempting to join the great quest to fundamentally alter how every person uses software, and by extension, how every person thinks. But part of me doubts the realism of their grandiose pronouncements. On the other side of the coin, Matt Gemmell is an iOS developer very much concerned with the present and its tools. He thoroughly deconstructs iOS 7, seeing it as an improvement, but not extravagantly so. It’s a needed update after six years. In another six we’ll be able to see the next UI paradigm, but he doesn’t waste breath trying to guess it now. Next century is of no concern. Write this month’s app, get this week’s paycheck, enjoy dinner tonight with friends and family. We’ll create the tiniest bit of the future in the morning.

Abstraction and Standardization

What is the future of art? What media will it use? Computers, obviously. Information technology is very good at imitating old media: drawing programs, music programs, word processors designed for playwrights or authors. But none of these tap into the intrinsic strengths of the computer, able to do something no other medium can: simulate. Bret Victor, the man so demanding of user interfaces he left Apple, is dissatisfied with the tools available to artists that allow them to simulate. So he made his own, and gave a one-hour talk on it.

Those interested should definitely take the time to watch it, but to summarize, he demonstrates the power of simulation in creating art that is part animation and part performance, with the human and computer reacting to one another. He then lifts the curtain and show us the tools he used to simulate the characters in the scene, and it’s not code. Instead, it’s a drawing program, with lines and shapes, that he uses to define behavior. Code, he points out, is based on algebra, but his system is based on geometry. Finally, he concludes with a short performance that he built with these tools. Higher is the story of earth, from the stars to cells to civilization to space travel back to the stars.

What blew my mind about Higher is that a few years ago, I had independently created a short film on exactly that topic, with exactly the same background music (Kyle Gabler’s Best Of Times from World of Goo). Victor’s piece was far more polished, but we had both been inspired by the same music to express the same idea, the journey of life to the stars. Remember when I complained about not finding people who shared my narrative? So this is what that feels like.

What drove Victor to create his tools was the belief that art is an attempt to communicate that which cannot be put into words. By binding simulation to lingual code, we make it inaccessible and unsuitable for art and artists. Direct manipulation of the art, which is how art has been created going back to cave paintings, allows the artist to interact with and lend emotion to the art in ways not possible through code’s layer of indirection, of abstraction.

The reason artists’ needs have been neglected by developers is that, for the rest of the world, code works just fine. As I’ve previously blogged, language is one of humankind’s most powerful inventions. The direct manipulation that is liberating to the artist is confining to the engineer. Language is how we manage many layers of abstraction at once; without it we are reduced to pointing and grunting. It’s harder to communicate with a computer in code than a well-designed direct manipulation interface, but code is more powerful. In the sciences, a good result is consistent with what is already known; in art, a good piece is unexpected and shakes our established worldview. More fundamentally, the sciences observe and record some objective outside truth; art looks inward to offer one of many interpretations of the subjective human experience.

This tension that we see between science and art also shows up in schools. In a recent TED talk, Sir Ken Robinson extols diversity as a fundamental human trait, which schools attempt to erase and replace with standardization. We agree that standardization has its place, but I personally think he downplays its importance. Standardization is writing, is language; those things can’t happen without common ways of thinking. At first, children need to explore concepts and use their own terms, without a top-down lesson plan imposed by school administrators. Nevertheless, the capstone is always learning what the rest of the world calls it. That isn’t smashing creativity, but rather empowering the child to learn more about the topic from others and from reference sources. It’s creating a minimum level of knowledge common every adult member of society, which is assumed by all media. Being able to communicate  facts with others isn’t just the result of education, it’s what makes education possible in the first place. With language, groups of people can unambiguously refer to things not present, a shared imagination. Verbalization is a form of abstraction.

Let’s get back to the role of diversity in school. Students should be able to explore what interests them, but the converse is not true: some topics must be taught to everyone, even if some people do not find them interesting. This is especially true before high school. I know you’re not passionate about fractions, Little Johnny, but you need to learn them. Society expects everyone to have a minimum level of competence in every subject. Additionally, passion for a field isn’t always “love at first sight”. The future mathematician isn’t always the first in the class to get basic arithmetic.

Although the curriculum needs to be largely standardized, the pedagogy does not. The neglect of diversity in schools is most heavily felt not in what kids are or are not learning, but how they are learning it. The inflexibility imposed on lesson plans is degrading to teachers and failing our kids. Teachers should be trusted to adapt lessons to their class, and empowered with testing results they find useful, early enough to use it. Standardized testing as it exists today does not fit the bill. Every student needs to achieve the same core competencies, but the paths to doing so will be as diverse as the children themselves. A broad exposure to both methods and topics promotes the development not just of knowledge, but of personality and identity. The reason to have art in school isn’t to improve test scores but because it’s part of being human.

To be more precise, we should distinguish between “the arts” and “art”. The arts are how to create with the media classically used for art: paint, music, poetry, drama, dance, and so on. Like any other discipline, the arts require a standardized language to record and transfer this knowledge. Sometimes it’s plain English, sometimes it’s jargon, sometimes it’s symbols, but it’s still an agreed-upon abstraction. Diversity of ideas expressed in the language is inventive and healthy; diversity of the language itself is nonstandard and chaotic. With this in mind, the arts take their place at one end of a spectrum of knowledge: mathematics, natural science, social science, and history. And the arts.

But art is something entirely different. It is the personal and emotional perception of an experience that communicates without words. Art is direct and concrete; it is subjective and sublime. Much of the arts attempt to create art. Victor’s tools advance the arts; what he creates with them is art.

It’s a defensible position to say that art, because it does not rely on language as all the other fields of knowledge do, is not knowledge at all. But I’ll indulge Victor and say that not all knowledge can be verbalized. That doesn’t mean that art is beyond classification; Victor and I saw the same artistic ideas in the same piece of lyricless music. Conversely, just because something is written down doesn’t mean it’s standardized or useful knowledge. Recently, the mathematics community has been bewildered by an inscrutable set of papers which claim to prove a fundamental piece of number theory. No one can decipher them to tell if the proof is valid, and their author has not been forthcoming with an oral explanation. So in extreme cases, the analogy between language and standardization breaks down. The wordless expression is more coherent than words.

For all the knowledge that abstract language has brought us, ineffable art remains part of the human experience. It is important for our children to learn about art to become mature and thoughtful adults. It is equally important for us to provide tools that support the nonverbal side of thought, to engage the visual and auditory parts of our brains in ways words never can. These are the same failure: the refuge in abstraction, the desire to have everything neat and orderly and predictable. Art exists to explore ambiguity and paradox; it does not demand simple answers but asks complex questions.

A lot of futurists imagine a time when technology makes everything easy. There is a faith in technological convergence, where everything speaks the same language and interacts intelligently and flawlessly. But historically we see technologies become incompatible. If there’s an open standard underneath, such as email, you still get dozens of providers and clients; and if there’s not, you get the walled gardens of social media, loosely tied together by third-party “integration”. What’s important to realize is that the path of technology is not fixed. Our gadgets don’t have to make us more productive and connected; they can make us more artistic and provide privacy, if we design them so. We should stop aspiring to a monoculture of technology because, not only will it not happen for technical and economic reasons, it shouldn’t happen. Standardized technology leads to standardized thinking, especially when coupled with standardized social institutions. Creativity is  not only what drives technology further, but art and humanity as well.

Automated Grading Done Right

The Khan Academy dashboard is meant to provide students and teachers with information that can help them target where a student is struggling, and improve. Unfortunately, the data given isn’t what’s useful to teachers, just what’s easy for computers to measure. These metrics include time spent watching and rewinding videos, time spent on different topics (broken up by videos and quizzes), and proficiency levels in exercises. But as one of the teachers I follow on Twitter points out, these programs don’t let him “SEE my students’ work so I can know HOW/WHY they got questions wrong.” attempts to counteract that. Math teachers send in anonymized samples of student errors that they find telling, common, or inscrutable. Teachers comment as to what they think the student is not understanding and how to fix it. In the process, newer teachers get to see the thought process of their more experienced colleagues. There are patterns to these mistakes, so can a computer be programmed to recognize them?

The most obvious example, which I doubt I’m the first to come up with, is to anticipate patterns of wrong answers. Let’s say you’re testing a physics problem where students need to plug values into an equation. (Yes, this is a naive view of physics but go with it.) Have experienced teachers compile patterns of mistakes students are likely to make: forget to square something, leave off the constant, divide instead of multiply, use a different formula, and so forth. Then the grading software picks new values to plug into the formula, and calculates all the wrong answers for these values (picking numbers so that none of the wrong answer patterns lead to the same numeric answer). Then, if the student gives any of the anticipated wrong answers, the program knows exactly what mistake the student made and can correct them. Hopefully finding a mechanistic error will provide the human teacher with a window into finding and fixing a qualitative misconception.

Let’s take a more complex, real-world example. In some computer science classes at Tufts, the programs written for homework are subjected to a battery of tests, written by both the professor and the students (and their predecessors). In one case, the assignment is to create a programming language interpreter that determines the “type” of pieces of code. For example, it needs to know that true is a boolean, 7 is an integer, and asking if true equals 7 is an error. To clarify, there’s the code the students write (called an interpreter) and then the code that it tries to type, as a test. An interpreter can fail in a number of ways: it can find the wrong type, find a type when it should raise an error, raise and error when it should find a type, stop unexpectedly with an exception, or never stop at all.

I know that’s a bit much to wrap your head around, but (1) that’s the sort of complexity we’re up against and (2) it’s not just an example, it’s a case study. I have a visualization for this data already made, as a class project. My group wanted to take this data and provide actionable reccomendations for the professor, to be able to say, “you’re not handling this properly” or “you don’t understand this very specific detail”. So we hand-built an automated classifier using what we knew about the errors. Here’s part of the visualization we came up with.

Circle errors

The size of each circle represents the number of students who failed at least one test with that error. The vertical position of the circle corresponds to the average number of tests passed by the students who got that error. Colors encode categories, and the horizontal spread means nothing (just a way to prevent overlap). Click on a circle and you’ll get:

Error bars by student

Each bar is a student, identified by an anonymized hash. Their errors are grouped together, with the taller bars being the error we have selected. On the real thing (not these images), you can click on any other bar to jump to that error. Hover over the bar and move around to show each of the tests failed with that error below the circles. This highly-specific information allows the user to look at the individual tests and hypothesize the underlying cause of the error.

You can play with the interactive visualization online here.

Education isn’t a no-computers-allowed clubhouse, but software developers are must be smarter about how we approach these tools. Programmers need to work with educators and fill their needs, not just offer up whatever statistics are easy for them to collect. We have powerful tools like machine learning and visualizations, and teachers with decades of experience. We can make useful automated systems, if we stop acting as if it’s a trivial job.

And yet… all of this takes the views education as a series of questions with right and wrong answers. This is largely true in the STEM fields and even in the humanities, but not in the arts. There really is no good way to automate grading of the arts (or to grade the arts, for that matter). We need to nurture our future artists, but more importantly, we need to teach the skills necessary to appreciate the arts, and to dabble in them. As part of the human condition, we all find ourselves with emotions and ideas that we need to express, through music, painting, or writing (writes the blogger). The fact that very few people will appreciate these works is fine; what matters is the catharsis they give to their creator. That’s something no machine could ever understand.

The Diamond Age: An Edtech Reading

I recently reread Neal Stephenson’s The Diamond Age. It’s a work of science fiction that depicts a future infused with nanotechnology, set in Shanghai and the surrounding areas. It offers some great material for a discussion on the role of technology in education and the limits of computers. Its themes are also relevant to edtech, which is pretty impressive for something published in 1995.

As a quick summary, Lord Finkle-McGraw asks engineer John Hackworth to create a computerized book (the Primer) to supplement his granddaughter Elizabeth’s schooling. Hackworth attempts to create a second copy for his own daughter illicitly, but is mugged and the book falls into the hands of the young street urchin Nell. The Primer guides Nell through leaving her abusive domestic situation and educates her using a customized fantasy story. Though the Primer is capable of reacting to voice commands and displaying a wealth of information, its narration is performed by a human actress Miranda whom Nell does not know. Hackworth, charged with intellectual property theft, makes a plea bargain to provide the source code of the Primer so copies may be distributed to tens of thousands of young abandoned Chinese girls. In the process of modifying the Primer to use a computerized voice, Hackworth is finally able to secure a copy for his daughter Fiona, before disappearing to serve his ten-year sentence.

That’s the first act. To do a proper analysis, I’m going to have to drop a few more spoilers from the most memorable parts of the book, so be warned.

Continue reading