Archive for the ‘Communication’ Category

Prefer Verbs to Nouns

My principle, v0.2

Prefer verbs to nouns.

When Bret Victor introduced the concept of a principle, he said a good principle can be applied “in a fairly objective way”. This is the biggest problem with my first draft, which took several sentences to define what a powerful way of thinking was. A principle must be general enough to apply to many situations, but also able to operationalize to find meaning in any specific situation. Devising or crafting a principle requires inductive reasoning (specific to general), but applying it demands deductive reasoning (general to specific). Forging a principle resembles Paul Lockhart’s vision of mathematics: an idea that at first may be questioned and refined, but at some point begins “talk back”, instructing its creator rather than being shaped by it.

I could have formulated the principle as verbs, not nouns or similar, but the principle itself demands a verb. I have chosen prefer, but I fear that may not be active enough; something closer to choose or emphasize verbs over nouns may more fitting. As the principle predicts, identifying a dichotomy and even choosing one side is easy compared to selecting the verb to encompass the process and relationship. This principle retains status as a draft, although unlike its predecessor it does not have the glaring flaw of subjective application. The verb (and preposition serving it) are still to be determined, and the possibility of cutting a new principle from whole cloth also remains open.

All of this without a discussion of the principle itself! Human language is endlessly versatile and adaptive, and therefore (in hindsight!) it is quite fitting that I use the terms of language itself. Of course the principle does not apply specifically to language, but any field that involves structures and the relationships between them, which is to say, any field at all. It can apply to essays, presentations, or works of art. Finding the verbs and nouns of a particular field is often easy, even if it is difficult to abstract the process. With that said, verbs are not always grammatically verbs; -ing and -tion nouns can be fine verbs for the purpose of the principle.

The verbs should be emphasized to your audience, but the setting will determine how you craft their experience. Most of the liberal arts require grappling with verbs directly; a good thesis is architected around a verb that relates otherwise disparate observations or schools of thought. By emphasizing the verbs, one communicates causal mechanisms, transformations, relationships, and differences across time, location, demographics, and other variables. The goal is not merely to show that the nouns differ (“the a had x but the b had y”), but why, what acted on them to cause the differences. Frequently the base material (often historical events or written works) are already known to your audience, and you need to contribute more than just a summary. You need to justify a distinction.

However, in the presence of detailed, substructured, and numeric nouns, it is often best to let them speak directly. Often the evidence itself is novel, such as a research finding, and you want to present it objectively. In such cases, more frequent in science and engineering, placing your audience’s focus on verbs requires that you place yours on presenting the nouns. The more nouns you have, the more ways they can relate to each other; the more detailed the nouns, the more nuanced those relationships can be. When the nouns are shown correctly, your audience will have a wide array of verbs available to them; Edward Tufte gives examples (Envisioning Information, 50):

select, edit, single out, structure, highlight, group, pair, merge, harmonize, synthesize, focus, organize, condense, reduce, boil down, choose, categorize, catalog, classify, list, abstract, scan, look into, idealize, isolate, discriminate, distinguish, screen, pigeonhole, pick over, sort, integrate, blend, inspect, filter, lump, skip, smooth, chunk, average, approximate, cluster, aggregate, outline, summarize, itemize, review, dip into, flip through, browse, glance into, leaf through, skim, refine, enumerate, glean, synopsize, winnow the wheat from the chaff, and separate the sheep from the goats

The ability to act in these ways is fragile.  Inferior works destroy verb possibilities (science and engineering) or never present them at all (liberal arts). Verbs are the casualties of PowerPoint bullets; nouns can often be picked out from the shrapnel but the connections between them are lost. But conversely, a focus on verbs promotes reason and the human intellect. Verbs manifest cognition and intelligence. Emphasizing verbs is a proxy and litmus test for cogent thought.

Advertisements

Of Signs, Skyscrapers, and Software

I was looking for a book on data visualization. Having gone through Edward Tufte’s classics, I browsed the Tufts library catalog by “visualization”. The two keepers were only tangentially related, but I’ve learned that I sometimes attack problems too directly, so I checked out Signage and Wayfinding Design by Chris Calori and The Heights: Anatomy of a Skyscraper by Kate Ascher. Both have involve service through the built environment. That is, unlike the social justice programs that many of my classmates engage in that serve interpersonally, these books see service conducted through the proxy of a created object. This model appeals to me because the object can serve many more people than I could personally interact with. The object endures to serve in the future, as opposed to the many charitable acts that have vanishingly immediate returns. That is, service through objects is more efficient.

In the case of signage, it is intellectual service. Wayfinding provides empowering knowledge much the way that software can (should). Signage also serves a secondary placemaking role. Signs not only describe the built environment; they are part of it; they should embody emotions, preferring to lend character to its environment over abstract minimalism. Therefore, signs walk the same tightrope that infographics do, between information clarity and contextual aesthetics.

Chris Calori leaves no stone unturned as she documents the full process of signage production. On one hand, it is ruthlessly physical, with details such as mounting, dimensions, lighting, materials, and finishes to be specified prior to fabrication. Much of the waterfall-like process is principled on preventing the full construction of a faulty signage program, by using detailed plans, miniatures, prototypes, and renderings. On the other hand, signage is a great example of the divide between art and design. Art attempts to communicate non-obvious truths through subtety that takes time to absorb. Signage (and design) is just the opposite: communicate rather mundane information quickly and unambiguously. Calori defines the pyramid model of signage, which encompasses the information content, the graphics, and the hardware. Design subdivides into the abstract information, concerned with hierarchies and placement, and graphics, concerned with symbols, diagrams, and typefaces. The book thoroughly addresses each of these, as well as the regulatory and legal concerns one is likely to encounter along the way. It also includes thirty-two pages of color photographs of finished signage programs, which are not to be missed.

As for The Heights, the book itself is intellectual service. Printed in full color, it’s a surprisingly detailed-yet-accessible look at the engineering and planning behind tall buildings. Want to know about cranes, air handlers, curtain walls, elevator safety, green roofs, fire sprinklers, floor plates, pile drivers, and window washers? It’s all there, with helpful illustrations. Section titles are printed in a 3D extruded typeface that resembles a building (for once, a justified use case) and the table of contents is done delightfully as an elevator directory, reading from bottom to top. Wayfinding is not mentioned in the text, but its principles are applied to the presentation itself.

Of course, the very act of creating a well-designed skyscraper contributes tremendously to the built environment. Such a building can provide living and working space for thousands of people for a century. Design decisions can become crucially important or confining decades after they were made. Unlike Calori’s laser-focus, the skyscrapers involve thousands of people of diverse education and wealth backgrounds, from construction workers to financiers, tenants to janitors. Construction on this scale is an act with huge societal ramifications. Engineering is not neutral, politically, socially, or ethically.

But the authors are undaunted. They strive to make objects, whether signs or skyscrapers, that make life more enjoyable, more comprehensible, and more fair for all who come into contact with them. Through technical competence, goal-oriented design, hard work, and luck, objects large and small come to enrich our lives. What kind of object do you want to make?

Infographics and Data Graphics

I’d like to set the record straight about two types of graphical documents floating around the internet. Most people don’t make a distinction between infographics and data graphics. Here are some of each – open them in new tabs and see if you can tell them apart.

No peeking!

No, really, stop reading and do it. I can wait.

Okay, had a look and made your categorizations? As I see it, dog food, energy, and job titles are infographics, and Chicago buildings, movie earnings, and gay rights are data graphics. Why? Here are some distinctions to look for, which will make much more sense now that you’ve seen some examples. Naturally these are generalizations and some documents will be hard to classify, but not as often as you might think.

Infographics emphasize typography, aesthetic color choice, and gratuitous illustration.
Data graphics are pictorially muted and focused; color is used to convey data.

Infographics have many small paragraphs of text communicate the information.
Data graphics are largely wordless except for labels and an explanation of the visual encoding.

In infographics, numeric data is scant, sparse, and piecemeal.
In data graphics, numeric data is plentiful, dense, and multivariate.

Infographics have many components that relate different datasets; sectioning is used.
Data graphics have single detailed image, or less commonly multiple windows into the same data.

An infographic is meant to be read through sequentially.
A data graphic is meant to be scrutinized for several minutes.

In infographics, the visual encoding of numeric information is either concrete (e.g. world map, human body), common (e.g. bar or pie charts), or nonexistent (e.g. tables).
In data graphics, the visual encoding is abstract, bespoke, and must be learned.

Infographics tell a story and have a message.
Data graphics show patterns and anomalies; readers form their own conclusions.

You may have heard the related term visualization – a data graphic is a visualization on steroids. (An infographic is a visualization on coffee and artificial sweetener.) A single bar, line, or pie chart is most likely a visualization but not a data graphic, unless it takes several minutes to absorb. However, visualizations and infographics are both generated automatically, usually by code. It should be fairly easy to add new data to a visualization or data graphic; not so for infographics.

If you look at sites like visual.ly which collects visualizations of all stripes, you’ll see that infographics far outnumber data graphics. Selection bias is partially at fault. Data graphics require large amounts of data that companies likely want to keep private. Infographics are far better suited to marketing and social campaigns, so they tend to be more visible. Some datasets are better suited to infographics than data graphics. However, even accounting for those facts, I think we have too many infographics and too few data graphics. This is a shame, because the two have fundamentally different worldviews.

An infographic is meant to persuade or inspire action. Infographics drive an argument or relate a story in a way that happens to use data, rather than allowing the user to infer more subtle and multifaceted meanings. A well-designed data graphic can be an encounter with the sublime. It is visceral, non-verbal, profound; a harmony of knowledge and wonder.

Infographics already have all the answers, and serve only to communicate them to the reader. A data graphic has no obvious answers, and in fact no obvious questions. It may seem that infographics convey knowledge, and data graphics convey only the scale of our ignorance, but in fact the opposite is true. An infographic offers shallow justifications and phony authority; it presents that facts as they are. (“Facts” as they “are”.) A data graphic does not foster any conclusion upon its reader, but at one level of remove, provides its readers with tools to draw conclusions. Pedagogically, infographics embrace the fundamentally flawed idea that learning is simply copying knowledge from one mind to another. Data graphics accept that learning is a process, which moves from mystery to complexity to familiarity to intuition. Epistemologically, infographics ask that knowledge be accepted on little to no evidence, while data graphics encourage using evidence to synthesize knowledge, with no prior conception of what this knowledge will be. It is akin to memorizing a fact about the world, or accepting the validity of the scientific method.

However, many of the design features that impart data graphics with these superior qualities can be exported back to infographics, with compelling results. Let’s take this example about ivory poaching. First off, it takes itself seriously: there’s no ostentatious typography and the colors are muted and harmonious. Second, subject matter is not a single unified dataset but multiple datasets that describe a unified subject matter. They are supplemented with non-numeric diagrams and illustrations, embracing their eclectic nature. Unlike most infographics, this specimen makes excellent use of layout to achieve density of information. Related pieces are placed in close proximity rather than relying on sections; the reader is free to explore in any order. This is what an infographic should be, or perhaps it’s worthy of a different and more dignified name, information graphic. It may even approach what Tufte calls “beautiful evidence”.

It’s also possible to implement a data graphic poorly. Usually this comes down to a poor choice of visual encoding, although criticism is somewhat subjective. Take this example of hurricanes since 1960. The circular arrangement is best used for months or other cyclical data. Time proceeds unintuitively counterclockwise. The strength of hurricanes is not depicted, only the number of them (presumably – the radial axis is not labeled!). The stacked bars make it difficult to compare hurricanes from particular regions. If one wants to compare the total number of hurricanes, one is again stymied by the polar layout. Finally, the legend is placed at the bottom, where it will be read last. Data graphics need to explain their encoding first; even better is to explain the encoding on the diagram itself rather than in a separate legend. For example, if the data were rendered as a line chart (in Cartesian coordinates), labels could be placed alongside the lines themselves. (Here is a proper data graphic on hurricane history.)

An infographic typically starts with a message to tell, but designers intent on honesty must allow the data to support their message. This is a leap of faith, that their message will survive first contact with the data. The ivory poaching information graphic never says that poaching is bad and should be stopped, in such simple words. Rather it guides us to that conclusion without us even realizing it. Detecting bias in such a document becomes much more difficult, but it also becomes much more persuasive (for sufficiently educated and skeptical readers). Similarly, poor data graphics obscure the data, either intentionally because they don’t support the predecided message, or unintentionally because of poor visual encoding. In information visualization, as in any field, we must be open to the hard process of understanding the truth, rather than blithely accepting what someone else wants us to believe.

I know which type of document I want to spend my life making.

As We May Have Thought

Vannevar Bush wanted to build machines that made people smarter. His 1945 paper, As We May Think, described analog computers that captured and stored information, the earliest vision of today’s internet. All of Bush’s hopes for chemical photography have been surpasses by today’s digital cameras, and digital storage media are more compact than the most hopeful predictions of microfilm. He also predicts dictation, and though today’s software does a passable but not perfect job, it has not reached the level of ubiquity Bush predicts. He is also wrong about the form factor of cameras, predicting a walnut-sized lens mounted like a miner’s lamp. The result is similar to Google Glass, and no other product:

One can now picture a future investigator in his laboratory. His hands are free, and he is not anchored. As he moves about and observes, he photographs and comments. Time is automatically recorded to tie the two records together.

As for selecting information from the ensuing gigantic database, Bush posits the “Memex”, a desk with displays built into it. The memex is personal, allowing users to connect pieces of information together into trails for later examination. The memex is personal and built on connections, much like the mind.

The late Douglas Engelbart expanded on the purely hypothetical Memex with NLS, short for oNLine System. In “the mother of all demos”, he showed how users traverse and manipulate trees of data, with rich transclusion of content. Unlike the Memex, real-time sharing is possible by way of video chat. Like the memex, NLS was primary text, and the user-facing component was the size of a desk.

And yet … Bush and Englebart’s systems are not psychologically or sociologically realistic. Though Bush was writing in 1945, his vision seemed Victorian: a facade of proper intellectualism with no grounding in the less dapper side of human nature. One can hardly imagine multimedia beyond classical music and Old Master paintings emanating from the memex.  Bush used the effectiveness of Turkish bows in the crusades as an example of what one could research on a Memex. He missed the target. The Memex and NLS were designed for a race of hyper-rational superhumans that do not exist.

The fictitious enlightened user would emphasize restraint, but today’s technology can, for all intents and purposes, do anything. The ability to do anything is less productive and more dangerous than it sounds. Intellectually, such a system encourages slapdash and incomplete thought. It does not force you to abandon weaker ways of thinking; it gives you no guidance towards what will work, what will work well, and what will not work at all. Sociologically, the availability of information on a scale beyond what Bush could have dreamed hasn’t made us an enlightened society. Having correct information about (for example) evolution readily available online has not made a dent in the number of people who read Genesis literally. And it gets worse.

Moore’s law may be the worst thing to happen to information technology. With computing so cheap and so ubiquitous, with the ability to do anything, we have overshot the island of scarcity inhabited by Bush and Engelbart and into the land of social media, entertainment, and vice. The universal systems for the betterment of humanity have fallen to fragmented competitors in an open market. The emphasis on mobile these last six years has led to apps of reduced capability, used in bursts, on small screens with weak interaction paradigms. This is what happens when there’s more computing power in your pocket than Neil Armstrong took to the moon: we stop going to the moon.

Recreational computation is here to stay, but we may yet partially reclaim the medium. Clay Shirky is found of pointing out that erotic novels appeared centuries before scientific journals. Analogously, we should not be deterred by the initial explosion of inanity accompanying the birth of a new, more open medium.

I can only hazard a guess as to how this can be done for recreational computing: teach the internet to forget. (h/t AJ Keen, Digital Vertigo) One’s entire life should not be online (contrary to Facebook’s Timeline – it’s always hard to change ideas when corporations are profiting on them). A good social website would limit the ways in which content can be produced and shared, in an attempt to promote quality over quantity. Snapchat is a promising experiment in this regard. There’s been talk of having links decay and die over time, but this sees like a patch on not having proper transclusion in the first place.

As for programming, though, the future is constrained, even ascetic. If Python embodies the ability to do anything, then the future is Haskell, the most widely-used [citation needed] functional programming language.

Functional programming is a more abstract way of programming than most traditional languages, which use the imperative paradigm. If I had to describe the difference between imperative programming and functional programming to a layperson, it would be this: imperative programming is like prose, and functional programming is like poetry. In imperative programming, it’s easy to tack on one more thing. Imperative code likes to grow. Functional code demands revision, and absolute clarity of ideas that must be reforged for another feature to be added. In functional languages, there are fewer tools available, so one needs to be familiar with most of them in order to be proficient. Needless to say, imperative languages predominate outside of the ivory tower. Which is a shame, because imperative languages blithely let you think anything.

The problem with thinking anything is similar to doing anything: there’s no structure. But if we can’t think anything than some good ideas may remain unthought. There is a tension between thinking only good ideas and thinking any idea. In theory at least, this is the relationship between industry and academia. While companies want to produce a product quickly,  academia has developed programming paradigms that are harder to use in the short term but create more manageable code over time. These are all various ways of imposing constraints, of moving away from the ability to do anything. Maybe then we’ll get something done.

Visualizing Complexity

This post could be considered the third of three responding to Bret Victor’s trio of talks; the previous ones were Abstraction and Standardization and Media for Thinking the Eminently Thinkable.

Question: what makes a program complex? Answer: time.

There are multiple types of time. The obvious one, the one measured by stopwatches, I’ll call physical time. A small program promotes a tight feedback loop, where the effect of a change is visible in a few seconds. Large programs take longer to compile and run, as do those dealing with large amounts of data. To help offset this, programming languages developed threading and concurrency. An expensive task can be queued up, so it will happen at some point in the future. This sort of parallelism makes programs much harder to reason about and test.

Then there’s logical time. This includes event-based programming, which usually involves a GUI. A user’s clicks and drags become events to which the system responds. Code is called when something happens, not when the thing before it ends. Rather than a procedural recipe racing towards the final answer, these programs are state machines, looping indefinitely and branching in response to an arbitrary sequence of events.

Finally, for completeness, there’s developer time. Memories fade, requirements change, people join or leave the project, the technology itself changes. Like people, code deteriorates as it ages, although measures can be taken to mitigate the decline. Any large codebase has annoying “legacy code” kept around for “historical reasons” and “backwards compatibility”.

In his talk Drawing Dynamic Visualizations, Bret Victor presents a drawing tool where the user creates a program by direct manipulation. This program can be used to create custom pictures for multiple datasets (different measurements of the same variables). The results feel very “PowerPoint-y”, the sort of thing you’d present in a board room or paper scientific journal. The method of creation is new, but the results emulate old media.

If you’re reading this, Bret, and think I’m a bit obsessed, (1) thanks for noticing (2) this will likely be the last post about you for awhile.

Users can step through the drawing program, but it’s capable of running instantaneously (i.e. without time and therefore much complexity) . There’s no need for a visualization to wait for a network event, but a good visualization reacts to mouse motions, like hover, click and drag. Interactivity is a crucial component of visualization. Victor has made a great way to make charts for boardroom presentations, but he hasn’t allowed the user to interact with the data. The visualization is “dynamic” in that it accommodates new data, but it doesn’t react to the user. I’m not asking to drag a bar to set the data to that point; we’re working under the assumption that the underlying data is fixed. And I can abstract any color or magic number as a parameter, so I can set data-independent features of the graph like error bars dynamically. But without interactivity, we can only accomplish the first third of Shneiderman’s mantra:

Overview first, zoom and filter, then details-on-demand.

Without interactivity, we can’t zoom or filter or otherwise adjust the visualization to get our details on demand. All we have is an overview. Our visualization sits there, dead. This is not good enough. Data today is so complex that even the best static image isn’t going to elucidate subtle meanings. Instead we need interactivity so that the visualization reacts to the user, not just the designer. I should clarify straight away that I’m not talking about the interactivity Victor rails against in his paper Magic Ink, where the user has a specific, often mundane question she wants answered. (How long will I wait for the bus? What move theater should I go to?) I’m talking about systems where the user doesn’t know what she wants, and needs to explore it to develop an intuition or notice something strange.

There is a disconnect, an asymmetry, in Victor’s work. In Stop Drawing Dead Fish, we had a novel direct manipulation tool and interactivity with its results. In Drawing Dynamic Visualizations, we had a novel direct manipulation tool and dynamic data. In Media For Thinking The Unthinkable, specifically the part on signal processing, we had interactivity and dynamic data. Create with direct manipulation, load dynamic data, interact with the result: pick two.

How can we add interactivity to Victor’s tool? Let’s start with mouseover. As a function of the mouse position, which can be considered an input, hover is stateless. The tool can run geometric collision detection on its primitives and provide a boolean measurement (output) as to whether the shape is being hovered over. If you want to use this information to, say, change the shape’s color, you have to have shapes able to read their own outputs. This can lead to feedback loops. If we draw nothing on mouseover, then we’re not being moused over anymore, so we go back to drawing our shape, which is now being moused over…and suddenly our system has state, and “flickers” indefinitely. Worse, by creating a place for code that is executed only if certain external conditions are true, we create asynchronous, “jumpy” code. This is a large increase in physical and logical time.

Selecting a shape runs into similar problems, an additionally requires a global variable (internal, not a parameter) to keep track of the selection. It gets even worse if we want to add collision detection between objects, like a force-directed diagram. (Actually that’s not collisions, just proximity-based repulsion, but roughly the same idea.) Even if the system manages to detect when two subpictures touch, they now will need a place for code that is called when that is the case. Victor briefly demonstrated how physics can be simulated by using a guide vector to geometrically represent velocity, which we could presumably shift in reaction to the angle of line contact. But we’re starting to think of shapes as their own entities, having state represented in guides associated with them, rather than being drawn and forgotten about. This brings us into Object Oriented Programming, the traditional choice for this sort of interactivity. It’s a great paradigm but it’s quite foreign to Victor’s tool. (Although his Stop Drawing Dead Fish tool lets its user do exactly this. Maybe the line between data scientists and visual artists, or perhaps their tools, is blurring…)

There’s a brief moment where Victor abstracts a color from a shape, and makes it a parameter. This parameter starts off as numeric but changes to a color, likely in response to a keyboard command. If the tool already has a notion of types, that a number is not a color, then there’s a piece of complexity already imported from traditional programming. Types, OOP, collision detection, encapsulated mutable state – all of these ideas have their place in code. There are also a number of mainstays of drawing tools we could include, like gradients, z-order, manual kerning, and the dropper tool. Deciding which of these to include and which to omit, and how to integrate them into the tool, will require some careful design decisions. I wonder how many of these ideas we can pile into the tool before either the graphical interface or the layperson’s acuity collapses under the weight of the complexity.

I, like Victor, and frustrated at the state of software and computing. Most artists are able to react with their medium, to try something, revise it, and use mistakes as inspiration. Listen to how creators in fields as diverse as painting and carpentry discuss a problem. It’s always about sculpting the unruly workpiece material with capable and precise tools, man vs. nature. So why then are computers, where the tool and the medium are one and the same, so limiting that our society can’t see them as anything other than frustratingly complex? Instead of using the medium to find out what she wants to create, the creator has trouble merely realizing what’s in her head. We need a fundamental overhaul of the tools we use for thinking. These tools should let humans, data, and simulation work together. Victor produces tantalizing glimpses of such tools with ease.

Maybe there are true geniuses out there. People who effortlessly sift through complexity, and can make bold pronouncements with confidence. The rest of us have to fake it. We get mired in the cacophany of voices and ideas, paths to choose and tools to use. We do our best to make sense of them but always wonder, in a grass-is-greener sort of way, if we’re missing the point entirely.

Media for Thinking the Eminently Thinkable

Bret Victor has done it again. His latest talk, Media for Thinking the Unthinkable, gives some clues as to how scientists and engineers can better explore their systems, such as circuits or differential equations. He’s previously shown an artistic flair, demoing systems that allow artists to interact directly with their work, without the use of language. In both cases, the metric for success is subjective. What the novel user interface is doing is not determining the “right” outcome but allowing a human to better see and select an outcome. Victor is trying to help users develop intuition about these systems.

That strategy looks promising for a sufficiently complex systems, but when it comes to pre-college education, the goal is not to instill semiconscious, unarticulatable hunches. This relatively simple material demands full, clear, and lingual understanding. This decidedly different goal will, or at least should, guide the design of educational technology in a different direction than Victor’s demos.

What is an educational technology? In today’s world. you’re probably thinking of a game, video, recorded demonstration, or something else with a screen. Those all qualify, but so do manipulatives (teacher-speak for blocks and other objects for kids to play with the grok a concept) and pencil and paper. For the simplest of problems, say 2+2, there’s no need to get out the iPad, just grab four blocks. For advanced math, a computer is necessary. So where is the meeting point, where the see-saw of objects and devices balances?

What follows is a case study of educational technology applied to a specific mathematical idea, multiplication of negative numbers, which is pretty close to that balance point. We’re trying to explain why a negative times a negative equals a positive, a classic grade school trouble spot. (Instead of fixing a specific misconception, we could have to open-ended exploration of a slightly larger topic. Touch Mathematics has some interesting material on that front.)

The MIND Research Institute has developed a series of visual math puzzles that all involve helping a penguin traverse the screen. In their multiplication unit, the penguin stands on a platform in front of a gap. Multiplying by a number stretches the platform proportionally. A negative number will rotate the platform 180 degrees, so two rotations cancel each other. When a student interacts with this system, they gain a useful metaphor for thinking about negative numbers: go left on the number line. However, many of the more advanced puzzles seem contrived, and are difficult to do without resorting to pencil and paper because they don’t provide a cognitively useful metaphor. Moreover, adopting the unit, setting up computers for the students, and setting aside time has significant overhead. Can we go simpler?

Let’s take the rotation metaphor and find physical objects for it. We could use plastic spinners with telescoping arms that allow them to stretch. The arm should lock into place at regular intervals, and the rotation in two opposite directions. I don’t think this will work because the system no longer reacts, and the student has to provide the ideas, and it’s susceptible to mechanical breakage. We could have pentagonal pieces that look like squares with a slight arrow on one side that allows us to indicate direction. Then we place groups of them together, three groups of two for 2×3, and rotate them when we have a negative. But then we have to rotate all of them, which is time consuming. We could put blocks into “boats”, some way of grouping them and assigning a rotation, but that’s even more cumbersome. All of these methods require special manipulative to be purchased, organized, stored, and cleaned. To summarize, I can’t think of a good way to adopt the rotation metaphor into physical objects.

At even less fidelity, we can use square blocks. These are generic enough to justify keeping around for other lessons, and we can have magnetic squares for the teacher to put on the board and actual blocks for the kids to play with at their desks. We can use the grouping idea, and different colors for positive and negative blocks. Here’s how I envision it working:

multiplication

So the blue squares are positive numbers, and the red squares are negative ones, except in the second two examples when we’re on the other side of the number lines, in Negative Land. In Negative Land, the blue squares are negative! So that means that the red squares that were negative are now positive.

That doesn’t make much sense. We’ve created an abstract visualization, so that even as the kids grasp the squares they won’t grasp the concept. The Negative Land mumbo jumbo winds up hand waving away the crux of the issue: why is a negative times a negative positive? We’ve illustrated the idea of reversing something twice but haven’t done much to justify it. Even worse, we’ve created two representations of the same number. The middle two lines are both -2, and the outer two lines are +2, but they don’t look the same.

Even more low tech: repeat the same exercise on paper with Xs and Os standing in for red and blue squares. This is eminently practical but the students now lose the ability to hold something concrete in their hands. We’ve pared the metaphor of flipping twice down to its essence, but lost a lot of its effectiveness in the process.

To quote Edison, “I haven’t failed. I’ve found ten thousand ways that don’t work.” Well, not quite so many, but same idea. Hopefully I’ve illustrated some of the pitfalls in making educational technology both practical and effective. And maybe you learned something about negative numbers in the pro — wait a sec. If explaining different ideas for explanations can actually work, then we may not have to come away empty handed after all.

The computer is assisted imagination. We can take the metaphors expressed most clearly in software and give them to the kids directly. Tell them to imagine themselves on a stretching, rotating platform. Better yet, line up groups of students and have them all rotate together, sensing the math with their own bodies.

The hard part of crafting a lesson plan, whether in person or over technology, is devising the metaphor, a new way to think about a topic so that it makes sense. Once we see negative numbers as rotation, systems of inequalities as unbalanced mobiles, complex numbers as spinners, then the students can explore them. That can be in spoken or written word, symbols, movement, sculpture, drawing, or on the computer. That’s the easy part.

This doesn’t bode well for educational technology companies. If their products are effective, their central metaphors can be easily expatriated to classroom exercises. At best, the metaphor is wrapped in unnecessary packaging; at worst, the packaging is all there is, hawked by cargo cults worshipping motion and interactivity as if these things only exist on a screen.

* * *

An addendum, back to Bret Victor. In a note about the talk, he defines its subject matter as “a way of thinking. In particular — a way of using representations to think powerfully about systems.” (Emphasis his.) He is striving to create the next generation of tools that make unthinkable thoughts thinkable. Only these “powerful, general, technology-independent ways of thinking” will still be relevant a hundred years from now. It’s a daunting, open-ended task, especially considering how much trouble we got into just with arithmetic.

With the announcement of iOS 7, John Maeda criticized not just the OS but the debate it engendered. By phrasing interface design as a binary, of photorealism vs. flat abstraction, “To Skeu or Not To Skeu”, we lose sight of the possibilities that lie before us. Maeda writes, “What we need now is to move beyond the superficial conversation about styles and incremental adjustments to boldly invent the next frontier of interface design.” What do those new designs look like? “Something we haven’t even dreamed of yet.”

Reading visionaries like Victor and Maeda, it’s tempting to join the great quest to fundamentally alter how every person uses software, and by extension, how every person thinks. But part of me doubts the realism of their grandiose pronouncements. On the other side of the coin, Matt Gemmell is an iOS developer very much concerned with the present and its tools. He thoroughly deconstructs iOS 7, seeing it as an improvement, but not extravagantly so. It’s a needed update after six years. In another six we’ll be able to see the next UI paradigm, but he doesn’t waste breath trying to guess it now. Next century is of no concern. Write this month’s app, get this week’s paycheck, enjoy dinner tonight with friends and family. We’ll create the tiniest bit of the future in the morning.

This Is You: Agency in Education

This is the opening of the ambient puzzle game Osmos, by Hemisphere Games. “This is you,” is all it says, as if you’ve always been a glowing blue orb. Most games start by introducing the player to their avatar, but it’s usually a human character with a backstory. Puzzle games are an exception: they rarely give the player an avatar whatsoever. Normally you play an unacknowledged manipulator of abstract blocks according to fixed rules and patterns. Osmos is an exception to the exception.

Osmos also has masterful use of player training and learning curve. It begins in the simplest possible setting with the simplest possible task: “Move to the blue circle and come to a stop.” You accelerate by ejecting mass, which propels the rest of you in the opposite direction. The game tells you these things in the same order I relayed them to you: first objective, then the means. Osmos could have said, “Hey, look how you can move by ejecting mass! Now use this ability to move to this outlined circle.” But it didn’t. The progression is guided, focused, and objective-based, especially at first. The levels build on each other, reinforcing knowledge from previous levels as the player gains experience.

Impasse 1 In a rare moment of explanation, Osmos introduces players to the idea of using ejected mass to move the red motes out of the way so they can get to the blue ones.

Impasse 2 Immediately afterwards, players are asked to apply that principle in a puzzle that looks harder than it is.

Osmos presents players with the Odyssey, an sequence of levels that introduce gameplay concepts in a logical order. The Odyssey runs from the tutorial described above up through medium difficulty levels. After that, players gain access to a Sandbox mode where they can explore different game types at different difficulties. That is, a level of Osmos is distinguished not only by quantitative difficulty by qualitatively, by the kinds of game mechanics and obstacles found. More fundamentally, Osmos is played in discrete levels that can be won, lost, restarted, and randomized, rather than an endless arcade of increasing difficulty like Bejeweled. Players can skip to and play any level they have unlocked at will; a session of Osmos can last three minutes or three hours.

Players are incentivized to complete the Odyssey and get as far as they can in the sandbox, but there’s no climactic end of the game. No explanation is given why some levels seem to take place in a petri dish while others in orbit around a star; it’s wonderfully abstract in that regard. It’s impossible to “win” and there’s no victory cutscene. It’s neither so boring and open-ended you don’t want to play nor so scripted you only want to play once. There are achievements (badges) awarded but they seem extraneous to me.

And now to the point of all this: what can we learn from Osmos when designing software for education?

By the structure of its gameplay and incentives, Osmos lends itself to the sporadic and time-limited technology access found in many schools. Instead of leaving students behind who didn’t win the game, or trying to pry a child who’s “gotten far” away from the computer or tablet, it’s easier to take a break from Osmos. Meanwhile the nature of gameplay means that it’s very much a solitary experience, a personal journey of discovery. For all the hype given to social gaming over the last few years, it’s not conducive to deep thinking.

And yes: agency, in the sense of being a specific agent. In Osmos, the player is someoneor at least something: this is you. As I’ve said, most puzzle games don’t give the player anything to latch on to. Neither does formal arithmetic, nor algebra. Symbol manipulation provides no agency. It forces mathematics into the unnatural third person perspective (unnatural from a human’s point of view). When I played with blocks as a child I would often imagine an ant climbing and exploring my structures. Pen and paper mathematics allows the mathematician to move blocks around but not to be an ant inside his or her own creation.

Seymour Papert developed LOGO to provide children with agency. LOGO is a cross between a game and a programming language. Players manipulate the “Turtle” by telling it to turn left or right or move forward, where forward is relative to how it is turned. When children first encounter difficulties, they are told to “play turtle”. By moving their own bodies through space, they are able to debug and refine their program. And by thinking how they move their own bodies through space, they are given a tangible grip on both computation and thinking.

Scratch was developed at the MIT Media Lab, which was co-founded by Papert. Scratch, though very much a descendent of LOGO, adds more commands to increase the user’s power and control. Many of the commands were discussed by Papert in his book Mindstorms or seem to be reasonable extensions of it. Others (thought balloons, sounds, arithmetic, color effects) are superfluous. Still others, like if-else statements, while loops, and Boolean operations, are taken from the nuts and bolts of programming. This comes at the cost of downplaying the two high-level skills which Papert thought were so vital to learning any subject: subprocedures and debugging. With LOGO, children learned to compartmentalize knowledge into reusable pieces, and to make incremental improvements to see the results they wanted.

One of LOGO’s defining characteristics was its limited set of commands, which are relative to the current position and heading of the Turtle. Osmos players can eject mass in any direction, but nothing more. In both cases, artificial scarcity of control forces users to think in a particular way. On the other hand, Scratch freely mixes LOGO-style “move forward” with Cartesian commands, both relative (“move up by”) and absolute (“move to”). It’s impossible to have agency with something that can be teleported across the map. Rather than force the user out of lazy and weak ways of thinking, Scratch offers multiple paths and users take the one of least resistance. Often this will be a hodgepodge of many different styles and systems of commands, reflecting incomplete and imprecise thinking.

The large number of commands create a cluttered and unintuitive interface. 78% of Scratch’s default interface is control while only 22% of it is the canvas. The results, the thing the user cares about, are squished in a corner. Osmos has minimal controls that disappear when not in use, leaving the entire screen as the portal into the game world. Moreover, Osmos has just enough visual detail to be eye candy and not clutter. Games, in general, have excellent usability because bad usability is by definition not fun.

Scratch’s default user interface, with overlaid percentages. A similar image for Osmos would be 100% purple.

The differences in the command set and user interfaces belie the different purposes of the software. Scratch is meant to provide a canvas for a play or a animation, and so gives the user plenty of options for control. Osmos and LOGO are both puzzles in the sense that the controls are extremely few, yet powerful. A tool is designed to give a competent user maximum power to create; a puzzle is designed to teach new ways of thinking under constraints. By this metric, Scratch has more in common with CAD software used by engineers to design mechanical parts than it has with Osmos and LOGO.

But there is another feature that groups the three differently. Both LOGO and Scratch are sandboxes; they enforce no requirements or value judgements on the player’s actions. Papert envisioned a teacher guiding the student and keeping her on task. Osmos takes a different route. As a game, it has clear objectives to complete and obstacles to avoid. There are good moves and bad moves. There are levels, with definite beginnings and ends. The Odyssey is just a long tutorial: it presents each feature and some advanced ideas before handing the player full control. Scratch and LOGO do just that as soon as they’re opened. In particular, Scratch provides no guidance on its cockpit’s worth of controls.

There is a misconception, common among edtech types but not among traditional teachers, that the answer to all problems is better distribution. People are ignorant because they don’t have access to knowledge. People can’t code because they don’t have access to the software and documentation. But this is simply not true. Give people tools and they won’t know what to do with them or how to use them. Instead, we need to give students of all ages training, knowledge, and understanding. We need to force students to think about wrong ideas and make them right, and to see why they are right. We need to to show students the metacognitive tools to solve problems. An educational game isn’t about what to think, but how to think.

Now read the follow-up post: Beyond Agency: Why Good Ideas Only Take Us So Far.