When they think about note-taking, most people think about textual notes. But it’s also often important to take graphical notes. It’s tricky to develop cognitively productive workflows for note-taking in general, and graphical note taking in particular. An example of the latter point is the following topic on the Mac Power Users forum:
Adding diagrams to Zettelkastens: (Luhmann) Note-taking App that can form a Quick Access Knowledge Base.

This blog post sheds some light on the importance of diagramming from a cognitive science perspective. It discusses problems in understanding mental representations underlying the interpretation and production of diagrams. It explains why people don’t take as many graphical notes as they should, and what they can do about it. And then it goes (or you go) to sleep.

That’s a lot of ground to cover in a brief blog post. However, the post contains links to some extremely interesting articles. If you read the source materials, I’m sure you will find they stretch your imagination.

On the benefits of drawing and diagrammatic reasoning

Fernandes, Wammes, and Meade (University of Waterloo) published an interesting paper recently that demonstrated benefits from drawing. They argued that the benefits were due to the fact that drawing requires integrating information in multiple mental representations: elaborative, pictorial and motor representations. Their data suggest that drawing is an important strategy that can counterbalance the effects of aging, because it draws on mechanisms that are more robust to brain aging.

Another argument for the benefit of drawing is that it relies on proto-gestures, which predate theoretic knowledge in human evolution. (Although this was not mentioned by Fernandes, Wammes, and Meade, it does support their argument.) This is an extrapolation from Merlin Donald’s excellent book, A Mind So Rare. Often, gesture survives stroke. And deaf children schooled together spontaneously sign (cf. studies on Nicaraguan deaf children). And often Down’s syndrome children can sign when their speech is otherwise affected. (Ask your friendly neighbourhood speech and language therapist to comment on this. I am married to one.)

Similarly, one should expect that gesture without drawing (also not discussed in Fernandes, Wammes, and Meade) should also be useful for mnemonics.

In fact, one should expect that imagined gesture and imagined drawing can also be very effective for mnemonics.

I myself have been using overt and imagined gesture in productive practice for years. While that is anecdotal evidence, it is also what you would expect from research on the benefits of mental simulation (imagination), a technique used to great effect by many professional athletes. (If my memory serves me well, Wayne Gretzky said he used this technique.)

Analogical representations underlie diagramming

But what is going on in the mind while one draws diagrams or reasons with or about them? That is a tough problem that hasn’t yielded to cognitive psychology, i.e. to empirical methods. In fact, it can’t be solved by cognitive psychology per se. It requires building, or trying to build, artificial intelligence that can execute the tasks at hand. (That is to say that it calls for cognitive science, an interdisciplinary approach.)

I will return to diagramming later. For now, I will simply refer you to these very interesting and amusing papers.

• One of the first papers in Artificial Intelligence on the subject was by Brian Funt, a colleague and friend of mine at Simon Fraser University: “Problem-Solving with Diagrammatic Representations. I happened to cite his 1980 paper in my own Ph.D. thesis.

Aaron Sloman is the most prolific AI researcher in this area. Some papers that I highly recommend:

• Intuition and Analogical Reasoning
• Afterthoughts on analogical representations.
• Diagrams in the Mind? (a PDF). This one includes an amusing example involving Mr. Bean swapping his underwear for a swimsuit without fully undressing. (Here’s an interesting challenge: prove a priori (not empirically!) that this is more difficult than changing bras without undressing.)

Aaron’s work in the last decade is focused on understanding some of these very hard problems in AI / cognitive science, in particular the evolution of cognitive capabilities underlying our abilities to know that something is true. (If ever an AI marketer tries to convince you that human-like intelligence is just around the corner, ask them to explain how they have solved meta-morphogenesis problems, which Aaron believes Alan Turing may have been working towards.)

Does the mind create images or simuli?

Marvin Minsky suggested that it’s misleading to refer to mental representations as images. He coined the terms “simuli” (“simulus” , singular) to refer to schematic analogical mental representations. See “Arguments against the pictorial nature of imagery” in Mental Imagery (Stanford Encyclopedia of Philosophy).

How special are we?

Depends who we are. Consider:

New Caledonian Crows Use Mental Representations to Solve Metatool Problems

Enter Hook

Granted that drawing diagrams is beneficial, why is it that people don’t draw as many diagrams as they should? Here are some reasons:

1. Drawing diagrams, even with old technology (paper, pencil and ruler) is difficult. It involves more operations, and more difficult knowledge building, than verbalizing.
2. It’s also partly a matter of habit. Most people are not (or no longer) in the habit of drawing diagrams. (It’s easier to do things that you habitually do. A potentially virtuous circle.)
3. The technology isn’t (or wasn’t until recently) conducive to diagramming. I explored this angle in a recent blog post. For instance, not everyone has a great drawing app at their disposal. macOS does not ship with anything like OmniGraffle (it should). And even if it did, to be helpful there needs to be a mechanism to link diagrams to just about anything one might be reading about or learning.

That’s where Hook comes in. Hook is a cognitive productivity app for macOS. With Hook you can link your diagrams to screencasts, web pages, PDFs, emails, and more.

Link your iPad Pro drawings on your Mac

While Hook is currently only available for macOS, if you have an iPad with Apple Pencil (which I highly recommend), then you can draw in Apple Notes on your iPad, and link your Apple Notes with Hook on macOS to just about anything. For instance, you can create a link between a PDF and a drawing; or link a web page and a drawing.

There are also tools that allow you to draw with a pencil on your Mac. Those drawings can be linked with Hook.

Can this help you fall asleep?

This might help you fall asleep in two different ways. If you’re not interested in cognitive science (after all, Some Mothers Do ‘Ave ‘Em), then this might bore you to the point of sleep. (But you’d not be at all likely to understand how boredom can, or can’t, actually make you sleep).

More interestingly, in my somnolent information processing theory, I’ve posited that sleep onset imagining (not simply “imagery”) can facilitate sleep onset. There are papers here: http://www.sfu.ca/~lpb/pubs.html. And in fact, I am supposed to be writing a paper on the subject, rather than blogging. Ooops!