On online collaboration and our obligations as makers of software
2 June 2022 | 12:56 pm


On online collaboration and our obligations as makers of software
2 June 2022 | 12:56 pm

My reading corner at home. An old comfy chair. Bookcases. Books stacked
My reading corner.

This post is the final entry of three in a series where I go over what I learned from the user research I’ve been doing for Colophon Cards. The first was about markdown. The second was about the various different kinds of notetaking.

It all began with blogging

As I’ve mentioned earlier in this series, I’ve been blogging for almost two decades. What I didn’t mention was that I was a true believer in ‘Blogging’.

So, you believed in having a personal website, what of it?

No, you see, at the time blogging was supposed to be so much more than that. I truly believed that it heralded the future of collaboration, communications, publishing, and intellectual discourse. I believed that the future of social media was an interconnected web of personal blogs. I believed that the future of intellectual discourse was in cross-blog debates on lofty topics.

I believed the hype. I was quite wrong.

Blogs didn’t quite die. You could even say that they won as they have become a cornerstone of the web’s infrastructure:

  • 40% of the web runs on WordPress, which is blog software.
  • Substack and similar newsletter platforms offer a reverse-chronological list of posts and an RSS feed and are basically rebranded blogs.
  • Podcasts are blogging with audio files.
  • The affordances invented by blogs are everywhere. Many news sites maintain ongoing daily liveblogs on specific events, such as COVID-19 or the UK’s political scandal of the day. They may not call it a liveblog, but that’s what they are.
  • Much of the web’s automation is built on feed data that’s shuffled from service to service. Feeds are the quintessential blog tech.
  • Blogs transformed how we approach content management systems (CMSes), even when those CMSes don’t use any blog-related design affordances.
  • Youtuber video essays are an evolution of early attempts at “video blogging” or video-based podcasting.

Blog tech underpins almost everything you see online.

So, while blogs didn’t quite have the profound effect on society, publishing, and media that I expected them to have, their influence is respectable and far-reaching.

I am biased in favour of blogs, though. They had a major influence on my intellectual development, to the point where I still to this day revisit old blog post favourites for fresh inspiration.

One such major influence, one that probably changed my entire outlook on both writing and software development, was Kathy Sierra and her blog Creating Passionate Users. Her writing sent me down a path of studying the learning process, skills development, expertise, mastery, and why they matter to software development and UI design.

And Kathy Sierra’s social media story—that began with a group of self-styled ‘mean kids’ revelling in mean jokes and ended with literal nazis calling for mass murder—is the reason why I fell out of love with weblogs and began to distrust the tech industry’s approach to social media and online collaboration.


Getting back to notetaking, or the one where I piss off the punditry (again)

I reject the ‘external brain’ hypothesis for notetaking.

Or, more specifically, I think it’s a counterproductive metaphor for notetaking.

The hypothesis is straightforward: a complex notetaking system, built in the form of a structured web, acts as an extension of your brain and makes you smarter.

The theory is sometimes stated; sometimes merely implied. But it’s the trend among those making notetaking apps or selling courses on notetaking to talk as if structured, interlinked notetaking will raise your IQ, help you think more clearly, and prompt a wellspring of new ideas. Your collection of notes, dutifully linked and tagged, woven into an impenetrable fabric of collections and context, are therefore serious work and the promise is that all of that effort will be rewarded with intelligence, memory, and genius. The notetaking system is complex and tough to use because it’s supposed to expand the cognitive space that we use to structure our thoughts and generate ideas.

The problem is that even if our observations about these notetaking systems are correct, we don’t know if we have the arrow of causality pointing in the right direction.

Maybe that German academic was a genius and he definitely used a complex notetaking system. But was he a genius because he had a complex and involved notetaking system? Or did a complex and involved system work for him because he was a genius?

Is it the notetaking system that’s helping you think more clearly? Or is it the act of writing that forces you to clarify your thoughts?

Is it the complex interlinked web of notes that helps you get new ideas? Or is it all the reading you’re doing to fill that notetaking app bucket?

Is all of this notetaking work making you smarter? Or is it just indirectly forcing you into deliberate, goalless practice?

If the external brain hypothesis is correct, then complexity is essential. You can’t have a neurological extension of the brain without neurological complexity. The involved structures and regimented processes exist to tap into specific structures in our brain. There might be variations but all should have a baseline complexity that can’t be abstracted away because, hey, the brain is complex.

That’s the theory, at least.

However, if my suspicions are correct, then the primary benefit from notetaking comes from regular, deliberate practice. It doesn’t matter if you’re sketching, journaling, collaging, jotting down bullet points, recording a daily video, or just writing. It doesn’t matter if you’re filing it all away with a detailed ontology into a structured database or if you’re dumping it all into a box. It’s the habitual practice—in a way that fits your skill, personality, and practices—that matters.

If I’m right, then you can get the results of a complex notetaking system with a lot less work. Or, to be more specific, with a lot less wasted work—it should all go into the writing (or sketching, recording, etc.).

The key is that the object of notetaking is never to take notes. It’s to do a better job:

  • Build better software.
  • Write better blog posts or finally finish that novel.
  • Renovate that spare bedroom into something nice.
  • Become more thoughtful about what you buy and eat.
  • Write a better thesis.
  • Make better art.
  • Take better photographs.

Notetaking should always be less work than the actual work you’re doing. When the notetaking process takes over, something has gone seriously wrong.


How blogs influenced my thinking on the subject

Kathy Sierra’s work has a single recurring idea from the very beginning of her first blog, Creating Passionate Users to her amazing 2015 book Badass: Making Users Awesome: the job of the tools we make is to set the user on a path of mastery. The tool shouldn’t get in the way of them improving their skills. It shouldn’t overload the UI with distractions. It should enable experimentation and practice. It should be designed in such a way as to enable and foster mastery.

That requires us to understand what mastery and skills development is, how it comes from deliberate goalless practice, exposure to examples of work that exhibit the skills the user aspires to, from having a tight feedback loop that lets the end user be aware of how they are working when they need to, from increasing the resolution of the work when applicable. And the generally accepted key to mastery (or skill-specific expertise, if you want to be nitpicky), is twofold: deliberate practice and perceptual knowledge.

In Kathy Sierra’s words, from her book Badass, first on practice:

Practicing harder and longer can potentially make us even worse than if we did less practicing.

Building deep expertise takes work, but of a very specific type that’s often the opposite of what people do when practicing.

Practice does not make perfect

On perceptual knowledge:

The second attribute of those who became experts is this: they were exposed to high quantity, high quality examples of expertise. (page 128)

After enough exposure with feedback, your brain began detecting patterns and underlying structures, without your conscious awareness. With more exposure, your brain fine-tuned its perception and eventually figured out what really mattered. Your brain was making finer distinctions and sorting signal from noise even if you couldn’t explain how. (page 133)

These two ideas, deliberate practice and exposure to high-quality examples, should be the forces that drive the design and structure of most apps. That includes Colophon Cards.


The research so far

As I wrote in my last entry, I began my research into writing and notetaking well before I began the Colophon Cards project itself. I came to the project with a set of ideas. I’d based them on my research of existing studies and writing about expertise, mastery, creative work, and knowledge work. (And I reviewed a ton of forum threads on notetaking.)

The purpose of the survey and the interviews was to see if I could validate or invalidate some of my guesses before I advance to the implementation stage proper. Better now than after I’ve implemented something.

Finally, writing these entries where I analyse and outline the observations from the interviews while explaining where they stand relative to my other research, helped me consolidate my ideas into a cohesive theory of how a subset of the notetaking field works.

The research I’ve done isn’t enough to lend my theories scientific legitimacy (for that I’d need way more money) but it’s enough to give me confidence in going forward.

My hunches:

  • When it comes to notetaking, knowledge work (e.g. most kinds of office work that require expertise of some sort) has more in common with creative work than not and benefits from similar approaches.
  • That most kinds of notetaking don’t need complex organisational structures.
  • That two-dimensional spaces (like a pinboard) are underused as organisational metaphors in creative software.
  • That most of the usefulness of notetaking doesn’t come from a system, organisation, or references but the act of writing.
  • That, ultimately, the goal of the notetaking is to make something for somebody else.

There are many kinds of notes, but these are the kinds of notes I’m hoping to support. The app should help those in creative or knowledge work become better at their jobs.

Most of these kinds of jobs involve other people.

Couple that with the need for the high-quality examples required for mastery and the key to a great personal notetaking app is then, paradoxically, other people.

Other people (the various types of collaboration)

It can be jarring to listen to people describe the many ways they work with other people. Especially if you’re only used to the limited takes on collaboration that are the norm in the software industry.

Tech today favours only two approaches to collaborative work:

  1. Multiplayer spaces that are shared in real time (or close to real time).
  2. A library where everybody sees and contributes to the same shared space.

Both are essentially the same idea: a space with realtime synchronisation and presence signalling; and a space without.

Examples of the former include Google Docs, Slack, Figma, and Zoom. Their design makes you aware of everybody else in the space, and their actions affect your perception of that space in real time (or close to real time).

Examples of the latter include Wikipedia, various kinds of knowledge bases, Dropbox and Google Drive, and version control systems. They are all about making it easy for a group of people to work simultaneously in a space but without realtime awareness of each others actions.

The makers of both kinds of apps often invest a lot of engineering hours into making sure that multiple people can work on the same document or file at the same time without causing conflicts or losing data.

Other approaches that used to have favour in tech but are now out of fashion, despite their evident popularity among end users:

  • Forums and discussion groups. Arguably the world’s favourite form of online collaboration, used for everything from coordinating the development of multi-million dollar software projects to a family picnic.
  • Email: a federated messaging system where each user gets a personal data repository to manage shared data and messages.
  • Folksonomy or collaborative tagging. Helps groups of people organise an ad hoc library of documents, web pages, or files.

Most discussions of how to make collaborative software focus on one or more of these approaches.

This is like describing a chair in terms of the hammers, saws, and screwdrivers used to make it. Great when talking to other carpenters who want to make another exactly like it. Not so great when you want to find out whether the chair works for sitting.

We need to have a clear idea of what people are trying to accomplish with collaboration. Knowing the tools we can provide them with as UX designers is not enough.

A non-exhaustive list of the primary goals of most collaborative office work:

  • Consensus-building.
  • Feedback and review.
  • Knowledge sharing.
  • Parallel work.

Consensus-building

The point of most meetings, brainstorming sessions, and workplace discussions isn’t to generate ideas, make decisions, or understand the situation. The point is to build consensus. The ideas or decisions are the most straightforward part of the process. Without consensus, they are all meaningless.

Constructive examples from the interviews include:

  • Meeting minutes in Google Docs or Hack.md to make sure that they reflect the group’s consensus on what was said and decided.
  • Plan of action proposals that needed to be approved by several people to happen.

For consensus-building, you need everybody to have an overview of what everybody has said, who has participated, and what the overall argument has been.

Any time you have a group of people together where everybody is aware of everybody else’s presence and status, you tend to automatically end up with a group consensus, even if that wasn’t what you wanted. We are a social species, and we are really good at adjusting our own opinions to match that of the crowd.

The Bandwagon Effect (and the equal and opposite anti-Bandwagon Effect) is what drives most online social interactions. It’s what tore blogs apart; what makes Twitter a hellsite; and what turned Facebook into the propaganda machine it is today.

It also tends to lead to bad outcomes in collaborative work.

Consensus-building is an essential part of work collaboration but it can’t be the only part. The dominance of consensus-building online collaboration tools directly leads to worse decisions.

Here’s Kathy Sierra again talking about how this phenomenon is described in the book The Wisdom of Crowds (which, paradoxically, talks a lot about how crowds are dumb):

And that’s all great and intuitive… until you get to humans. Humans, he said, demonstrate the opposite principle: more interactions equals dumber behavior. When we come together and interact as a group seeking consensus, we lose sophistication and intelligence. Ants get smarter while we get dumber.

And.

At its simplest form, it means that if you take a bunch of people and ask them (as individuals) to answer a question, the average of each of those individual answers will likely be better than if the group works together to come up with a single answer.

Feedback and review

Another common scenario is to get feedback or a review of an idea, proposal, or piece of work. You have a thing. You would like to improve said thing. So, you ask a bunch of people what they think, giving more weight to those with relevant expertise. It’s a time-tested strategy.

The pitfall here is that if the participants are aware of each other’s contributions, they will almost always automatically switch to consensus-building instead of providing their honest feedback. Worst case scenario: the bandwagon effect gathers steam and drives you toward a crap decision.

These kinds of disasters are a routine occurrence today. You see them in corporations, small and large, in movie studios, at publishing houses, and in governments. A product or project is released and all of a sudden you have a bunch of people in your inbox pointing out glaring, obvious, and fatal flaws that everybody involved had missed.

To get the best feedback possible from the participants, you need to avoid the mechanisms of consensus-building. You need to ensure that everybody’s responses are kept separate and only visible to those responsible for integrating all of the feedback, something that’s surprisingly difficult to accomplish in modern collaborative software.

The interviewees all had examples where they needed to get feedback from a specific person on something they did. But only those who were specifically writing or teaching as a part of their job did feedback in a structured way. Most of the examples of feedback or reviews that came up in the interviews were more accurately examples of consensus building as described above as they all involved:

  • Groups of people.
  • Who could see each other’s work.
  • And who debated until they all agreed.

That’s just the consensus trap all over again. Which is fine, if that’s what you actually wanted, but counterproductive if needed that feedback.

Knowledge sharing

This is the one everybody loves to try to solve because it’s a problem that everybody seems to have. The reasoning is straightforward and compelling:

  • Most organisations have a lot of documents and data floating around that hardly ever gets revisited or used.
  • They all have research, reading, and relevant information collecting dust.

Stuff that should be informing the decisions and strategies of the company. Some of it sits unread in a knowledge base or a wiki. Some of it lies in the drives of individual employees who don’t have a way to share it productively.

So much knowledge not being applied!

Except that’s not how we work as human beings. If you haven’t read it, experienced it, and contextualised it, then it isn’t knowledge to you. Knowledge is a quality that people possess, not documents, and the only way to transfer it from one place to another is for people at both ends to apply themselves and make it their own.

In terms of preserving and transmitting knowledge, most of the systems being used are working about as well as they ever can, and even if you could improve upon them, it’s highly unlikely that anybody would pay you for that improvement as it’s a pizza crust problem, to use a phrase coined by Amy Hoy.

Using the term “knowledge” for knowledge sharing and knowledge bases is misleading. It’s the term of art, but it’s also wrong enough to lead you to make poor decisions. We should be talking about documentation and references. In other words: a library. At scale, when you’re dealing with a large collection of documents and papers, you’re probably big enough to hire somebody (or a team of somebodies) with an information science degree (or degrees).

If you’re big enough to have a library where everything gets lost all the time but still too small to hire people who have the relevant expertise to solve the problem, you’re kind of screwed. Search only works up to a point.

Thankfully, if keep your scope limited (small organisation or single department) there’s this neat thing that digital files can do that saves the day: digital files can be in multiple places at the same time.

The issue with large shared document libraries at most companies is:

  1. They generally don’t have an organising principle beyond the personal tastes of the employee who volunteered to organise the Google Drive.
  2. If they do have an organising principle, it isn’t understood by most of those who use it, which leads them to move things around and leave items in the wrong place.
  3. Almost every time somebody puts a document in a place that makes sense to them, it’s lost to everybody else.

Information science people know how to solve these problems and others that come up, but what if you don’t have one of those hanging around?

The simplest solution, one that works surprisingly often, is for everybody to maintain their organisation for the document library.

To use the shelf analogy: if you have a large enough shared bookshelf and you move a book to a place that makes sense to you, then that book is immediately lost to those who knew where it was in the old place.

If you, on the other hand, have a bookshelf for each employee and unlimited copies of all of the books, then each person can organise their version of the library in a way that they understand and can navigate easily.

If you ever wondered why every office has at least one person who emails themselves everything—files, PDFs, pictures, whatever—now you know.

This approach has obvious limitations. It’s crap at helping you find stuff that came up before you started work in the organisation and it doesn’t scale to a large organisation.

A bookmarking service that supports collaborative tagging (folksonomies) can often fill in that gap.

Doesn’t always work. Even when it doesn’t, it’s probably still going to be better than the black hole of data that passes as your shared Google Drive or Dropbox.

Almost everybody I interviewed described knowledge sharing as an ideal they aspired to but none of them practised it.

Parallel work

Finally, we get to the other kind of collaboration that modern software is good at (the first being consensus-building, which apps and the web excel at, to a fault).

Namely, working people down to the bone—I mean, helping people work in parallel.

With Google Docs, you can get a group of writers to work in parallel on whichever part of the document you want. With Git, a group of developers can be working on separate problems in the same codebase without issue. With Figma, you can have all of your designers working on your design library at the same time and the app will handle it without a hiccup.

This can be great and this can be awful. The great part is obvious: sometimes you can sic a bunch of people at a problem and they’ll tear it apart. But also sometimes a problem can only be solved by a single person with expertise and others would only get in their way.

The risk with these tools is that they are also frequently inadvertent consensus engines. Because tools like Google Docs and Figma show the work in a single shared space and make you aware of the actions and presence of others working in said space they end up having the same drive towards consensus as you get in UIs specifically designed to help a group come to a consensus.

Amazing division of labour; mediocre outcomes.

In Kathy Sierra’s words:

I’m not dissing teams–our books are all collaborative efforts, and far better because of it. And we consider ourselves to be on a team that includes our publisher O’Reilly. It’s not teams that are the problem, it’s the rabid insistence on teamwork. Group think. Committee decisions.

Most truly remarkable ideas did not come from teamwork. Most truly brave decisions were not made through teamwork. The team’s role should be to act as a supportive environment for a collection of individuals. People with their own unique voice, ideas, thoughts, perspectives. A team should be there to encourage one another to pursue the wild ass ideas, not get in lock step to keep everything cheery and pleasant.

Everybody I interviewed had experience with parallel work, thanks to Google Docs. They all mentioned having worked on a document or a spreadsheet in parallel with others as a part of their job. The developers all used GitHub as well.

Putting it all together

This sounds impossible! Might as well give up.

It would be impossible if you tried to solve all of the problems I’ve described with one app. Instead what the research helped me do is narrow down the focus of the app and validate or invalidate some of my assumptions.

As I described in the first entry in this series, heavy users in the notetaking scene are extraordinarily sensitive to lock-in and interoperability. While I’m hesitant to base the app on markdown—I think the format is a major roadblock for many on the path to master writing and notetaking—I need to make sure the app offers similar interop and compatibility guarantees as you get from a plain text format.

In the second entry I described the different kinds of notetaking people use. The research helped me decide that I wanted to focus on making a notetaking app for structured creative work, operating on the assumption that knowledge work has more in common with creative work than not. I also reviewed some of the different ways experienced writers commonly structure their notes and research, focusing on two that I called The Box and The Map.

And in this entry, I’ve been describing how practice and feedback are a vital part of helping people develop skills and expertise.

Through this work I’ve been finessing the concept of the app and cutting it down to four main spaces or views, each with a specific purpose, all connected through feedback loops.

  1. The Journal
  2. The Dot Grid
  3. The Reading Corner
  4. The Sharing Space

The Journal is your entry point. It ties everything together and prompts you to describe or contextualise the notes and bookmarks that you’ve added, edited, or deleted that day. As I described in my last entry, almost every book on writing or creative work strongly recommends a daily, goalless journal. It needs to be daily (or close to daily) because it’s practice. It needs to be goalless because it’s deliberate practice. You aren’t writing it to meet a deadline or finish a project. It’s there for you to empty your head, so to speak.

The Dot Grid is The Box. It’s the semi-structured space you use to collect your things.

Roy Peter Clark uses literal boxes, one for each project, and describes the process in his book Writing Tools in terms of saving string in a box:

To save string, I need a simple file box. I prefer the plastic ones that look like milk crates. I display the box in my office and put a label on it, say “The Plight of Boys.” As soon as I declare my interest in an important topic, a number of things happen. I notice more things about my topic. Then I have conversations about it with my friends and colleagues. They feed my interest. One by one, my box fills with items: an analysis of graduation rates of boys versus girls; a feature on whether video games help or hinder the development of boys; a story about decreasing participation by boys in high school sports. This is a big topic, so I take my time. Weeks and weeks pass, sometimes months and months, and one day I’ll look over at my box and hear it whisper, “It’s time.” I’m amazed at its fullness, and even more astonished at how much I’ve learned just by saving string. (p. 215)

The Reading Corner is where you read and review both the articles you’ve bookmarked and your own notes. It’s important when reviewing other people’s writing and your own that both happen in an identical environment. In theory, it should strengthen your perceptual knowledge to edit your writing in the same space as you read the (presumably) high-quality research that you’ve collected.

Finally, The Sharing Space is the same as The Reading Corner except it’s specifically for getting feedback from other people. You should be able to take a note or a draft and get independent feedback from each of your reviewers.

That’s the theory and the concept. This is what I hope Colophon Cards will be.


Our obligation as makers of software, as explained by the collapse of the blogosphere

In early 2007, a small group of popular bloggers started a site called Mean Kids, a tongue in cheek reference to some of the participants having previously been likened to “the mean kids in high school”. It claimed to be satire that poked fun at some of the people in the blogging world that annoyed them. That included Kathy Sierra (among others, it was a really mean-spirited effort).

The problem with the Bandwagon Effect and the anti-Bandwagon Effect online is that it leads to escalation. Everything keeps escalating unless you specifically take measures to prevent it, right at the beginning, which this group didn’t (out of ideological opposition to content moderation) until it was too late. The tech industry is filled with well-meaning enablers of abuse and 2007 wasn’t any different.

These events were well documented at the time. This take, by Joseph Reagle, is as good as any.

The man who escalated the furthest was Andrew “Weev” Auernheimer who has since confessed several times to not only spreading lies about Kathy Sierra but also publishing her home address and social security number, encouraging people to send her ‘presents’ that showed what they thought of her.

And they did. Some of them were death threats.

So, Kathy stopped speaking publicly and stopped blogging. She retreated from the public web while Weev rose to stardom as a ‘hacktivist’. Many people in tech adored him for fighting against unfair laws that ‘stifled’ progress in the industry.

This is when I lost faith in blogs and online social media. This dynamic, repeated endlessly since then, is why distributed social media like the ‘blogosphere’ collapses; why Twitter is a hellsite; why Facebook is the place that turns Granddad into a nazi; why federated social media like Mastodon breaks apart into smaller isolated clusters.

The tech industry either doesn’t understand the harm being done by their idea of how social interactions work or it doesn’t care.

More than any other, this is the event that made me a sceptic of the promises of progress that the software industry is fond of making. It hammered home the obligation the rest of us have, those who do care, in making sure that the software and technology that we work on don’t enable the kinds of abuse and harm that keeps happening with mainstream social and collaborative media.

No app I make or work on should prioritise collaboration and work over the well-being of others. The obscurity or lack of popularity of my software or yours should not be the only thing that prevents it from being used for abuse.


Kathy Sierra returned to social media and the public web in 2013, joining Twitter as seriouspony and launching a website to promote her upcoming book.

For a while it seemed to be going well. The book, after all, is amazing. I highly recommend it. But then the crap kept coming. People kept repeating the lies Weev had told about her. He kept being heralded as a hacktivist and an important man in the fight against government overreach. Every time she tried to push back against the lies, she ended up being the one accused of being a troll.

The tech industry loves its abusive dudes.

So, she decided to stop. Only a few months after her return, she wrote a post explaining why, republished by Wired as Why the Trolls Will Always Win:

I don’t have the luxury of assuming “it’s just online. Not REAL. It’s not like these people would ever do anything in the real world.” And what you don’t hear much about is what most targeted women find the most frightening of all: the stalkerish energy, time, effort, focus on… YOU. The drive-by hate/threat comment, no matter how vile, is just that, a comment that took someone 2.5 seconds to think and execute. It might be annoying, offensive, maybe intimidating the first few times. But you get used to those, after all, it’s not like somebody put time and effort into it.

But Photoshopped images? Stories drawn from your own work? There’s a creepy and invasive horror knowing someone is pouring over your words, doing Google and Flickr image searches to find the perfect photo to manipulate. That someone is using their time and talent to write code even, about you. That’s not trolling, that’s obsession. That’s the point where you know it’s not really even about the Koolaid now…they’re obsessed with you.

This is a very long way from the favorite troll talking point “Oh boohoo someone was mean on the internet.”

Being online wasn’t worth the risk, so she left.

Since then Weev came out as a Nazi, got himself a swastika tattoo, and more than once has excused or advocated for mass murder while maintaining one of the more prominent online neo-nazi communities.

Despite this, I haven’t seen a single person who took his side over Kathy Sierra’s recant and apologise. Many instead doubled down, saying that in principle they were correct to defend a nazi’s freedom of speech.

At most, when you press them, you get a vague apology about how in nerd circles you have to have a tolerance for strange politics. Which is why it was apparently impossible for them to register what sort of person Weev was until it was too late.

It should worry you that these people, and others with similar priorities and similarly poor judgement, are still so influential in the tech industry.

And the rest of us need to try to be better.

The next steps for Colophon Cards

Next, it’s time to start implementing the prototype proper and start testing actual designs.


The Colophon Cards project is made possible by a grant from The Icelandic Centre for Research’s Technology Development Fund


The different kinds of notes
6 May 2022 | 12:56 pm

This is the second entry of three where I go over what I learned from the user research I’ve been doing for Colophon Cards.

The ratty old spiral notebook that is the oldest notebook I own
My oldest notebook

The post where I outline my general theory of notetaking for all to disagree with

At the end of my last post, I mentioned that I had switched to pen and paper notetaking in the early 2010s, after years of serious digital notetaking using plain text files.

I wrote about how much more enjoyable the analogue notetaking was but what I didn’t mention was why I abandoned my plain text system. Nor did I write about my first foray into serious notetaking.

The notebook above, from 1994, is the oldest notebook of mine I’ve found. It’s the first proper notebook I kept. It wasn’t a journal, sketchbook, drafts, or workbook for school. Just notes.

And it petered out pretty quickly. I had forgotten all about it until I found it in a box a few months ago. For a good reason: it was a random mess that I didn’t take seriously.

Notetaking seriousness didn’t come for me until college, which makes sense. College is when I first started to test myself with more structured projects. Essays, theses, and my early blogging, all required a little bit more thought and planning than a teenager’s haphazard writing hobby.

Serious writing required serious notetaking, both in terms of scheduling and structure. For years, I had a strict notetaking system. Each day I made a new text file with the current date as a file name. Each file was filled with whatever notes I needed for whatever I was working on at the time. Every day I wrote something, I crossed it off on the calendar with a black marker. On the days when I didn’t write something, I marked in red.

The goal was to keep the run of black unbroken. It worked. Sort of. This regimented system was based on a random quote from Jerry Seinfeld, of all people, and a misunderstanding on my part of the purpose of Julia Cameron’s Morning Pages from her book The Artist’s Way.

I’d fallen into one of the oldest process trap of all: I had cargo culted methods whose principles, concepts, and motivations I did not understand. I had copied the ritual without understanding the mechanism.

Over time this turned into a slog, _no matter what, write three pages!, and by the time I got to my PhD, it made everything about a bad experience worse.

When I began my PhD in 2001, we were all still recovering from the dot-com crash. It made sense to work on a thesis that was grounded in an evolutionary, small-c conservative, view of media. Not only was web- and new media scepticism abundant at the time, it was very strong in the specific academic environment where I was working. Over the years that I worked on my PhD, that view became untenable to me. Interactive media—the web—was and is fundamentally different from its predecessors, in a way that promised to be completely revolutionary to both culture and society. But I was locked in a reactionary thesis. Starting from scratch would have almost inevitably led to me having to abandon the PhD. This meant that I had to write a thesis I utterly disagreed with, making an argument I didn’t believe in, using references that I felt were borderline nonsense in the context I was pulling them into, and making the words I wrote feel like ash.

The regimented daily pages notetaking routine made everything worse. It turned the writing process into a multi-year death march, filling the folders of my hard drive with unusable nonsense that I didn’t believe in then and don’t believe in now.

I managed to complete and defend my PhD and swore to leave academia behind for the rest of my life.

It took me years to fully recover.

To get there—to recover from my unrelenting thesis death march—I had to get more thoughtful and serious about writing and my notetaking. As is the way of my people (overthinkers, that is), this meant research. Lots and lots of research. On writing. On creativity. On notes and notetaking. On planning. Even research on skills development, the cognitive component in expertise, as well as memory formation. I dug through the entire Cambridge Handbook of Expertise and Expert Performance (over 900 pages). I read fluffy, self-help books on creativity. I read serious studies on the effects stress has on memory formation. I read old books. I read new books. Some of them dated. Some of them timeless. Some worked. Some didn’t.

I also began to understand the mechanism behind Julia Cameron’s morning pages, how it was supposed to work, and what it had in common with other approaches, even the ones that superficially look completely different.

I began to understand, at least for my own purposes, what notetaking was for.

And I think—based on the research I’ve been doing—that I’m not alone.

It’s all about boxes and maps.


The different kinds of notes

I recently interviewed about a dozen people on their notetaking habits. Before that, I had done a survey.

There’s a lot to cover as notes seem to come in an endless variety. Even from a group as small as twelve people you already got more than twelve different kinds of notetaking:

  • Household management (shopping lists, chores, etc.)
  • Task management
  • Mnemonic notes (write to help you remember)
  • Various kinds of annotations
  • Shared work notes (meeting minutes, documentation, etc.)
  • Personal work notes
  • Idea exploration
  • Journalling
  • Scrapbooking (digital and analogue)
  • Research management
  • Outlining
  • Information management (bookmarks and the like)
  • Project management
  • Client management
  • Concept database (save your ideas)
A blackboard. On it I've written 'Notes. Workspace vs DB vs Mnemonic. Idea management. Mnemonic. Revision. Shared work --- wiki. Collaboration. Journaling. Problem solving. Idea exploration. Project Management. Task management. Client management. Research management. Concept databases. Rading. Fragmented context.
The above list in blackboard form

This is not an exhaustive list. The closer you look, the more varieties of notetaking you find, even in such a small group of people.

There’s a temptation there to dig in, get into even more detail, and generally act as if you’ve hit a motherlode, a vein that’s rich and deep and rewarding. Just really get into it, interview dozens or hundreds of people, do study after study and spend years cataloguing all the varieties of notetaking

But that would be a mistake.

When you’re in a domain where increasing the resolution and detail only reveals an escalating amount of previously undiscovered complexity, odds are you are tackling what E.F. Schumacher called a ‘divergent problem’. These problems aren’t technical. They can’t be broken down into parts to solve each part step by step. A problem like this becomes more complex and hard to solve under close analysis, not simpler and easier to solve. The more detail you see, the more conflicting data points you encounter and the harder it gets to reconcile the diversity you’re witnessing with a cohesive mental model.

From A Guide for the Perplexed, page 126:

I have said that to solve a problem is to kill it. There is nothing wrong with “killing” a convergent problem, for it relates to what remains after life, consciousness, and self-awareness have already been eliminated. But can—or should—divergent problems be killed? (The words “final solution” still have a terrible ring in the ears of my generation.)

Divergent problems cannot be killed; they cannot be solved in the sense of establishing a “correct formula”; they can, however, be transcended. A pair of opposites—like freedom and order—are opposites at the level of ordinary life, but they cease to be opposites at the higher level, the really human level, where self-awareness plays its proper role.

So, when I step back, squint a bit, and stare at my blackboard and my notes, it strikes me that most of the notetaking methods the interviewees described are trying to answer at least one of the three following questions:

  1. How do I manage my creativity?
  2. How do I manage my knowledge?
  3. How do I manage my understanding?

Or, more to the point, the notetaking methods are a way for the writer to continuously ask themselves these questions and adjust their tactics as needed. None of the methods are prescriptive or rigid; they are all constantly being adapted.

The job of notes for creativity is to:

  • Generate ideas in a structured way through research and sketching.
  • Preserve those ideas.
  • Explore the ideas until they have gelled into a cohesive plan or solved a problem.

The job of notes for knowledge is to:

  • Extend your memory to help you keep track of useful information (client data, meeting notes, references).
  • Connect that information to your current tasks or projects so that you can find it when you need it.

The job of notes for understanding is to:

  • Break apart, reframe, and contextualise information and ideas so that they become a part of your own thought process.
  • Turn learning into something you can outline in your own words.

In terms of software, that means we have these three broad approaches:

  • Notes for creativity tend to favour loosely structured workspaces. Scrivener and Ulysses probably come the closest, though, in practice, I have doubts that either of them is loosely structured enough.
  • Notes for knowledge favour databases, ‘everything-buckets’ (apps that expect you to store everything in them), and hypertextual ‘link-everything’ note apps.
  • Notes for understanding tend to favour tools that have powerful writing or drawing features (which you favour will depend on your skill set and comfort).

I don’t think you can make software that perfectly serves all three of these kinds of notetaking. Knowledge bases become too rigid to serve as workspaces for creativity. The creative spaces are too loosely structured to work well as knowledge bases. You can integrate writing and drawing tools in either, but that serves notetaking for understanding only up to a point. Most knowledge bases preserve too much detail and context, which gets in the way of reframing and contextualization. And too fully featured writing or drawing tools could make the creativity tools too complex to use.

In the parlance of roleplaying games, these are the three character statistics of your notetaking app, and you only get to assign 15 points between them. You could choose to be average in all three (5, 5, 5, uninteresting). You could choose to dump all of the points in one stat and be just plain bad at the rest, which would simplify the design a bit. Or, you could think of it as a priority list: most important, second important, unimportant (dump stat!).

'The three stats of note app design.' Followed by a table where Creativity, Knowledge, and Understanding run down the side and the numbers 1-10 run along the top. At the bottom: 'You have 15 points!'

(If you aren’t familiar with RPGs, you can think of them as three sliders with a fixed sum of values between themselves. Every time you push a slider above ‘5’, another slider has to go down a corresponding amount.)

How would I divvy up the points for Colophon Cards? What would the Colophon Cards character sheet look like?

At the moment, I’m leaning towards this:

  • Creativity: 7
  • Knowledge: 2 (dump stat!)
  • Understanding: 6
The same image as above except this one has the values coloured in as bars: Creativity: 7, Knowledge: 2, Understanding: 6.

A workspace that:

  1. Favours lightweight and flexible structure over detailed organisation and connections.
  2. Prioritises speed over accuracy when creating and finding notes.
  3. Has better-than-average writing tools.

Now, I’m not going to pretend that I’m basing those priorities on the comparatively little user research I’ve been doing. They instead represent the kind of app I want to use and enjoy. Because it turns out that enjoyment is essential for long-term notetaking. The research helped me clarify what I wanted, give it focus, and place it properly in the larger context. But, ultimately, I want to make an app I enjoy.

Also, it made me drop a bunch of smaller ideas that research revealed to be less likely to work, but that’s honestly a little less interesting. Though possibly a topic for a future blog post.


Boxes and Maps

You don’t have to copy the methods of a German academic with a tendency for the abstruse to take notes

In my relentless daily writing during my PhD, I had missed the point of it. By using a regimented notetaking structure I drove my creativity into the ground, my joy and motivation cratered with it, and burnout inevitably followed.

What I had missed was that the point of these techniques wasn’t to continuously force march forward according to a predetermined master plan. Just writing or getting things done isn’t the purpose, at least not in any of the writing methods I’ve looked at.

And I’ve looked at a few—they all cover some form of notetaking or journaling as a core part of their process:

Umberto Eco’s How to Write a Thesis is a book about writing that contains a fully-formed (and very usable) notetaking system based on index cards. It’s structured so that everything hangs off the Table of Contents or outline. The thesis’s ToC becomes the map you use to structure your notes.

(I wish that I had read Eco’s book before I did my thesis. Alas.)

Anne Lamott’s Bird by Bird describes an ad hoc and improvised system of index cards. She presents it as an almost random process but then proceeds to spend an entire chapter explaining the value of a loosely structured notetaking system. Her notes are more of a box full of unsorted items that prod the work forward and guide it.

A good chunk of David Hewson’s Writing: A User Manual is about how various kinds of notes help you write your book. His method has elements of both the Box and the Map. He recommends plotting and planning in advance, but he also uses an unstructured daily journal and a ‘box’ of notes on characters, concepts, places, and ideas to guide the process.

The Story Grid by Shawn Coyne is pretty much just an excel-based, fiction-oriented version of Eco’s “How to Write a Thesis”. Instead of Eco’s table of contents, the story grid uses a list of scenes. But it’s otherwise a very similar principle: the map guides the project and structures your notes. The map evolves as you go but no matter how much it changes it remains the primary organizing principle.

Brenda Ueland, in her 1938 book If You Want to Write recommends keeping a “slovenly, headlong, impulsive, honest diary”. The way she puts it:

Write every day, or as often as you possibly can, as fast as and as carelessly as you possibly can, without reading it again, anything you happened to have thought, seen or felt the day before.

In her amazing The Creative Habit Twyla Tharp talks about The Box. A literal box. Each project of hers gets a box which is full of items that document the active research and work on that project. It’s filled with objects, notes, files, clippings, toys and just stuff. Some people store things in files. She prefers boxes.

And, last but not least, the idea I had misunderstood so badly years ago, seems so clear today when I reread Julia Cameron’s words in The Artist’s Way:

There is no wrong way to do morning pages. These daily morning meanderings are not meant to be art. Or even writing. I stress that point to reassure the nonwriters working with this book. Writing is simply one of the tools. Pages are meant to be, simply, the act of moving the hand across the page and writing down whatever comes to mind. Nothing is too petty, too silly, too stupid, or too weird to be included.

It’s about re-contextualising your experiences, reading, thoughts, ideas, and goals in words. This serves both as a mnemonic and to increase understanding as the act of reframing it all in words, in black and white, serves to help your mind integrate it. It turns external ideas and information into integrated thoughts that you understand and can act on.

Flipping to the next page, I can also immediately see the one paragraph that later led to all of my notetaking misery:

Morning pages are nonnegotiable. Never skip or skimp on morning pages. Your mood doesn’t matter.

If you ever read her book, ignore this paragraph. Your mood does matter in the long run. Don’t beat yourself up about it. There are other ways to accomplish the same thing without torturing yourself. What’s especially galling is that I clearly didn’t finish reading the book at the time as its entire purpose is to prevent people from making the mistake that I made.

Once you take in these many approaches to writing and study them as wholes, as opposed to just teasing out individual components, you begin to spot a few common threads running through them, among both the Box and Map creators.

The first one is that the difference between the Box and the Map isn’t as great as you think.

Those who favour the Box maintain an evolving internal map of where they are going with the project. The purpose of the items in the box is to both make it easy for you to maintain that internal representation and to give it enough context to adapt it to changing circumstances.

Those who favour the Map instead flip this around. The map is external and evolving and serves to make it easy for you to maintain the context internally so that you can adapt the project to changing circumstances. The goal is the same. The methods are remarkably similar. It’s just a question of which bit is serving as a mnemonic for the other bits.

Evidence boards are a compromise between the two that shows just how little the difference between the two is in actual practice. Depending on how you look at it, the evidence board is either the items of the box pinned to a board or a map that has been loosely transposed over to a wall.

Which general approach you should choose is entirely personal. As the difference is primarily in mnemonic tactics, the optimal approach depends on how your own memory and other cognitive processes work. Some people’s memory is “out of sight, out of mind.' Or, you might find it easy to remember details but have a harder time maintaining a big picture context. Some have an internalised heuristic or rule of thumb for how a project should work, so they only need reminders and bookmarks that they check against their mental rubric. Some have a hard time visualising the final product that is the goal of their project and so need a tactic that keeps them motivated and more observant of the bigger picture. People vary and so must their methods.

The tactic to use depends on how your mind works. Experiment. Try different approaches out. But, once you’ve found your process, trust that the overall process is roughly the same:

Collect

All the things you fancy. Note down ideas and thoughts. Take notes on what you read. Take pictures. Sketch when the whim strikes. Collect what interests you in life using whatever method is the most convenient for you. Anne Lamott uses index cards. Twyla Tharp uses whatever can be collected in her boxes. David Hewson uses a notetaking app (Ulysses, last I heard). Some use pens and notebooks. Whatever you use, remember that these notes are necessarily only an intermediate form. They don’t become understanding or thoughts until you integrate them. You need to be able to review them, and storing them safely is always useful, but, paradoxically, notes aren’t the be all or end all of the notetaking process.

Contextualise

The purpose of the various journalling tactics as well as some of the mapping tactics (like the evidence board) is to get you into the habit of reframing and contextualising your thoughts, your reading, and your ideas. By reframing them in words (or sketches) you integrate them which makes them available to your thinking and decision-making processes. If you don’t integrate what you collect, are just building an ever more intimidating database of opaque words and alien ideas.

An external brain is a brain that can’t drive your thoughts and actions and will just make your work harder.

A daily written journal is a common way to accomplish this integration. But people have also used daily sketches, voice memos (on tape in the olden days), or video diaries. The goal isn’t to preserve or convince but to reframe and understand in your own terms what you’ve experienced, read, and thought. Blogs work great. Twitter might be stretching it because of the character limit. Use whatever works for you.

Rule of thumb: let the contextualisation method match the bones of the project. A writing project benefits from writing tools.

Map

The final part is the map, the bird’s eye picture of where you are, what lies ahead, and where you want to end up. It’ll start off vague, whatever tactic you use. As you work it’ll become clearer, no matter whether you’re maintaining it in your mind, on paper, or in an Excel spreadsheet. The map might end up being an outline, a diagram, a building plan, a list of tasks in Trello, or a list of issues on GitHub. It will change over time, but don’t worry if it doesn’t. It’s important to keep in mind that the map isn’t the end product, just one more tool among many to help you build the end product.

What does this all have to do with a notetaking app?

The first, most obvious conclusion is that there is no one best method. The above meta-framework for notetaking is broad enough to encompass both Zettelkasten and Twyla Tharp’s boxes filled with stuff. If I’m right about notetaking methods being especially sensitive to individual variation in neurocognition, then there will never be one method or one app to rule all of notetaking.

It becomes a question then of deciding on what my own particular take on this software genre might be. And, if you’ve been following my earlier posts, you probably have some idea of the direction I’m taking. At least in broad strokes.

But the meta-framework doesn’t answer one, very important, question. One that people have been tackling since the dawn of humanity.

What happens when you need to collaborate with other people?

Because nobody, not even your lone wolf novelist, truly works alone.

So…

Next: the final entry in the Colophon Cards research blog series, where I write about how collaboration and sharing fit into all of this.



More News from this Feed See Full Web Site