Roam Research: a new way of working with qualitative research data
There’s a new tool in town. It has been the subject of a lot of hype in the productivity space, some of it warranted, some of it not. It’s a power tool and, admittedly, not immediately intuitive. There is some cult-like devotion to it, and, as one might expect, a related backlash.
In this article, I’m going to step through a process of using this new tool — Roam Research — to capture and analyze qualitative data from user research.
What is Roam Research?
At its most basic, Roam is a note-taking and knowledge management application, along the general lines of Evernote or DevonThink. However, neither of these applications are good models for how Roam works. These applications use the prevailing document/folder/tag paradigm to structure information. That’s not how Roam works.
In Roam, content is connected using bi-directional links: If I link from Page A to Page B, there is not only the link I created in Page A, but Roam automatically makes a linked reference from Page B to Page A.
For example, here’s a page in Roam for the book The Splendid and the Vile:
Within this page, I’ve created links to the author [[Erik Larson]], and some related topics [[Battle of Britain]], [[Blitz]], [[Winston Churchill]].
If I click on the link to the author [[Erik Larson]], Roam shows me the page for Erik Larson, which in this case, is the set of linked references Roam has created (text blocks where Erik Larson is mentioned).
This type of linking might seem simple enough, but linked references and other features in Roam have revolutionized how I take and manage notes.
To get a better sense of how Roam works, Drew Coffman has an excellent 7-minute overview video. If you’re not familiar with Roam, I suggest you watch it before continuing.
Caveats on Roam
Roam is both early-stage software and somewhat expensive ($15/month). I was an early adopter and took Nat Eliason’s Effortless Output in Roam course to learn it. I’m sure I could have learned Roam on my own — but probably not as easily or quickly.
Additionally, I am probably a poster-child target user for Roam, in that I do a lot of writing, including daily journaling, and already have an established note-taking habit for all manner of topics in both my professional and personal life. For people without these established habits, Roam may be of less overall interest and utility.
That said, I think for anyone engaged in research (any type of research) it’s worth the time to explore Roam as a possibility for managing ideas. There is a 30-day trial, and so if you’re game, I suggest signing up for the trial and work along with this article.
What this Article Will Cover
This article will not be a full-blown tutorial on Roam. I will explain just enough to support the specialized use case of using Roam for qualitative research. I’m still in the early days of using Roam for this purpose — I expect my process will evolve as I gain experience, and Roam adds new features. I want to give you enough information to try it on your own.
We’ll look at these steps:
- Setting up a database for the study (what Roam calls a Graph)
- Creating a set of four TOC pages that represent the main “data views”
- Creating a page for each participant
- Creating a page for each research question or main study task
- Capturing notes during the study
- Identifying and coding keywords in session notes
- Identifying and tagging different types of findings
- Exploring the resulting bi-direction links to identify themes and other patterns in the data
- Summarizing data in Roam
- Moving data from Roam into a writing application (such as Word or Google Docs)
I obviously can’t use real study data, so I’ve created data for a fictional study around designing a smart ring device.
Setting up the Study Database
Each Roam subscription allows for the creation of up to three databases or what Roam calls Graphs. I have two main Graphs: one for professional interests and one for personal interests. These both have content that is “everlasting” — information that I’d want to refer back to as part of a personal knowledge management system (PKM).
For now, I am using the third Graph as a test-bed for more transient use — like a bunch of study notes that wouldn’t usually have lasting value to me. Once I’m done with a study, I export the study-related pages and delete them in Roam. Creating this sort of temporary Graph isn’t an intended use case for Roam, but it’s certainly possible to use Roam in this way.
So, at least for now, my first step is starting with an empty Roam Graph.
Creating TOC pages
Creating TOC pages is another practice outside of Roam orthodoxy, since, in theory, linked references render what I am about to describe unnecessary. However, a current constraint with linked references is that they can’t be re-ordered manually or sorted based on defined criteria. They are “hard-coded” sorted by recency, which isn’t always that helpful.
So, the first step is to create four TOC pages, which represent the primary data views:
- [[TOC: Participants]]
- [[TOC: Questions]]
- [[TOC: Keywords]]
- [[TOC: Types of Findings]]
The easiest way to create these pages is by typing the above into Roam’s Daily Page. Roam revolves around the idea of Daily Pages, which isn’t as applicable to our use case here, but it’s a quick way to get the pages created. Here’s a Daily Page with the set of links for the pages:
Next, we want to add each of these pages to the left sidebar as a shortcut. Go to each page and click on the star icon in the upper right toolbar.
We’ve created shortcuts to these pages because we want to have quick access to them. Here is what the sidebar setup looks like so far:
Creating Participant Pages
Our fictional study will have eight participants, and we want to create a page in Roam for each one.
Go to the [[TOC: Participants]] page, and create a page for each participant by typing [[P01]], [[P02]], [[P03]], etc. Pro Tip: If you use a text expansion tool like TextExpander, I highly recommend setting up a snippet for the participant list because we’ll be using this elsewhere.
Your [[TOC: Participants]] page should look like this:
Now, for each participant, we’ll create a set of participant attributes that will help us explore the data downstream. For this study, our attributes are:
- Age::
- Gender::
- Marital Status::
- Number of Rings Owned::
Attributes in Roam are entered using the double colon but are displayed in bold text with a single colon. For example, the [[P01]] page looks like this:
Currently, attributes in Roam function just like the pages we created above using double-brackets. (If you click on an attribute, you’ll see that there is a corresponding page.) However, Roam has plans to build out additional functionality for attributes, so I’ve gotten into the habit of using them.
Now, go to each participant page and enter the set of attributes (copy & paste is your friend).
Thus far, we have our set of participant pages, each containing the set of attributes. Next, go back to the [[TOC: Participants]] page and update it to include both the list of participants and the list of attributes. The page in this format will be handy when we’re working with the data later:
Let’s pretend recruiting is done for this study. Now you can enter the attribute values for each participant. As a matter of styling and presentation, I enter attribute values as tags in Roam. Functionally, tags are equivalent to pages, but they allow for a different presentation and use. Here is how you would enter a set of attribute values as tags:
- Age:: #[[25–34]]
- Gender:: #[[Male]]
- Marital Status:: #[[Married]]
- Number of Rings Owned:: #[[3–5 Rings]]
Here is what the attributes look like on the page, with the attributes in bold and the tags in gray:
At this point, we can see how linked references work. If I go to the Age:: page, I see all the text blocks containing a reference to Age::, which in this case shows me the age of each participant:
If I click on an attribute value, such as #[[35–44]], I see all participants with that attribute in the linked references:
Thus far, these connections aren’t providing much value, but they do demonstrate how linked references work.
Creating Pages for Study Questions
Next, we’re going to create a set of pages for our research questions. Our example study is generative research; if you were doing evaluative research, such as a usability test, you would want to create pages for the tasks in your study (along with any associated interview questions).
Go to the [[TOC: Questions]] page and create the following pages:
- [[Ring Wearing Behavior]]
- [[Smart Watch Use]]
- [[Smart Ring Functionality]]
These pages represent the main categories of interview questions. I find it’s easier to use these category pages and enter individual questions in Roam as text blocks (we’ll see this in a moment).
The [[TOC: Questions]] page should look like this:
Next, we’re going to populate the [[Ring Wearing Behavior]] page with specific questions we’ll ask in the study.
For each question, we will create a text block for each participant using the same format as the [[TOC: Participants]] page: [[P01]], [[P02]], [[P03]], etc. Pro-Tip Reminder: setting up the participant list as a snippet in a text expansion tool such as TextExpander greatly speeds this process.
Now our [[Ring Wearing Behavior]] page is ready to capture findings for each participant:
Now we’ll imagine we’ve prepared the other pages listed on the [[TOC: Questions]] page in the same way.
Getting Findings Into Roam
I know researchers can have strong preferences for how they like to capture notes during a study. I am on Team iPad, taking handwritten notes in Notability. Pre-Roam, I later typed notes into a spreadsheet with columns representing participants and rows representing the research questions.
With Roam, I summarize notes in the Questions pages. Example notes for the [[Ring Wearing Behavior]] page are shown below. Again, this is a fictional study created for illustration purposes, so the data isn’t 100% realistic:
Coding Keywords and Key Phrases
Thus far, the process hasn’t been that much different than structuring findings with other tools, such as a spreadsheet.
But now we are going to imagine that the study is complete, all our notes are in Roam, and we’re going to take the next step, which is to code our findings. We will:
- Look for keywords in our findings
- Turn each keyword into a page by surrounding it with double-brackets
- Add each keyword to the [[TOC: Keywords]] so we can see, all-up, all the keywords we’ve identified (and, later, group similar keywords together)
In the main Roam window, open the first question page, the [[Ring Wearing Behavior]] page.
In the left sidebar, Shift + Click on the [[TOC: Keywords]] page to open that page in Roam’s right sidebar. The right sidebar is an uber-important feature of Roam; it makes it super-easy to add keywords as we identify them to the [[TOC: Keywords]] page.
Coding findings involves looking for keywords and turning those words or phrases into Roam pages. To do this, highlight a word or set of words and type “[[“ — that action transforms the text into a page.
Below you can see finished work on keywording the findings and listing the keywords in the [[TOC: Keywords]] page:
If you compare the coded findings to the uncoded findings, you’ll find that I’ve done some “language normalization” where I wanted to use the same keyword to eliminate synonyms. For example, the phrases “coordinate with my clothes” and “coordinate with my outfit” have been replaced by [[Match My Outfit]].
Organizing Keyword into Categories
It can be helpful to organize the keywords into descriptive categories. We are using a very limited example for illustration purposes, but here’s how categories would look in our sample study:
Next, for later analysis, we want to turn each of the keyword categories into a page. Again, you can select the text and type “[[“ to transform it into a page.
This is what the [[TOC: Keywords]] page looks like now:
Even this limited example shows, I think, how organizing keywords can help start identifying themes.
Tagging Types of Findings
Next, we will tag findings, characterizing them by type. Whereas keywords and key phrases tend to be emergent and study-specific, I think it’s easier to develop a starting set of findings types. Some examples are show below:
- Existing Behavior
- Pain Point
- Need
- Negative Experience
- Positive Experience
- Quote
Note that these example findings types aren’t mutually-exclusive tags. The last tag, Quote, is a way to quickly pull together on a single [[Quote]] page anything that is quote-worthy.
Before we start tagging findings, we want to Shift-Click on [[TOC: Types of Findings]] to open the page in the right sidebar. Note that when opening a new page in the right sidebar, anything that is already open persists:
A great thing about the Roam right sidebar is that you can edit pages directly in the sidebar. So I’m going to add my starting set of findings types to it by entering them in the sidebar. The [[TOC: Types of Findings]] page in the sidebar now looks like this:
Next, let’s look at how I tagged findings under the question: “Are there situations where you would never wear a ring? Why?”
I can see that some of these findings represent user painpoints, so I’ve tagged them as such with #[[Painpoint]].
The example above also gives us an opportunity to talk about Roam text blocks (represented by each bullet point). Roam uses text blocks as the “information unit” it pulls in for linked references.
Text blocks can be nested — and I’ve used that fact here to tag the parent — the question — as #[[Existing Behavior]] since all the children, in this case, pertain to that finding type. If we go to the [[Existing Behavior]] page, we can see a gray highlight around the bullet indicating there are child text blocks:
But, I can open the parent to see all the children:
A Word about Coding and Tagging Findings
Coding and tagging findings is not the excessive amount of work it might seem. I recently coded and tagged all my notes for a study with 90-minute sessions and nine participants in about five hours.
This was not busy work. The time I spent coding and tagging forced me to actively engage and think about the findings in a way I might not otherwise. Even if I had stopped here and started writing the report, Roam would have provided value.
But we’re not done yet.
Data Exploration Techniques
We’ll now assume the entire set of notes has been coded and tagged. And also, each participant’s photo has been added to the participant pages for easy reference.
We will look at some data exploration techniques in this section, including:
- Using the left sidebar
- Using the right sidebar
- Filtering linked references
Using the Left Sidebar
Use the shortcuts on the left sidebar to quickly navigate the four main data views: participants, questions, keywords, and types of findings:
Using the Right Sidebar
Use the right sidebar (Shift + Click on any link) to explore related information without losing your current context.
To view linked references for a page that’s open in the right sidebar, click on the corresponding number of references in the upper right. The linked references open in a panel above:
Filtering Linked References
It’s also possible to filter linked references, whether displayed in the main content area or in the right sidebar.
Click the filter icon at the top right of the list of linked references.
Main Data Views
At this point, the set of pages in Roam look like this:
Content for the gray-shaded pages at this point in the process is mostly generated by Roam. For example, the Participant pages look like this, with associated findings automatically generated as linked references:
Types of Findings pages consist only of linked references because we’ve not manually added any content to them yet:
Keyword pages also consist only of linked references at this point:
The Types of Findings and Keyword pages are essentially blank canvases we can now use to summarize data.
Writing Data Summaries
The text entry portion of each page is where you can capture summarizing thoughts on the data as you review it. These summaries can start to form the basis of your report:
Note that I’ve used #[[Summary]] as a header text block and put the summary under it as a child text block. This ensures that all your summaries show up as linked references on the [[Summary]] page.
The [[Summary]] page is another primary data view. Go to the page now and add it as a shortcut to the left sidebar (using the star icon in the upper right of that page). Move the page to the top of the shortcuts, as shown below:
The process at this point is a matter of going to [[TOC: Types of Findings]], [[TOC: Questions]], [[TOC: Keywords]] to summarize the findings, as seems warrented, from these perspectives.
For [[TOC: Keywords]], it is likely most valuable to summarize by the keyword categories created when organizing the keywords.
Remember you can open any links in the right sidebar using Shift + Click. This feature allows you to explore additional data while keeping your current context.
The benefit of summarizing within Roam is that it keeps the summaries in context of the supporting data. The abilty to filter linked references makes it easy to pull data that matches certain criteria, for example, seeing how many participants matched certain attributes.
I can’t emphasize enough how helpful Roam is at this point in summarizing data. Everything you need is quickly at your fingertips through bi-directional linking and other page-level features such as the right sidebar and linked reference filtering.
Creating a Head Page for the Study
Once you finished summarizing the data, the next step is creating a head page for the study. This page will serve as the “narrative outline” for your report. Use the top search bar (or Daily Page) to create a new page called [[Head Page]]. Go to that page and add it to the left sidebar as a shortcut. Your sidebar should look like this:
While viewing the [[Head Page]], Shift + Click on the [[Summary]] page to open it in the sidebar. In the sidebar, click on the number representing the linked references. You should now see in the sidebar, as linked references, all the summaries you’ve written:
Next, I suggest creating a rough outline for your report in the main content area of the [[Head Page]]. This outline gives you a starting structure into which you can insert your summaries. I also suggest formatting topics as headings using the Roam contextual menu.
Here is the [[Head Page]] with the very abreviated outline for our sample study:
You can now Option + Drag the summaries to insert them into the [[Head Page]] as block references. You could also drag them in as text, but inserting summaries using Option + Drag preserves the text block’s connection to its source.
Block references are more advanced Roam functionality — and I suggest that you learn more about them before using them. Otherwise, you can certainly Drag (rather than Option + Drag) summaries into the [[Head Page]], which copies them in as unlinked text.
The goal of the [[Head Page]] is to get a running start on the report, with all main points represented. You can, of course, open any pages, beyond the [[Summary]] page, in the right sidebar to help capture what you want to cover in the report.
In theory, you could author the entire report in Roam; however, I prefer to get my head page content out of Roam as soon as I think it’s ready to put together into a more cohesive narrative. Though there are advocates of using Roam has a writing tool, it’s not purpose-built to serve that function. I can always refer to Roam as necessary as I write the full report for the study.
Getting the Head Page Content Out of Roam
Assume now you’ve finished creating the [[Head Page]] and want to get it into a “destination” writing tool such as Microsoft Word or Google Docs.
Simple Copy & Paste
You can simply copy the content of your [[Head Page]] and paste it into the writing tool of your choice. To go this route, first change the view of the page from the bulleted view (the default) to the document view using the document contextual menu (right-click on the document name). This action removes the bullets from the page.
Once you copy text into its final destination, use global search and replace to remove any unwanted Roam markup characters (such as double-brackets).
The only disadvantage with the simple Copy & Paste approach is that you lose any header designations (H1, H2, H3) in your text. But, depending on your formatting needs, the header designations might not be of concern.
Using Roam Tools + Markdown Text Editor
Because I use Roam more broadly for writing, I prefer a different method, which preserves the header designations I’ve added within Roam. If you’re familiar with Markdown — or even customarily write using Markdown — you might also prefer the process below. It involves four steps:
- Copy the text from the [[Head Page]] into Ryan Guill’s Roam Tools
- Use Roam Tools to remove any undesired Roam markup
- Copy the Markdown view from Roam Tools and paste it into a Markdown editor (such as iA Writer or Ulysses), continuing to author the report within the editor as desired
- Export the Markdown text to its final destination (typically Google Docs or Word)
Archiving The Roam Data
As I mentioned, the whole procedure I’ve described starts with an empty Roam database. Because Roam currently caps the number of databases per account to three, at some point you will need to clear out a database so you can re-use it for another study.
It is possible to export your Roam database as either JSON or Markdown files. However, there is no data portability in Roam currently — meaning you can’t use these files in a different tool to view the data as you built it in Roam (with all the linked references). For more information, I suggest reading Data Portability & Roam Research.
That said, at a minimum, you can use the exported Markdown files in the future to reference raw session notes and the data summaries. Because I work in a consulting capacity, for me it’s enough to have Markdown files and the finished report, rather than the study database, in perpetuity.
I have been pleased and excited about using Roam for analyzing qualitative data. I think Roam both forces deeper thinking about findings and streamlines the reporting process.
Both the process I’ve described in this article (and Roam itself) is early-stage. However, if you are the adventuring type — and enjoy exploring new tools — it’s worth trying Roam. I suggest taking data from a past study, pulling it into Roam, and practicing the coding, tagging, and analysis.
In closing, I should mention that Roam represents an emergent category of apps that allow for bi-directional data structuring. The leading alternative to Roam is Obsidian (also in early-stage). However, thus far, I’ve found Roam to be more powerful and better suited to my needs.
Happy Roaming!
