📟

Building a Terminal Client for Linear in Rust

🔔
This post is not particularly linear (haha), besides the intro and conclusion. Feel free to skip around.

The Goal

Linear is an issue tracker designed for software; if you’ve used Jira/Trello/etc. you’ll get the general idea. In this post, I’ll be focusing on one specific Linear feature: Custom Views. Custom Views are essentially a way to persist a set of filters for a query on a set of issues.

For example, you could create and save a Custom View with the following filters:

  • Issue has label: “critical”
  • Issue assignee is “John Doe”
  • Issue has workflow state: “In Progress”

Then, rather than digging through a pile of issues all the time, you can just use a Custom View to focus on the issues that matter to you at any given moment.

An example Custom View:

image

So far, so good!

My one annoyance was that it felt obnoxious to keep switching between views for different projects and for different kinds of work across projects.

To give some context, at the time my typical workflow was something like:

  1. Pick an issue to work on
  2. Complete issue
  3. Go to commit/push from CLI
  4. Go to Linear and find issue identifier to reference in commit message
  5. Finish commit/push.

With some workloads involving high context-switching it seemed tedious to have to keep jumping back to Linear to properly reference an issue.

Thus, the goal was to remove step 4, by bringing all the relevant info to the CLI, closer to my git work.

Essentially, I wanted to have something like this:

image

…without keeping six Linear windows open. I was also at the time noticing a lot of other terminal user-interface (TUI) applications, such as spotify-tui and gitui.

😉
I may also have been motivated by working on something relatively low-stakes and fun (using Rust) after working on multiple products every day for ~ a year.

Showcase

Before going further, you can find the project at its current state here

Here’s what the end-result ended up looking like:

image

Getting Started

I started by grabbing some crates to handle essential recurring tasks:

Specifically:

  • tokio - a runtime for async execution.
  • tui - a library for building rich terminal widgets.
    • Backed by crossterm - a cross-platform terminal manipulation library
  • reqwest - HTTP client.
  • serde, serde_json - frameworks for ser/deserializing Rust structures generically; used in this case for JSON GraphQL query objects and responses.
  • simplelog - a simple logging utility for logging to a file (no stdout to terminal).

Code Layout

After iterating for a bit, the core components took shape. They’re all pretty simple; most of the complexity came from things like interactivity, layout, and edge-cases.

A simplified code layout overview:

  • main.rs
    • Handle init work (access token loading, async task for team timezone fetch)
    • Create an asynchronous _manager task to serve as an interface for API requests initiated by user action, services app.rs.
    • Draw the UI every tick (UI loop; also maintain tick count for loading icons)
    • On input: map key-codes to any matching command and call relevant method in command.rs (e.g. exec_confirm_cmd).
  • app.rs
    • pub struct App<'a>: Source of truth for all application state
      • pub fn dispatch_event(&mut self, event: AppEvent, tx: &tokio::sync::mpsc::Sender<IOEvent>)
        • Creates a new asynchronous task for any network-related workload (querying Linear API)
        • Interfaces with _manager task in main.rs and handles passing returned values to correct UI components.
          • 🔮
          app.rs used many instances of Arc<Mutex<T>> and Arc::clone(T) for thread-safe data handling, but thankfully the reqwest HTTP client uses an Arc internally, so connection pooling only requires one unwrapped instance of reqwest::Client.
  • linear/client.rs
    • Dispatches queries combining: API access token, variables (GraphQL cursor, JSON objects referenced in queries), and configurable variables (query page sizes).
    • Extracts relevant response fields and returns data to calling _manager task which communicates back to user-command initiated app.rs async task.

Features/Design Choices

Usage/UX

Density

One of the core goals was to experiment with increasing information density. The main view is an example of this: up to six linear Custom Views can be sandwiched together and their issues can be paginated/examined/modified via an overlay on the same route.

Controls

Linear does a great job providing usable keyboard shortcuts, but I wanted something where I never needed to consciously remember a single command. That meant:

  • Navigation, pagination and confirmation/submission are all handled by arrow keys + esc.
    • Pagination: triggered by attempting to move down from the last table row.
  • Selecting a specific Custom View is done by selecting its corresponding (visible) label number
  • Everything else: there’s an always-visible dynamic command bar to show available actions.

Command Bar:

image

Query Resolver (Deprecated)

Minimizing API Requests to Resolve a Custom View

When I began this project there wasn’t a single endpoint for resolving a Custom View, so significant effort had to be put into breaking down the filters in any Custom View and determining the querying strategy to minimize the number of requests needed to resolve the issues for the given Custom View.

In practice this meant mapping every Custom View filter to an endpoint (if one existed), determining how filters interacted with each other (UNION or INTERSECT?), and handling cases with 0-n directly queryable filters. There were two general strategies, based on if there were any directly queryable filters in a given Custom View or not.

This work was deprecated, but fragments of my thinking/process can be found in the wiki:

Dealing with Timezones

Working with the earlier iterations of the Linear API, I ran into the issue that I couldn’t replicate the functionality of Custom View filters such as: DueSoon, DueToday, Overdue reliably due to timezone differences (no direct endpoint to query). The timezones were also in a format I hadn’t seen before, e.g.: America/Chicago.

My first (deprecated) approach for handling this was:

  1. Scrape/export the timezones from Linear’s web-app HTML.
  2. Write a python script with some regex to output a JSON file.
  3. Import the JSON file and use a map (name→offset) to calculate DueDate distance from current time in relevant timezone.

Artifacts from this work can be found in scripts/.

After some API changes, I only needed to get the timezone for each team, so I switched to just using an async task on startup to fetch relevant timezone information for each team.

GraphQL

Integrating GQL Queries with Rust

There are a number of useful crates for working with GraphQL from Rust. Some rely on parsing a GraphQL schema, some generate queries from structs defined in Rust. Starting out, I decided to ignore all of these options and make my own approach for convenience. The approach taken:

  1. .gql queries live in queries/linear/
  2. A build.rs script at the root of the project is compiled/executed by cargo just before the package is compiled. This reads in all of the .gql files and throws each GQL query into a variable in query_raw.rs.
    1. So queries/linear/fetch_custom_views.graphql:
      1. query fetchCustomViews($firstNum: Int, $afterCursor: String)...
    2. Becomes (in query_raw.rs):
      1. pub const FETCH_CUSTOM_VIEWS: &str = r#"query fetchCustomViews($firstNum: Int, $afterCursor: String)..."#
    3. Now the program can parse the queries into JSON objects at run-time:
      4. image

This approach was super convenient for my workflow, which was to debug/test/iterate on queries in Insomnia, then throw them into a .gql file.

However, next time I’d probably try a different approach better integrated with Rust and able to take advantage of Rust’s type-safety as opposed to dodging it altogether.

Linear API Bugs

The Linear API underwent significant change over the course of my work on this client.

Initially the challenge was lacking the endpoints to map a Custom View to its issues (e.g. querying for “issues without labels”).

That changed after the introduction of the issues(filter: IssueFilter) querying capability, which allowed for directly using the field filterData: JSONObject! from a CustomView object to resolve a Custom View’s issues.

Unfortunately, working with and testing against nearly all combination of Custom Views filters exposed me to a lot of the bugs present in a rapidly-changing API, many of which were blocking.

Happy Ending Bug

A bug that I was able to report and was quickly fixed by the Linear team:

  • Null constraint in the following GQL snippet issues(filter: { dueDate: { null: true } }) only returning issues with defined dueDates, opposite of expected behavior

Sad Ending Bugs

❗
As of Oct. 2022 the bugs below seem to have been largely resolved. I will keep one example (a case-inconsistency bug) to showcase impact on work discussed in this post. For context, I reported these in the Linear Slack in Dec-Jan 2021-2022.

An example of a common class of bugs around using IssueFilter and querying for issues that still exist (and have been around for a while now) was case-inconsistency. An example:

  1. Fetch data about a Custom View’s filters with a query of the general form:
    2. image
  2. Get a response such as:
    3. image
  3. Throw this JSON object into a query over Issues:
    4. image
  4. Get nothing back… huh?
  5. Turns out our initial response returned lower-case workflow state names (”in progress”, “test-column-1”). However, fetchIssues() expected case-sensitive names (”In Progress”, “Test-Column-1”).
  6. Adjusting the cases and re-running Step 3 will return the correct result.

These kinds of challenges posed a major impediment to progress, resolving issues such as case-inconsistency would imply querying many resources on startup (e.g. all a team’s labels) and then attempting to reconcile differently cased identifiers with the filters given by a Custom View.

Other issues, such as broken and combinators, implied even more esoteric strategies to circumvent.

Layout & Wrapping

Working in the terminal meant losing access to a lot of layout functionality that I’d come to take for granted in browser-like contexts. I didn’t have to access to an easy method to make my components (based on tui-rs) scale with content in the ways I wanted.

Some examples of frustrations were:

  • Table column widths not scaling dynamically to fit content.
  • Table row heights not adjusting in reliable/consistent ways.
  • Resizes not being handled well.
  • Small, obnoxious things: text wrapping (cutting a word in half), truncation indicators (I wanted ellipses), etc.

A small example (silent truncation, columns not allocating space based on contents, etc):

image

So I wrote a small set of layout utilities to fix the problem, loosely inspired by CSS Flexbox, the three most relevant aspects being:

image

Note: There was some pain dealing with UTF-8 Rust strings initially, crates like unicode-segmentation came in handy.

The result:

image

Note: The row height growing across all rows, the reasonable text-wrapping (acknowledges words), the truncation indicator (..), and the lack of wasted column width.

Of course, these are all relatively incremental changes and all problems that have been solved many times before, but cumulatively they make a big impact on usability.

Testing

My primary concern with regards to testing was ensuring compatibility with the Linear API over time and preventing regressions.

So the provided tests, which run against a dummy account, are intended to test a variety of different CustomViews to provide robust coverage for most possible filter combinations (assignee, duedate, priority, label, etc). The tests are validated against an “answer key” mapping CustomView ids to the mocked id values for the issues belonging to the relevant view (the issues you would see in the view in the Linear app).

Some Testing Hiccups

There were some surprising challenges involved with getting the testing framework to function as I desired:

  1. Problem: Individual tests expect access to a full list of Linear GQL CustomView objects; mocking the entire object is undesirable (silences API incompatibility), so we only have the id field. Consequently, many unnecessary queries/roundtrips per test as each test queries the Linear API for a given CustomView.

    2. Complication: Rust likes to generally run tests in parallel and the alternative; restricting testing to one thread isn’t necessary for API testing (and we still would have to find some way to specify order). Solution: Move initialization work into the call_once() method of an std::sync::Once instance and just call a wrapper method initialize() at the start of every test.

    image
  2. Problem: Trying to run asynchronous methods for execution via tokio wasn’t playing nicely in a synchronous testing context (an individually synchronous test function).

    3. Solution: tokio_test::block_on

    Runs the provided future, blocking the current thread until the future completes.

    We can even wrap up it in a macro so the end result looks like:

    image

Moving Forward

Some Takeaways

  • I was amused by how much this resembled making a generic React app, I recreated the typical state management (redux), routing, and component-based approach I’ve used 100x before.
  • Tokio and GraphQL are very fun. Hunting through a GraphQL schema to reverse-engineer a piece of non-exposed functionality was a blast.
  • Looking back, there were many more Rust-idiomatic approaches (I didn’t touch traits, though I did roll my own errors) I could’ve taken. Still, concurrency never felt like a major headache. Fearless concurrency in action?
  • Having to replicate things the browser/CSS take care of automagically is tedious.

Future Work

Progress on this project largely stalled out after encountering multiple blocking API bugs without any idea on when to expect fixes (not blaming anyone at Linear!); I couldn’t justify investing time in fragile workarounds which I knew would be quickly deprecated.

However, there are still things I would like to implement if I ever come back to this in the future (more likely now that it looks API bugs have been handled):

  • Configurability
    • Allow for sorting Custom Views by attributes (e.g. date created)
    • Allow for customizing key-codes for commands (e.g. “Modify Title”)
    • Allow for more flexible access token handling, currently can either supply via env var or input field.
  • More issue edit functionality
    • Labels, description, etc.
  • Webhooks
    • Would be nice to have some kind of webhook integration, either for alerting/notifications or better caching.
  • Embedded DB
    • Would be nice for caching, many things change infrequently but are useful/necessary and need to be fetched on startup every time. Something like Sled, for example.
  • Github + other platforms
    • Add support for Github Issues and other platforms.
  • Workflows
    • Add better support for common workflows, possible workflow automation similar to what’s possible with Github issues.
  • Nerdfonts
    • Better icons

👋