Advanced Search

Tarinoi is built to accommodate big projects, big teams, and serious production lifecycles. If you're dealing with a million-word production, being able to find the exact thing you're looking may not be so easy. We built a search system that lets you do this. This was a pretty cool engineering challenge since Tarinoi also gives you a lot of flexibility about how you design your data, which means we can only make so many assumptions about how it's structured.

Tarinoi actually has two entirely distinct search features.

The first one searches the data cached on your workstation. This doesn't require a connection to the server so it will also work offline. It supports basic full-text search and all of the targeted searches Tarinoi needs for its consistency checks. It's also really simple to use – you type into a search box, and results appear. If you're technically minded, you can even use it to search for IDs, variable references, or function calls. In practice it should cover most everyday work.

However, that's not always enough. There are times you want to narrow down your search to something very specific: all the SkillCheck cards for the Machinery skill, any card where the Threshold is "heroic," or any cards that reference both the SetFlag function and the "met_ferryman" variable. IndexedDB – the data store where the cached data lives – doesn't lend itself well to these kinds of searches, and its performance falls off rapidly with project size.

Therefore, we built a second, server-driven advanced search that lets you do just this kind of thing.

The server-driven advanced search lets you zero in on what you're looking for precisely, regardless of the scale and complexity of your project.

The first challenge with this feature was that Tarinoi can't assume that there is a card type called SkillCheck or a property called "threshold." These are all defined in templates which are fully under your control. Matching properties inside every card in a very large project would be technically possible, but it would get progressively slower as your project grows, and nobody wants to wait tens of seconds for a search result to come back. This could be mitigated by database indexes, but since the templates can be different in every project, we'd need to maintain project-specific indexes and change them every time you modify a template. That would also be slow and cumbersome.

Therefore, instead of attempting this, we use materialised search projections. This means that whenever you send a card to the server, we'll create or update another data object from it, extracting everything that needs to be searchable into a standard format that's independent of your template. So all the stuff that needs to be full-text-searchable goes into one property, everything that needs to be exact-matchable is packaged in a way that makes it easy to search, and so on and so forth. We put this in a different table, and index that table so it supports the search scenarios we want to support. The actual data remains templated with a changeable, fluid, dynamic structure, while the search projection structure remains stable, and the table that contains them has efficient indexes that match that structure.

This way, we can support complex advanced search queries that remain extremely fast: you can query for "all the SkillCheck cards for the 'machinery' skill where threshold = 'challenging' on all the boards in Chapter 2, where the word 'ferryman' appears," and see this query get executed in a matter of milliseconds on a million-card project. This search will just return the IDs of the matching cards, which we'll then hydrate from the data stored on your workstation – meaning: we'll look up the matching cards and display them to you as search results.

There's a lot more we plan to do with the search projections later this year, when we start working on our Tier 2 features involving production support – for example, to count things for production dashboards.