Introduction

Full-text search is a common feature, and Elasticsearch is often the default answer. But for many apps that’s more infrastructure than the problem needs. PostgreSQL has solid full-text search built in, and if you’re already running Postgres you can get a long way without adding anything.

What full-text search does

Full-text search matches natural-language documents by terms, phrases or boolean queries, and can rank results by relevance. It’s more than substring matching: it handles derived words, ignores stop words like “the” and “and”, and scores results.

Search for “running” and it also finds “run” or “runner”. That works through tokenization (splitting text into meaningful units) and normalization (stemming words back to their root).

Why PostgreSQL instead of Elasticsearch

Elasticsearch is a powerful distributed search engine, but it’s another service to run. PostgreSQL’s full-text search is enough when:

  • your app already uses PostgreSQL,
  • your search stays within a single database,
  • you want to keep operational overhead low.

Elasticsearch still earns its place for horizontal scaling across nodes, advanced queries like fuzzy matching, and high-speed indexing of large, frequently updated datasets. For simpler cases, Postgres does the job.

Tokenizing and searching text

PostgreSQL provides two core functions:

  • to_tsvector: turns text into a searchable document (a tsvector) by tokenizing, removing stop words and normalizing.
  • to_tsquery: turns a query string into something that matches against a tsvector.

Tokenizing text

SELECT to_tsvector('english', 'You live in a dream world. You despise the outside and you fear you are the next one');
'despise':8 'dream':5 'fear':13 'live':2 'next':17 'one':18 'outside':10 'world':6

Words like “you”, “in” and “the” are dropped as stop words; the rest are indexed.

SELECT to_tsvector('english', 'You live in a dream world') @@ to_tsquery('english', 'dream');
 true

The @@ operator checks whether the document matches the query.

Boolean queries

SELECT to_tsvector('english', 'You live in a dream world') @@ to_tsquery('english', 'dream & world');
 true

You can combine terms with & (AND), | (OR) and ! (NOT).

Searching table data

In a Rails app you’ll search across columns. For a posts table with title and body:

SELECT * FROM posts WHERE to_tsvector('english', title || ' ' || body) @@ to_tsquery('english', 'musician');

This finds posts containing “musician”, “music” or similar in either column.

Indexing for performance

Full-text search gets slow on large tables, but a GIN index on the tsvector fixes that:

CREATE INDEX index_posts_on_title_and_body ON posts USING gin(to_tsvector('english', title || ' ' || body));

With the index in place, these queries stay fast even on large datasets.

Doing it from Rails

The SQL works directly, but the pg_search and textacular gems make it idiomatic in a model.

class Post < ApplicationRecord
  include PgSearch::Model

  pg_search_scope :search_by_title_and_body,
                  against: %i[title body],
                  using: {
                    tsearch: { prefix: true } # partial matches, e.g. "run" matches "running"
                  }
end
Post.search_by_title_and_body('musician')

textacular

class Post < ApplicationRecord
  extend Textacular
end

Post.basic_search('musician')

Ranking by relevance

ts_rank computes a relevance score you can order by:

SELECT *, ts_rank(to_tsvector('english', title || ' ' || body), to_tsquery('english', 'musician')) AS rank
FROM posts
WHERE to_tsvector('english', title || ' ' || body) @@ to_tsquery('english', 'musician')
ORDER BY rank DESC;

When to reach for Elasticsearch

Postgres full-text search has limits worth knowing:

  • Scaling: it’s tied to a single database instance and doesn’t distribute.
  • Advanced features: fuzzy matching, typo tolerance and rich analytics are Elasticsearch’s territory.
  • Very large volumes: Elasticsearch’s inverted indexes are built for that scale.

If you outgrow Postgres, you can move to Elasticsearch or run both.

Wrapping up

For a lot of apps, PostgreSQL’s built-in full-text search is enough: real tokenization and ranking, a fast GIN index, and Rails-friendly gems, with no extra service to operate. Start here, and only add Elasticsearch when you’ve actually hit a wall Postgres can’t handle.

Resources

Have comments or want to discuss this topic?

Send an email to ~bounga/bounga.org-discuss@lists.sr.ht