Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ This is the Jekyll-based source for the official Ruby programming language websi
### Jekyll Site Operations

```bash
# Build the site (takes several minutes)
# Build the site and its search index (takes several minutes)
bundle exec rake build

# Serve locally at http://localhost:4000/
# Build, watch and serve at http://localhost:4000/, search index included
bundle exec rake serve

# Alternative: Jekyll direct serve with incremental builds
Expand All @@ -40,6 +40,7 @@ bundle exec rake test
# Run individual test suites
bundle exec rake test-news-plugin # News archive plugin tests
bundle exec rake test-linter # Linter library tests
bundle exec rake test-search-index # Search index library tests

# Linting
bundle exec rake lint # Markdown linter
Expand Down Expand Up @@ -110,6 +111,31 @@ The news system is powered by a custom Jekyll plugin (`_plugins/news.rb`):
- **Archive pages**: Index, yearly archives, monthly archives
- **RSS feeds**: Generated per language via `news_feed.rss` layout

### Search (Pagefind)

Search is client-side and needs no backend. `_plugins/search_index.rb` hooks
Jekyll's `post_write` and calls `lib/search_index.rb`, so every build and every
regeneration under `rake serve` ends with a matching index. It has to run after
the write because Pagefind reads the generated HTML, and it has to run every
time because Jekyll deletes destination files with no source counterpart, the
previous bundle included. Without Node installed the hook warns and skips.

- **Indexed region**: `_includes/search_body.html` emits `data-pagefind-body` on
the `<article>` of `page.html` and `news_post.html`. Everything outside it, and
anything marked `data-pagefind-ignore` inside it, stays out of results.
- **Per-language indexes**: Pagefind splits the index by the `lang` attribute on
`<html>` and the browser loads the one matching the page, so a translation only
ever finds its own pages. The same attribute picks the interface language, so
no UI strings live in `_data/locales`.
- **Unsupported languages**: `search.unsupported` in `_config.yml` lists the
languages Pagefind cannot index well (currently `bg`). Their pages get no
`data-pagefind-body`, and `lib/search_index.rb` rewrites `pagefind-entry.json`
so they resolve to `search.fallback`. Without that rewrite Pagefind would fall
back to whichever index has the most pages, which is not a language we choose.
- **UI**: `_includes/search.html` (button, modal, config) and
`_includes/search_assets.html` (bundle, loaded before `compiled.css` so the
site's `--pf-*` overrides win).

### Linter (`lib/linter.rb`)

Enforces strict content quality rules:
Expand Down Expand Up @@ -179,6 +205,7 @@ lang: en
**Node (package.json):**
- `tailwindcss` - CSS framework
- `@tailwindcss/typography` - Prose styling plugin
- `pagefind` - Static search index and UI

## Important Conventions

Expand Down
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,28 @@ npm run watch-css # watch and rebuild CSS automatically
**Note:** You need to have Node.js installed to run these commands.


## Search

Search is [Pagefind](https://pagefind.app/), which indexes the HTML Jekyll
produced rather than the Markdown behind it, so it needs no server of its own.
`_plugins/search_index.rb` runs it whenever Jekyll finishes writing the site,
which means `rake build` and `rake serve` both leave you with a search index
matching the site they just produced. Nothing extra to run.

Indexing adds a few seconds to a build and needs Node, so run `npm install`
first. Without it `bundle exec jekyll build` still finishes and only reports that
the index was skipped, which is enough for editing content. The `rake` tasks
stop earlier than that, since they build the CSS with Tailwind first.

Pagefind builds one index per language and picks one by the `lang` attribute of
the page the visitor is on, so each translation searches its own pages and reads
its own interface labels. Bulgarian is the exception: Pagefind has neither word
stemming nor interface translations for it, so `search.unsupported` in
`_config.yml` keeps its pages out of the index and its search runs against
English instead. Adding a language there is all it takes to do the same for
another one.


## Testing

Besides generating and previewing the site
Expand Down
20 changes: 17 additions & 3 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ task :"build-css" do
sh "npm run build-css"
end

desc "Run tests (test-linter, lint, build)"
task test: %i[test-news-plugin test-html-lang-plugin test-linter lint build]
desc "Run every test suite, the markdown linter and a full build"
task test: %i[test-news-plugin test-html-lang-plugin test-linter test-search-index lint build]

desc "Build the Jekyll site"
task build: :"build-css" do
Expand All @@ -32,7 +32,13 @@ desc "Serve the Jekyll site locally"
task serve: :"build-css" do
require "jekyll"

Jekyll::Commands::Serve.process({})
# Same pair `jekyll serve` runs. Serving on its own only hands out whatever
# _site already holds, which leaves a fresh clone with nothing to serve and
# never picks up an edit.
options = { "serving" => true, "watch" => true }

Jekyll::Commands::Build.process(options)
Jekyll::Commands::Serve.process(options)
end

namespace :new_post do
Expand Down Expand Up @@ -142,3 +148,11 @@ Rake::TestTask.new(:"test-html-lang-plugin") do |t|
t.test_files = FileList['test/test_plugin_html_lang.rb']
t.verbose = true
end

require "rake/testtask"
Rake::TestTask.new(:"test-search-index") do |t|
t.description = "Run tests for the search index library"
t.libs = ["test", "lib"]
t.test_files = FileList['test/test_search_index.rb']
t.verbose = true
end
10 changes: 10 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ exclude:
- tsconfig.json
- CLAUDE.md

# Pagefind builds one search index per language and the browser picks one by the
# `lang` of the page it is on. Pagefind has neither word stemming nor interface
# translations for the languages under `unsupported`, so their pages are left
# out of the index and their search runs against `fallback` instead. Both the
# layouts and lib/search_index.rb read these keys.
search:
fallback: en
unsupported:
- bg

url: https://www.ruby-lang.org

license:
Expand Down
5 changes: 5 additions & 0 deletions _data/locales/bg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,11 @@ feed:
description: Последните новини от ruby-lang.org.
lang_code: bg

search:
fallback_notice: >-
Търсенето не поддържа български, затова обхваща страниците на английски
език. Въведете заявката си на английски.

news:
other_news: Други новини
more_news: Още новини...
Expand Down
7 changes: 7 additions & 0 deletions _data/locales/en.yml
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,13 @@ feed:
description: The latest news from ruby-lang.org.
lang_code: en-US

# Shown inside the search modal on pages whose language is listed under
# `search.unsupported` in _config.yml. Other languages never see it.
search:
fallback_notice: >-
Search is not available in this language, so it looks through the English
pages. Type your search in English.

news:
other_news: Other News
more_news: More News...
Expand Down
10 changes: 9 additions & 1 deletion _includes/header.html
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,11 @@
{% endfor %}
</nav>

<!-- Right section: Language selector + Theme toggle + Mobile menu button -->
<!-- Right section: Search + Language selector + Theme toggle + Mobile menu button -->
<div class="flex items-center gap-0.5 sm:gap-2">
<!-- Search -->
{% include search.html %}

<!-- Language Selector -->
{% include language_selector.html %}

Expand All @@ -51,6 +54,11 @@
<!-- Mobile Navigation Menu -->
<div id="mobile-menu" class="hidden lg:hidden border-t border-stone-200 dark:border-stone-700">
<nav class="container mx-auto px-4 py-4">
<!-- Search, for the widths where the top bar has no room for it -->
<div class="search-trigger-menu mb-3">
<pagefind-modal-trigger></pagefind-modal-trigger>
</div>

{% for item in nav_items %}
<a href="{{ item.url }}"
class="block px-5 py-2 text-sm font-bold text-stone-900 dark:text-stone-300 hover:text-semantic-text-hovered dark:hover:text-ruby-400 hover:bg-stone-100 dark:hover:bg-stone-800 rounded-md transition-colors mb-2">
Expand Down
2 changes: 1 addition & 1 deletion _includes/pagination.html
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
{% assign header_parts = page.header | split: '</div>' %}
{% for part in header_parts %}
{% if part contains 'class="multi-page"' %}
<div class="pagination-footer">
<div class="pagination-footer" data-pagefind-ignore>
{{ part }}</div>
</div>
{% break %}
Expand Down
2 changes: 1 addition & 1 deletion _includes/recent_news.html
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
{% endfor %}

{% if recent_posts.size > 0 %}
<section class="not-prose mt-16 p-6 md:p-8 border border-stone-200 dark:border-stone-700 rounded-xl">
<section class="not-prose mt-16 p-6 md:p-8 border border-stone-200 dark:border-stone-700 rounded-xl" data-pagefind-ignore>
<h2 class="text-2xl font-bold mb-6">
<a href="/{{ page.lang }}/news/" class="text-stone-900 dark:text-stone-100 hover:text-ruby-600 dark:hover:text-ruby-400 transition-colors inline-flex items-center">
{{ news_locale.recent_news }}<span class="icon-chevron-right text-2xl" aria-hidden="true"></span>
Expand Down
49 changes: 49 additions & 0 deletions _includes/search.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{%- comment -%}
Site search. The button opens a modal; Cmd+K or Ctrl+K opens it from anywhere.

Pagefind picks both the index it searches and the language of its own interface
from the `lang` attribute on `<html>`, so every translation searches its own
pages and reads its own labels without anything to maintain here. The `lang`
override below is for the languages under `search.unsupported` in _config.yml,
whose pages are not indexed: lib/search_index.rb points their index at
`search.fallback` and this points their interface at the same language, so the
two cannot drift apart.

`pagefind-config` has to come first. Its attributes are read once, when the
first component of the instance connects. The trigger here is the one in the top
bar; below 640px it gives way to the one in the mobile menu, which shares this
modal through the default instance.

An empty `pagefind-modal` builds its own contents. Those languages that fall
back get the same parts written out so a notice can sit above the results,
because otherwise a search in their own language just returns nothing and reads
as a broken feature.
{%- endcomment -%}
{%- assign search_lang = page.lang -%}
{%- assign falls_back = false -%}
{%- if site.search.unsupported contains page.lang -%}
{%- assign search_lang = site.search.fallback -%}
{%- assign falls_back = true -%}
{%- endif -%}
<pagefind-config highlight-param="highlight" lang="{{ search_lang | to_html_lang }}"></pagefind-config>
<div class="search-trigger-bar">
<pagefind-modal-trigger compact></pagefind-modal-trigger>
</div>
{%- if falls_back -%}
{%- assign search_locale = site.data.locales[page.lang].search | default: site.data.locales.en.search -%}
<pagefind-modal reset-on-close>
<pagefind-modal-header>
<pagefind-input></pagefind-input>
</pagefind-modal-header>
<pagefind-modal-body>
<p class="search-notice">{{ search_locale.fallback_notice }}</p>
<pagefind-summary></pagefind-summary>
<pagefind-results></pagefind-results>
</pagefind-modal-body>
<pagefind-modal-footer>
<pagefind-keyboard-hints></pagefind-keyboard-hints>
</pagefind-modal-footer>
</pagefind-modal>
{%- else -%}
<pagefind-modal reset-on-close></pagefind-modal>
{%- endif -%}
23 changes: 23 additions & 0 deletions _includes/search_assets.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{%- comment -%}
Pagefind's search bundle, written to /pagefind/ by _plugins/search_index.rb once
Jekyll has finished writing the site. Every build produces it, `rake serve` and
its regenerations included, so these files are missing only when the index was
skipped for want of Node.

This has to come before compiled.css: Pagefind ships its `--pf-*` defaults on
`:root`, the same specificity stylesheets/components/search.css uses to override
them, so whichever stylesheet loads last would win.
{%- endcomment -%}
<link rel="stylesheet" type="text/css" href="/pagefind/pagefind-component-ui.css">
<script type="module" src="/pagefind/pagefind-component-ui.js"></script>
<script type="module">
// pagefind-highlight.js carries mark.js with it, so it is only worth loading
// on the pages a search result linked to. A build that skipped the index has
// no such file, and highlighting is not worth an unhandled rejection.
if (new URLSearchParams(window.location.search).has('highlight')) {
try {
await import('/pagefind/pagefind-highlight.js');
new PagefindHighlight({ highlightParam: 'highlight' });
} catch {}
}
</script>
17 changes: 17 additions & 0 deletions _includes/search_body.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{%- comment -%}
Marks the element this is placed on as the only region Pagefind indexes, which
keeps navigation, the table of contents and the footer out of search results.

Pages in a language listed under `search.unsupported` in _config.yml emit
nothing here. Once Pagefind has seen a `data-pagefind-body` anywhere on the site
it skips every page without one, so leaving the attribute off is what keeps
those pages out of the index. lib/search_index.rb then points the language at
`search.fallback` so its visitors still get results.

A page can opt out on its own with `searchable: false` in its front matter.
{%- endcomment -%}
{%- unless site.search.unsupported contains page.lang -%}
{%- if page.searchable != false -%}
data-pagefind-body data-pagefind-meta="title:{{ page.title | strip_html | escape }}"
{%- endif -%}
{%- endunless -%}
1 change: 1 addition & 0 deletions _layouts/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
<meta itemprop="description" content="{{ meta_description }}">
<meta itemprop="image" content="{{ site.url }}/images/og-image.png">

{% include search_assets.html %}
<link rel="stylesheet" type="text/css" href="/stylesheets/compiled.css">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
Expand Down
1 change: 1 addition & 0 deletions _layouts/homepage.html
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
<meta itemprop="description" content="{{ meta_description }}">
<meta itemprop="image" content="{{ site.url }}/images/og-image.png">

{% include search_assets.html %}
<link rel="stylesheet" type="text/css" href="/stylesheets/compiled.css">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
Expand Down
2 changes: 1 addition & 1 deletion _layouts/news_post.html
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
</div>

<!-- Main content -->
<article id="content-wrapper" class="prose md:px-4 lg:px-6">
<article id="content-wrapper" class="prose md:px-4 lg:px-6" {% include search_body.html %}>
{% include title.html %}

<div id="content">
Expand Down
4 changes: 2 additions & 2 deletions _layouts/page.html
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
{% if is_homepage %}
<!-- Home page: Single column layout -->
<div class="max-w-4xl mx-auto">
<article id="content-wrapper" class="prose">
<article id="content-wrapper" class="prose" {% include search_body.html %}>
{% include title.html %}

<div id="content">
Expand All @@ -61,7 +61,7 @@
</div>

<!-- Main content -->
<article id="content-wrapper" class="prose md:px-4 lg:px-6">
<article id="content-wrapper" class="prose md:px-4 lg:px-6" {% include search_body.html %}>
{% include title.html %}

<div id="content">
Expand Down
21 changes: 21 additions & 0 deletions _plugins/search_index.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# frozen_string_literal: true

require_relative "../lib/search_index"

# Pagefind indexes the HTML Jekyll wrote, so it hangs off post_write rather than
# off a Rake task. That way `rake build`, `rake serve` and a bare `jekyll build`
# all end with a search index that matches the site they just produced.
Jekyll::Hooks.register :site, :post_write do |site|
entry = SearchIndex.new(
bundle_dir: File.join(site.dest, "pagefind"),
search_config: site.config["search"],
).build

next unless entry

# The languages that fall back share another language's index, so count the
# indexes rather than the keys pointing at them.
indexes = entry["languages"].values.uniq
pages = indexes.sum { |index| index["page_count"] }
Jekyll.logger.info "Search index:", "#{indexes.size} languages, #{pages} pages"
end
1 change: 1 addition & 0 deletions admin/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
layout: page
title: "Admin Area"
lang: en
searchable: false
---

- [News Post Translation Status](translation-status)
Expand Down
8 changes: 8 additions & 0 deletions javascripts/theme-toggle.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@
root.classList.remove('dark');
}
}

// Pagefind's search components read `data-pf-theme` and nothing else, so
// derive it from the class rather than repeating the decision above.
if (root.classList.contains('dark')) {
root.setAttribute('data-pf-theme', 'dark');
} else {
root.removeAttribute('data-pf-theme');
}
}

// Update toggle button icon
Expand Down
Loading