hackertracker-export

Go CLI for exporting HackerTracker Firestore data into raw inspection files or the normalized static JSON artifacts consumed by info.defcon.org and junctor/hackertracker-info.

The CLI has two primary workflows:

• fetch writes Firestore-shaped JSON for inspection, fixtures, audits, and debugging.
• info transforms HackerTracker data into the static artifact tree served by the web app.

Install

Run from source:

go run ./cmd/hackertracker --help

Or install a local binary:

go install ./cmd/hackertracker
hackertracker --help

Firestore-backed commands require network access and permission to read the HackerTracker Firestore project.

CLI Usage

hackertracker conferences
hackertracker fetch <target> --conference <code> [--stdout] [--out <dir>]
hackertracker info [--out <dir>] --conference <code> [--conference <code>]

The same commands can be run through go run:

go run ./cmd/hackertracker conferences
go run ./cmd/hackertracker fetch content --conference DEFCON34 --stdout
go run ./cmd/hackertracker info --conference DEFCON34

info also accepts additional conference codes as positional arguments after the flags.

Fetch Raw Data

Fetch targets are:

conference
articles
content
documents
locations
organizations
speakers
tagtypes
all

Examples:

go run ./cmd/hackertracker fetch conference --conference DEFCON34 --stdout
go run ./cmd/hackertracker fetch content --conference DEFCON34 --stdout
go run ./cmd/hackertracker fetch speakers --conference DEFCON34 --stdout
go run ./cmd/hackertracker fetch all --conference DEFCON34

By default, raw files are written to:

out/ht/<lowercase-conference>/raw/
  conference.json
  articles.json
  content.json
  documents.json
  locations.json
  organizations.json
  speakers.json
  tagtypes.json

Use --out to choose the exact raw output directory. The command writes files directly into that directory:

go run ./cmd/hackertracker fetch all --conference DEFCON34 --out ./tmp/defcon34/raw

Use --stdout to print JSON instead of writing files. For fetch all, stdout contains the conference document and all supported raw collections.

Generate Web Artifacts

Generate one conference into the default output directory:

go run ./cmd/hackertracker info --conference DEFCON34

Default output:

out/ht/<lowercase-conference>/

Generate one conference into an exact output directory:

go run ./cmd/hackertracker info --conference DEFCON34 --out ./public/defcon34/data

Generate multiple conferences into one output root:

go run ./cmd/hackertracker info --out ./out/ht --conference DCSG2026 --conference DEFCON34

When multiple conferences are exported with --out, each conference is written below that root using the lower-case conference code.

Output Structure

The info command writes:

out/ht/<lowercase-conference>/
  conference.json
  manifest.json

  derived/
    tagIdsByLabel.json

  exports/
    schedule.json
    schedule.csv

  views/
    announcementsList.json
    bookmarkSessionsById.json
    contentCards.json
    documentsList.json
    locationCards.json
    organizationsCards.json
    peopleCards.json
    scheduleDays.json
    searchData.json
    tagTypesBrowse.json

  details/
    content.json
    documents.json
    organizations.json
    people.json
    tags.json

The website export is runtime-only. It intentionally does not publish raw/**/*.json, entities/*.json, indexes/*.json, details/sessions/<id>.json, or details/locations/<id>.json. Session detail pages and location detail pages are not part of the current info.defcon.org runtime contract; content, people, tag, organization, and document details remain available under details/ as aggregate lookup files keyed by string id. For example, details/content.json is a Record<string, ContentDetailView> replacement for the old details/content/<id>.json files, and the same lookup shape applies to documents, organizations, people, and tags.

Each info run recreates the generated subdirectories so stale JSON and CSV files are removed.

Public Schedule Exports

The exports/ directory contains exactly two stable public schedule exports:

• schedule.json
• schedule.csv

Both files contain one record or row per scheduled session occurrence. Content with multiple sessions appears once for each occurrence, and content without a scheduled session is not included. Organizations, tags, locations, and people are resolved during generation, so consumers do not need to download or join against files under details/. Schedule exports are scoped to the conference page or conference directory they are downloaded from, so they do not repeat the conference name in each session record or CSV row.

exports/schedule.json uses schemaVersion: 2 and is the rich, self-contained format for LLMs, agents, applications, search and retrieval, integrations, and archival use. The top-level object contains compact metadata followed by the sessions array. Metadata includes conference code, conference name, conference timezone, session count, timestamp timezone and format, and the maximum text snippet length. Session fields include sessionId, contentId, title, descriptionSnippet, RFC 3339 start and end timestamps in UTC, location name, nested speakers, nested organizations, and tags. Long text fields are whitespace-normalized and capped at 1200 characters, including the trailing ... marker when truncated.

exports/schedule.csv is the flat dataframe-oriented format for Polars, Pandas, R, the Tidyverse, DuckDB, spreadsheet applications, and similar tools. Column names are unique lower-case snake_case, one scheduled session occurrence appears per row, UTC timestamp columns are explicitly suffixed with _utc, and multi-value CSV fields use ;. The CSV intentionally avoids sparse speaker affiliation/title columns, full free-text biographies, descriptions, URLs, presentation asset fields, and related-content pointers; JSON remains the richer representation for LLMs and nested speaker, organization, and tag data.

CSV columns:

Column	Description
session_id	Scheduled occurrence ID.
content_id	Parent content ID.
start_utc	UTC RFC 3339 start timestamp.
end_utc	UTC RFC 3339 end timestamp.
location_name	Resolved location name.
title	Resolved session title.
speaker_names	;-delimited speaker names.
organization_names	;-delimited associated event or community organization names.
tag_names	;-delimited tag names.
description_snippet	Whitespace-normalized public description preview, capped at 1200 characters including ... when truncated.

views/scheduleDays.json remains an application view model for the website UI. It is not the stable public export contract.

Raw Data vs Generated Artifacts

Raw fetch output follows HackerTracker Firestore collection names. Generated web artifacts use the domain names expected by info.defcon.org.

Raw source	Generated artifacts
content	schedule views, content cards, content details
speakers	people, people cards, people details
tagtypes and embedded tag data	tag browse views, tag details
documents	document lists, document details
locations	location cards
organizations	organization cards, organization details
articles	announcement views

Use content for top-level HackerTracker content records, sessions for scheduled instances embedded in content records, speakers for the raw Firestore collection, and people for generated artifacts derived from speakers.

Development and Validation

Run local checks:

gofmt -w .
go test ./...
go run ./cmd/hackertracker --help
go run ./cmd/hackertracker fetch --help
go run ./cmd/hackertracker info --help

Run Firestore-backed checks when network access and Firestore permissions are available:

go run ./cmd/hackertracker conferences
go run ./cmd/hackertracker fetch all --conference DEFCON34
go run ./cmd/hackertracker info --conference DEFCON34

If the default Go build cache is not writable in a sandboxed environment, set a writable cache path:

GOCACHE=/tmp/hackertracker-go-build go test ./...