Skip to content
BranchesTags

Repository files navigation

BeforeShip

Your AI coding agent says the app is done. BeforeShip checks if it is safe to deploy.

BeforeShip is an open-source launch gate for solo founders and vibe coders building apps with Cursor, Claude Code, Codex, Replit, Lovable, Bolt, v0, and other AI coding tools.

It scans AI-built SaaS apps for the production basics coding agents often skip:

  • auth and authorization gaps
  • exposed secrets and unsafe env usage
  • Supabase RLS/database isolation issues
  • Stripe/Razorpay webhook safety
  • rate limits, CORS, validation, and security headers
  • missing smoke tests and deployment readiness
  • risky AI-agent diffs like deleted tests, huge rewrites, or auth/payment changes without verification

The goal is simple:

Run one command before deploy. Know if your vibe-coded app is safe to ship.

Quickstart

Run BeforeShip directly from npm:

npx beforeship@latest scan

Scan another local app:

npx beforeship@latest scan /path/to/your-app

Scan a public GitHub repo without cloning it:

npx beforeship@latest scan-github owner/repo

Full GitHub URLs work too:

npx beforeship@latest scan-github https://github.com/owner/repo

Scan a branch or tag:

npx beforeship@latest scan-github owner/repo --ref main

Create GitHub issues for the top findings:

GITHUB_TOKEN=ghp_your_token npx beforeship@latest scan-github owner/repo --create-issues

Limit issue creation:

GITHUB_TOKEN=ghp_your_token npx beforeship@latest scan-github owner/repo --create-issues --issue-limit 3

GITHUB_TOKEN is also recommended for large public repos to avoid unauthenticated API rate limits, and required for private repos.

Write a maintainer-friendly Markdown report from a GitHub scan:

npx beforeship@latest scan-github owner/repo --create-issues --dry-run --report-md beforeship-report.md

Run BeforeShip as an MCP server for AI coding tools:

npx beforeship@latest mcp

See docs/mcp.md for setup details.

Generate the highest-priority fix prompt:

npx beforeship@latest fix-prompts /path/to/your-app

Run BeforeShip in GitHub Actions:

name: BeforeShip

on:
  pull_request:

jobs:
  launch-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npx beforeship@latest scan . --no-color --fail-on high --min-score 90

Generate HTML, Markdown, JSON, and fix-prompt reports:

npx beforeship@latest scan /path/to/your-app

Reports are written into the scanned project:

.beforeship/
  launch-report.html
  launch-report.md
  findings.json
  fix-prompts.md

Status

BeforeShip is currently a local-first CLI beta.

npx beforeship@latest scan

Current checks:

  • basic stack detection for Next.js, Supabase, Neon/Postgres, Stripe/Razorpay, Vercel, and env files
  • committed env-file warning
  • secret-like token detection for OpenAI, Supabase JWT-style keys, Stripe, and Razorpay
  • Next.js API route auth-guard and trusted-user-id heuristics
  • Stripe/Razorpay webhook signature and idempotency heuristics
  • Supabase service-role, RLS, owner-policy, and public-bucket heuristics
  • API safety checks for wildcard CORS, auth-route rate limits, and input validation
  • Next.js deploy-readiness checks for tests, health routes, and .env.example
  • score/verdict engine
  • markdown and JSON reports under .beforeship/
  • bounded verification loop with named terminal states and loop memory
  • GitHub repository scans without cloning
  • optional GitHub issue creation for remote scan findings
  • agent-ready fix prompts
  • confidence and "fixed when" guidance for evidence-aware findings

Current limitations:

  • deterministic only; no AI review yet
  • focused mainly on Next.js/SaaS launch risks
  • findings are conservative and may need review
  • GitHub scans use API file fetching and enforce file count/size limits
  • GitHub Action support is currently a copy-paste workflow; no packaged action yet
  • no MCP server yet

Demo fixture:

npx beforeship scan examples/flawed-next-supabase-app --no-report

The fixture should return DO NOT SHIP and show the launch blockers BeforeShip is designed to catch.

Repository-level scans respect .beforeshipignore, which excludes this intentionally flawed fixture.

The intended MVP is:

CLI first + MCP server + optional GitHub Action

Initial target stack:

Next.js + Supabase + Stripe/Razorpay + Vercel

Why BeforeShip?

AI coding tools make it easy to create working apps, but hard to verify production readiness.

A vibe-coded app can look 90% done while missing the dangerous invisible pieces:

  • payment webhook signature verification
  • database row-level security
  • server-side auth checks
  • safe API key handling
  • rate limiting
  • rollback and smoke tests
  • deployment env separation

BeforeShip is the missing pre-deploy gate between:

"It works locally"

and:

"It is safe to launch"

Product thesis

Coding agents optimize for making code run. BeforeShip checks whether the generated app is safe to ship.

BeforeShip is not meant to replace Semgrep, Snyk, CodeQL, Gitleaks, or human review.

It is meant to orchestrate and explain the checks solo founders actually need before deploying an AI-built SaaS app.

CLI

npx beforeship init
npx beforeship scan
npx beforeship scan --json
npx beforeship scan --output .beforeship
npx beforeship scan --fail-on high --min-score 90
npx beforeship fix-prompts
npx beforeship loop --max-iterations 3
npx beforeship --version

Use From This Repo

npm install
npm test
npx beforeship --version
npx beforeship init .
npx beforeship scan . --no-report
npx beforeship scan . --json --no-report
npx beforeship scan . --output .beforeship
npx beforeship fix-prompts examples/flawed-next-supabase-app
npx beforeship loop . --max-iterations 3

Commands

Command Purpose
init [path] --name <name> Create beforeship.config.yml and add .beforeship/ to .gitignore
init [path] --no-gitignore Create config without editing .gitignore
ignore [path] Add .beforeship/ to .gitignore
scan [path] Scan a local app and write reports
scan [path] --no-report Print results without writing report files
scan [path] --json --no-report Print machine-readable JSON
scan [path] --output <dir> Write reports to a custom directory
scan [path] --fail-on high Fail CI when a high-or-higher finding is present
scan [path] --min-score 90 Fail CI when the score is below the threshold
fix-prompts [path] Print the highest-priority focused prompt
fix-prompts [path] --all Print every focused prompt
loop [path] --max-iterations 3 Run a bounded verification loop
--version Print CLI version

Use --no-color or set NO_COLOR=1 to disable ANSI styling.

Exit codes:

0  SHIP / command succeeded
1  tool error
2  launch gate blocked

Create a config:

npx beforeship init /path/to/app --name my-app

Example config:

project:
  name: my-app
  stack:
    - nextjs
    - supabase

scan:
  exclude:
    - "node_modules/**"
    - ".next/**"
    - "dist/**"

verdict:
  fail_on_critical: true
  minimum_score: 70
  caution_score: 90

output:
  directory: ".beforeship"
  formats:
    - markdown
    - json
    - html

Reports written by scan:

.beforeship/
  launch-report.md
  launch-report.html
  findings.json
  fix-prompts.md
  loop-memory.json

Example output:

BeforeShip launch gate
Score:   42/100
Verdict: DO NOT SHIP
Stack:   nextjs, supabase, stripe

Top findings:
- [CRITICAL] Stripe webhook does not verify its signature
  app/api/stripe/webhook/route.ts
- [CRITICAL] Supabase service role appears in client-reachable code
  lib/supabase-client.ts

Counts: 2 critical, 3 high, 4 medium, 1 low

Next steps:
1. Run `beforeship fix-prompts` to generate focused agent prompts.
2. Fix critical findings one at a time.
3. Rerun `beforeship scan` before deploy.

Planned MCP tools

BeforeShip should expose a small number of high-leverage MCP tools, not a huge overloaded tool list.

before_ship_scan
before_ship_explain
before_ship_fix_plan
before_ship_gate_deploy

Optional:

before_ship_agent_policy

Planned checks

Secrets

  • .env committed
  • API keys in frontend bundle
  • NEXT_PUBLIC_ misuse
  • Supabase service role exposed
  • OpenAI/Anthropic/GitHub keys exposed
  • Stripe/Razorpay secrets exposed
  • Cursor/agent context includes secret files

Auth

  • API routes without session/auth guard
  • admin pages without server-side protection
  • user ID trusted from body/query
  • IDOR patterns
  • auth middleware exists but is unused
  • role checks performed client-side only

Supabase / database

  • RLS disabled
  • tables with user data and no owner policy
  • service role used outside server
  • public anon key misuse
  • migration drift
  • unsafe storage buckets

Payments

  • Stripe/Razorpay webhook signature missing
  • missing idempotency
  • subscription status trusted from frontend
  • price/plan selected client-side
  • billing country/currency not server-validated
  • entitlement table missing
  • payment success page grants access without verified webhook

API safety

  • wildcard CORS
  • no rate limiting
  • no input validation
  • no CSRF for cookie auth flows
  • verbose errors
  • insecure file uploads
  • missing security headers
  • SSRF patterns

Deploy readiness

  • no health endpoint
  • no smoke tests
  • no rollback note
  • missing env vars
  • Vercel serverless timeout risks
  • critical routes lack logging
  • no staging/prod separation

AI-agent-specific checks

  • huge AI-generated files
  • repeated full-file rewrites
  • deleted tests
  • new dependency added without reason
  • changed auth/payment/db files without tests
  • destructive scripts/migrations
  • no AGENTS.md / CLAUDE.md
  • agent touched files outside declared scope
  • diff contains temporary auth bypasses
  • tests modified to fit broken code

Planned report files

.beforeship/
  config.yml
  launch-report.md
  findings.json
  fix-prompts.md
  evidence/

Example finding:

{
  "id": "payment.webhook.signature_missing",
  "severity": "critical",
  "file": "web/src/app/api/razorpay/webhook/route.ts",
  "title": "Razorpay webhook does not verify signature",
  "why_it_matters": "Anyone can fake a payment success event and unlock access.",
  "fix_prompt": "Add Razorpay webhook signature verification using RAZORPAY_WEBHOOK_SECRET. Reject unsigned or invalid payloads. Add a regression test."
}

How OpenAI fits

BeforeShip should use deterministic checks first and OpenAI reasoning second.

OpenAI can help with:

  • stack-aware risk triage
  • founder-friendly explanations
  • fix prompt generation
  • test plan generation
  • false-positive reduction
  • deploy-gate reasoning for AI agents

It should not rely only on LLM judgement for security.

Repository structure

Current repository is documentation-first.

Planned implementation shape:

beforeship/
  packages/
    cli/
    core/
    mcp/
    github-action/
  docs/
    research.md
    product-spec.md
    roadmap.md
  examples/
    flawed-next-supabase-app/

Contributing

BeforeShip is intended to be open source.

Good first contribution areas:

  • add stack-specific checks
  • add examples of common vibe-coded app failures
  • improve scoring rules
  • add tests for detectors
  • add MCP client setup docs
  • add GitHub Action workflow
  • improve report UX

See CONTRIBUTING.md.

Docs

License

MIT — see LICENSE.

About

Open-source launch gate for vibe-coded apps: CLI + MCP pre-deploy checks for auth, payments, secrets, Supabase, and production readiness.

buzz39.github.io/beforeship/

Topics

ai-codingappsecclimcpnextjsopen-sourcesecuritysolo-founderssupabasevibe-coding

Resources

Readme
MIT license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity

Stars

5 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages