Skip to content
Closed

Develop #3732

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions .github/workflows/open-tgate-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Open-TGate CI

on:
pull_request:
branches: [develop, staging, main, master]
push:
branches: [develop, staging, main, master]
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
repository-safety:
name: Repository safety
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Reject common committed secret files
shell: bash
run: |
set -euo pipefail
bad=$(git ls-files | grep -E '(^|/)(\.env|\.env\.(local|production|staging)|credentials\.json|service-account\.json)$' || true)
if [ -n "$bad" ]; then
echo "Forbidden secret-bearing files are tracked:"
echo "$bad"
exit 1
fi
- name: Verify application secret contract contains names only
shell: bash
run: |
test -f open-tgate/.env.example || exit 0
if grep -E '=(ghp_|sb_secret_|sbp_|cfk_|[0-9]+:AA[A-Za-z0-9_-]+)' open-tgate/.env.example; then
echo 'Credential-like value found in .env.example'
exit 1
fi

tdlib-configure:
name: TDLib configure smoke test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install build dependencies
run: sudo apt-get update && sudo apt-get install -y cmake g++ make zlib1g-dev libssl-dev gperf php-cli
- name: Configure
run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release

application-contract:
name: Application contract
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Validate Open-TGate structure
shell: bash
run: |
test -f docs/OPEN_TGATE_ARCHITECTURE.md
test -f open-tgate/README.md
test -f open-tgate/.env.example
test -f open-tgate/Dockerfile.api
test -f open-tgate/Dockerfile.worker
test -f open-tgate/dashboard/wrangler.toml

application-tests:
name: Application tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: open-tgate
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: '3.13'
cache: pip
cache-dependency-path: open-tgate/requirements-dev.txt
- run: pip install -r requirements-dev.txt
- run: ruff check app tests
- run: PYTHONPATH=. pytest -q
- name: Build API image
working-directory: ${{ github.workspace }}
run: docker build -f open-tgate/Dockerfile.api -t open-tgate-api:test .
- name: Validate Cloudflare Worker
working-directory: open-tgate/dashboard
run: npm install && npm run check
29 changes: 29 additions & 0 deletions .github/workflows/promote.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Promotion Guard

on:
pull_request:
branches: [staging, main]

permissions:
contents: read

jobs:
validate-promotion:
runs-on: ubuntu-latest
steps:
- name: Validate source and destination
env:
BASE: ${{ github.base_ref }}
HEAD: ${{ github.head_ref }}
shell: bash
run: |
set -euo pipefail
if [ "$BASE" = "staging" ] && [ "$HEAD" != "develop" ] && [[ "$HEAD" != release/* ]]; then
echo 'staging accepts develop or release/* promotion PRs only'
exit 1
fi
if [ "$BASE" = "main" ] && [ "$HEAD" != "staging" ] && [[ "$HEAD" != hotfix/* ]]; then
echo 'main accepts staging or hotfix/* promotion PRs only'
exit 1
fi
echo "Approved promotion path: $HEAD -> $BASE"
45 changes: 45 additions & 0 deletions .github/workflows/secret-contract.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Deployment Secret Contract

on:
workflow_dispatch:
push:
branches: [staging, main]
paths:
- '.github/workflows/secret-contract.yml'
- 'docs/BRANCHING_AND_SECRETS.md'

permissions:
contents: read

jobs:
validate-contract:
runs-on: ubuntu-latest
environment: ${{ github.ref_name == 'main' && 'production' || 'staging' }}
env:
SUPABASE_PROJECT_REF: ${{ secrets.SUPABASE_PROJECT_REF }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
TELEGRAM_API_ID: ${{ secrets.TELEGRAM_API_ID }}
TELEGRAM_API_HASH: ${{ secrets.TELEGRAM_API_HASH }}
steps:
- name: Validate required deployment secrets exist
shell: bash
run: |
set -euo pipefail
missing=0
for name in SUPABASE_PROJECT_REF SUPABASE_URL SUPABASE_SECRET_KEY TELEGRAM_API_ID TELEGRAM_API_HASH; do
if [ -z "${!name:-}" ]; then
echo "Missing required secret: $name"
missing=1
fi
done
exit "$missing"
- name: Confirm safe defaults
env:
EXTERNAL_SEND_ENABLED: ${{ vars.EXTERNAL_SEND_ENABLED }}
shell: bash
run: |
if [ "${EXTERNAL_SEND_ENABLED:-false}" != "false" ]; then
echo 'External Telegram sending must remain disabled until approval controls are validated.'
exit 1
fi
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,7 @@ docs/
/tdlib/
vcpkg/
.clang-tidy
open-tgate/.env
open-tgate/**/__pycache__/
open-tgate/.pytest_cache/
open-tgate/.venv/
141 changes: 141 additions & 0 deletions docs/BRANCHING_AND_SECRETS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Open-TGate Branching, Environments and Secrets

## Canonical branch model

- `main` — production only. Receives reviewed promotion PRs from `staging` or emergency `hotfix/*` branches.
- `staging` — pre-production integration and release verification.
- `develop` — active integration branch for completed feature work.
- `feature/*` — product features; branch from `develop`, PR back to `develop`.
- `fix/*` — non-emergency fixes; branch from `develop`, PR back to `develop`.
- `security/*` — security changes; branch from `develop`; expedited review allowed, never bypass validation.
- `infra/*` — infrastructure/IaC changes.
- `dependency/*` — dependency upgrades.
- `upstream-sync/*` — TDLib upstream imports. Never merge directly to production.
- `release/*` — optional stabilization branches cut from `staging`.
- `hotfix/*` — production emergency fixes cut from `main`, then back-merged to `staging` and `develop`.

`master` is the legacy/upstream TDLib baseline and must not be treated as the Open-TGate production deployment branch after migration to `main`.

## Promotion path

```text
feature/* | fix/* | security/* | infra/*
|
v
develop
|
PR + CI gate
v
staging
|
integration / smoke / security
|
PR + approval
v
main
|
immutable production release
```

## GitHub Environments

Create three GitHub Environments:

1. `development`
2. `staging`
3. `production`

Production should require manual approval and should only accept deployments from `main`.

## Repository/Environment secret contract

Store values in GitHub Actions Secrets or environment secrets. Never commit values.

### Telegram / TDLib
- `TELEGRAM_API_ID`
- `TELEGRAM_API_HASH`
- `TELEGRAM_BOT_TOKEN`

Telegram user-session database material must NOT be stored in GitHub Secrets. It belongs on the encrypted persistent runtime volume.

### Supabase
- `SUPABASE_PROJECT_REF`
- `SUPABASE_URL`
- `SUPABASE_PUBLISHABLE_KEY`
- `SUPABASE_SECRET_KEY`
- `SUPABASE_ACCESS_TOKEN`
- `DATABASE_URL`
- `DATABASE_DIRECT_URL`

### Cloudflare
Prefer a scoped API token over the Global API Key.
- `CLOUDFLARE_API_TOKEN`
- `CLOUDFLARE_ACCOUNT_ID`
- `CLOUDFLARE_ZONE_ID`
- `CLOUDFLARE_R2_ACCESS_KEY_ID`
- `CLOUDFLARE_R2_SECRET_ACCESS_KEY`
- `CLOUDFLARE_R2_ENDPOINT`

### Railway
- `RAILWAY_TOKEN`
- `RAILWAY_PROJECT_ID`
- `RAILWAY_ENVIRONMENT_ID`
- `RAILWAY_SERVICE_ID`

### FastAPI deployment
- `FASTAPI_CLOUD_TOKEN` (only when the selected FastAPI deployment provider supports token-based CI deployment)

### Notion
- `NOTION_TOKEN`
- `NOTION_PRIVATE_OPERATIONS_PAGE_ID`

### Pipedream
- `PIPEDREAM_CLIENT_ID`
- `PIPEDREAM_CLIENT_SECRET`
- `PIPEDREAM_PROJECT_ID`

### Container registry (optional)
- `DOCKERHUB_USERNAME`
- `DOCKERHUB_TOKEN`

## Public/non-secret configuration

Use GitHub Actions Variables for non-sensitive values:
- `APP_NAME=open-tgate`
- `SUPABASE_REGION=ap-northeast-1`
- `EXTERNAL_SEND_ENABLED=false`
- `TDLIB_DATABASE_DIRECTORY=/data/tdlib/db`
- `TDLIB_FILES_DIRECTORY=/data/tdlib/files`

## Required checks

Before `develop` merge:
- source formatting/lint
- TDLib build smoke test
- application unit tests
- secret scan
- dependency review

Before `staging` promotion:
- all develop checks
- container build
- migration validation
- API health/readiness tests
- TDLib worker startup test
- idempotency/restart test

Before `main` promotion:
- staging deployment healthy
- database migration reviewed
- security checks pass
- backup/recovery checkpoint recorded
- manual production approval

## Production rules

- Never use plaintext credentials in workflow YAML.
- Never print secrets or complete connection strings.
- Never expose Supabase secret/service credentials to browser code.
- Never use the GitHub PAT supplied by an operator as an application runtime credential.
- Prefer GitHub OIDC or narrowly scoped provider tokens where supported.
- `EXTERNAL_SEND_ENABLED` remains `false` until outbound Telegram approval controls pass acceptance tests.
Loading