SDK Publishing Guide

Publishing, versioning, and consuming the Microlearning Platform SDKs

Overview

The platform produces two SDK packages generated from the OpenAPI specification:

Package	Language	Registry	Scope
@musingly-ai/core	TypeScript/Node.js	GitHub Packages (npm)	Full API client
musingly-core	Python	GitHub Releases (wheel)	Content import/validation

Both SDKs follow the same version number and are published automatically as part of the CI/CD pipeline. Local development uses a Verdaccio local registry to enable cross-service dependency without requiring GitHub authentication.

---

Version Strategy

Semantic Versioning

All packages use SemVer with pre-release channels:

MAJOR.MINOR.PATCH[-PRERELEASE]

Examples:
  1.0.0           ← stable production release
  1.0.1-rc.3      ← 3rd release candidate for staging
  1.0.1-dev.5     ← 5th local development build

Release Channels

Channel	Branch	Format	NPM Tag	Trigger
production	main	X.Y.Z	latest	Push to main → deploy → publish
staging	develop	X.Y.Z-rc.N	rc	Push to develop → deploy → publish
dev	feature branches	X.Y.Z-dev.N	dev	Manual / local registry

Auto-Increment Rules

Scenario	Current Version	Next Version
Staging deploy (new RC)	1.0.0	1.0.1-rc.1
Staging deploy (bump RC)	1.0.1-rc.1	1.0.1-rc.2
Production deploy (promote RC)	1.0.1-rc.3	1.0.1
Production deploy (no RC exists)	1.0.0	1.0.1
Manual minor bump	1.0.1	1.1.0
Manual major bump	1.1.0	2.0.0

Major and minor bumps are always manual. Patch and pre-release numbers auto-increment.

Version Sync

When a version bump runs with --sync, these files are updated together:

• sdk/package.json — NPM SDK version
• python-sdk/pyproject.toml — Python SDK version
• shared/package.json — Shared utilities version

---

CI/CD Publishing Pipeline

Automatic Publishing

SDK publishing is integrated into the deployment workflows:

┌──────────────────────────────────────────────────────────────────┐
│  Push to develop                                                  │
│  └── deploy-staging.yml                                          │
│       ├── CI Validation (ci-validate.yml)                        │
│       ├── Deploy Edge Functions to staging                       │
│       └── publish-sdk job                                        │
│            ├── Bump version → X.Y.Z-rc.N                         │
│            ├── Check if version already published                │
│            ├── Build & publish NPM SDK (tag: rc)                 │
│            └── Build Python wheel (upload as artifact)           │
└──────────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────────┐
│  Push to main                                                     │
│  └── deploy-production.yml                                       │
│       ├── CI Validation (ci-validate.yml)                        │
│       ├── Deploy Edge Functions to production                    │
│       └── publish-sdk job                                        │
│            ├── Bump version → X.Y.Z (promote RC to stable)      │
│            ├── Check if version already published                │
│            ├── Build & publish NPM SDK (tag: latest)             │
│            ├── Build Python wheel                                │
│            └── Create GitHub Release + attach wheel              │
└──────────────────────────────────────────────────────────────────┘

Manual Publishing

Use the publish-sdk.yml workflow for manual or corrective publishes:

Via GitHub UI: Actions → Publish SDK → Run workflow

Inputs: - version: Explicit version (leave empty for auto-increment) - channel: production, staging, or dev - bump: auto, patch, minor, or major - target: all, npm, or python - dry_run: Validate without publishing

Via CLI (gh):

# Auto-increment staging RC
gh workflow run publish-sdk.yml -f channel=staging -f target=all

# Explicit version
gh workflow run publish-sdk.yml -f version=2.0.0 -f channel=production -f target=all

# Dry run
gh workflow run publish-sdk.yml -f channel=staging -f dry_run=true

Concurrency Protection

All publish jobs share the concurrency group sdk-publish with cancel-in-progress: false. This means: - Staging and production cannot publish simultaneously - If both are queued, the second waits for the first to complete - Duplicate version checks prevent re-publishing an existing version

---

Consuming the NPM SDK from GitHub Packages

Authentication Setup

GitHub Packages requires authentication even for installing packages. Each consuming project needs:

1. Create a Personal Access Token (PAT)

Go to GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic): - Scope: read:packages - For publishing: also write:packages

2. Configure .npmrc in the consuming project

Create .npmrc in the project root:

# Route @musingly-ai packages to GitHub Packages
@musingly-ai:registry=https://npm.pkg.github.com

# Authentication (use PAT or GITHUB_TOKEN)
//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}

Important: Use an environment variable for the token. Never commit the actual token.

3. Set the environment variable

# In your shell profile (~/.bashrc, ~/.zshrc)
export GITHUB_TOKEN=ghp_your_personal_access_token

# Or in a project .env file (add .env to .gitignore)
GITHUB_TOKEN=ghp_your_personal_access_token

4. Install the SDK

# Latest stable release
npm install @musingly-ai/core

# Specific version
npm install @musingly-ai/core@1.0.1

# Latest release candidate
npm install @musingly-ai/core@rc

# Specific RC
npm install @musingly-ai/core@1.0.2-rc.3

CI/CD Authentication in Consuming Projects

In GitHub Actions, use the built-in GITHUB_TOKEN:

- name: Setup Node.js
  uses: actions/setup-node@v4
  with:
    node-version: "20"
    registry-url: "https://npm.pkg.github.com"
    scope: "@musingly-ai"

- name: Install dependencies
  run: npm ci
  env:
    NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Note: The consuming repository must be in the same GitHub organization (musingly-ai) or use a PAT with read:packages scope stored as a repository secret.

---

Consuming the Python SDK

From GitHub Releases (CI/Production)

# Find the latest release
gh release list --repo musingly-ai/core | grep sdk-v

# Install specific version
pip install https://github.com/musingly-ai/core/releases/download/sdk-v1.0.1/musingly_core-1.0.1-py3-none-any.whl

# In requirements.txt
musingly-core @ https://github.com/musingly-ai/core/releases/download/sdk-v1.0.1/musingly_core-1.0.1-py3-none-any.whl

From Source (Development)

# Clone the repo and install in editable mode
git clone https://github.com/musingly-ai/core.git
cd core

# Generate the Python SDK first
deno task python-sdk:build

# Install in editable mode
pip install -e python-sdk/

---

Local Development Setup

For cross-service development where another local project needs to import @musingly-ai/core, use the local Verdaccio registry instead of GitHub Packages.

Prerequisites

• Node.js 20+
• Deno (for build scripts)
• npm (comes with Node.js)

Step 1: Start the Local Registry

# From the core repository root
deno task sdk:registry:start

This will: - Install Verdaccio if not present - Create a local configuration in .verdaccio/ - Start the registry at http://localhost:4873

Verify it's running:

deno task sdk:registry:status

Step 2: Publish to the Local Registry

# Build and publish with auto-incremented dev version
deno task sdk:publish:local

This runs the full pipeline: 1. Bumps version to X.Y.Z-dev.N 2. Builds the SDK from the OpenAPI spec 3. Publishes to http://localhost:4873

To publish with a specific channel:

# Publish as RC to local registry
deno run --allow-all tools/scripts/sdk/publish.ts --target npm --local --channel staging

# Publish with explicit version
deno run --allow-all tools/scripts/sdk/version.ts --set 2.0.0-dev.1
deno run --allow-all tools/scripts/sdk/publish.ts --target npm --local --skip-version

Step 3: Configure the Consuming Project

# Generate .npmrc for another project
deno task sdk:registry:configure --project-dir /path/to/other-project

This creates .npmrc in the target project:

# Local development - use Verdaccio for @musingly-ai packages
@musingly-ai:registry=http://localhost:4873

# Fallback to npmjs for everything else
registry=https://registry.npmjs.org/

Step 4: Install in the Consuming Project

cd /path/to/other-project
npm install @musingly-ai/core

The package resolves from the local Verdaccio registry. No GitHub authentication required.

Step 5: Iterate

When you make changes to the SDK source:

# In the core repo - rebuild and republish
deno task sdk:publish:local

In the consuming project:

# Update to the new version
npm update @musingly-ai/core

Stopping the Registry

deno task sdk:registry:stop

Cleaning Published Packages

deno task sdk:registry:clean

Switching Between Local and GitHub Packages

The .npmrc in the consuming project controls where packages resolve from:

# Local development (Verdaccio)
@musingly-ai:registry=http://localhost:4873

# GitHub Packages (CI/production)
@musingly-ai:registry=https://npm.pkg.github.com

Simply change the registry URL and re-run npm install.

---

CLI Reference

Version Management

# Auto-detect channel from current git branch
deno task sdk:version

# Explicit channels
deno task sdk:version:rc          # Bump RC for staging
deno task sdk:version:release     # Promote to stable release

# Preview without writing
deno task sdk:version:dry

# Manual bump
deno run --allow-all tools/scripts/sdk/version.ts --bump minor --channel production

# Set explicit version across all packages
deno run --allow-all tools/scripts/sdk/version.ts --set 2.0.0 --sync

Publishing

# Publish all SDKs (auto-detects CI vs local)
deno task sdk:publish

# Publish NPM only
deno task sdk:publish:npm

# Publish Python only
deno task sdk:publish:python

# Publish to local Verdaccio
deno task sdk:publish:local

# Dry run (build + pack, no publish)
deno task sdk:publish:dry

Local Registry

deno task sdk:registry:start      # Start Verdaccio
deno task sdk:registry:stop       # Stop Verdaccio
deno task sdk:registry:status     # Check status + list versions
deno task sdk:registry:configure  # Generate .npmrc for consumer
deno task sdk:registry:clean      # Clear all published packages

---

Troubleshooting

npm publish fails with 403

Cause: Authentication not configured or token lacks write:packages scope.

Fix: Ensure NODE_AUTH_TOKEN is set. In CI, the setup-node action + GITHUB_TOKEN handles this automatically. Locally, use a PAT.

npm install @musingly-ai/core fails with 404

Cause: GitHub Packages authentication not configured in the consuming project.

Fix: Add .npmrc with the registry and auth token. See Consuming the NPM SDK.

Version already published

Cause: A publish was attempted with a version that already exists in the registry.

Behaviour: The CI pipeline checks for existing versions before publishing and skips if already present. For manual publishes, bump the version first:

deno task sdk:version:rc    # or sdk:version:release

Local Verdaccio not starting

Cause: Port 4873 in use or Verdaccio not installed.

Fix:

# Check if port is in use
lsof -i :4873

# Kill existing process
deno task sdk:registry:stop

# Reinstall Verdaccio
npm install -g verdaccio

# Retry
deno task sdk:registry:start

Python wheel not found in release

Cause: Python SDK build failed during the publish workflow or datamodel-code-generator not installed.

Fix: Check the workflow run logs. The Python SDK requires datamodel-code-generator to generate Pydantic models from JSON Schema:

pip install datamodel-code-generator build
deno task python-sdk:build

---

Known Limitations

1. Python SDK not on PyPI. The Python wheel is distributed via GitHub Releases, not PyPI. Consumers must install from the release URL or from source.

2. Version is not committed to git. Version bumps in CI modify package.json in the workflow runner but do not create a git commit or tag. The version metadata is tracked via the GitHub Release tag (sdk-vX.Y.Z).

3. Shared package (@musingly-ai/shared) is versioned but not published. The shared utilities package has its version synced alongside the SDK for consistency, but it is not published to any registry. It is consumed internally via relative imports.

4. No automatic changelog. Version bumps do not generate a changelog. Release notes are added manually to GitHub Releases.

---

Architecture

File Structure

tools/scripts/sdk/
├── version.ts          # SemVer version management + auto-increment
├── publish.ts          # Publishing orchestrator (CI vs local routing)
└── local-registry.ts   # Verdaccio lifecycle management

.github/workflows/
├── deploy-staging.yml    # → publish-sdk job (RC to GitHub Packages)
├── deploy-production.yml # → publish-sdk job (stable to GitHub Packages + Release)
└── publish-sdk.yml       # Manual publish with version channels

Publish Flow

                    ┌─────────────┐
                    │  version.ts │
                    │  (SemVer)   │
                    └──────┬──────┘
                           │ writes version to
                           │ sdk/package.json
                           │ python-sdk/pyproject.toml
                           │ shared/package.json
                           ▼
                    ┌─────────────┐
                    │ publish.ts  │
                    │ (orchestr.) │
                    └──────┬──────┘
                           │
              ┌────────────┼────────────┐
              │            │            │
        ┌─────▼─────┐ ┌───▼───┐ ┌─────▼─────┐
        │ CI Mode   │ │ Both  │ │ Local Mode│
        │           │ │       │ │           │
        │ GitHub    │ │ Build │ │ Verdaccio │
        │ Packages  │ │  SDK  │ │ localhost │
        │ (npm)     │ │       │ │ :4873     │
        │           │ │       │ │           │
        │ GitHub    │ │       │ │ pip -e .  │
        │ Releases  │ │       │ │ (python)  │
        │ (wheel)   │ │       │ │           │
        └───────────┘ └───────┘ └───────────┘

---

Related Documentation

• CI/CD Guide — Full CI/CD pipeline overview
• Deployment Guide — Edge Function deployment
• Branching Setup — Environment branching strategy
• SDK Reference — SDK API documentation