API Stackr platform
FastAPI platform, API keys, accounts, usage, billing, plugin/provider dispatch, webhooks, admin, privacy, audit, status, AI advisor integration, and operations surfaces.
Orientation
How this handbook is organized
The goal is not to advertise services. It is to explain what each component does, how it connects to neighboring components, and when a design choice becomes useful or dangerous.
Current
Verified system state
Things known to exist now: API Stackr platform services, current hosts, API gateway capabilities, billing/usage structures, webhooks, admin/ops surfaces, Hotel Price Checks architecture, and AI/Core topology.
Documented
Known designs and service ideas
Things represented in project documentation or prior design work, even when not proven live on the current host.
Concept
Future directions
Ideas worth understanding or exploring: agent hosting, AEO/GEO, AI-native tooling, media automation, richer provider marketplaces, and new browser/runtime capabilities.
System map
The complete flow, from visitor to downstream services
People, apps, agents, crawlers, internal toolsHuman traffic and machine traffic both enter through public interfaces.
↓
Edge & deliveryDNS • TLS • CDN • WAF • DDoS controls • compression • cache headers.
Application & API boundaryRouting • sessions • API keys • auth • authorization • rate limiting • request IDs.
Fast data pathRedis/cache • computed snapshots • stale-while-revalidate • read replicas.
↓
Domain servicesInternal APIs own product/business rules.
Workers & schedulersQueues, jobs, retries, polling, batch work, notifications.
AI/CoreInference gateway, tools, retrieval, model providers, agents.
Provider gatewayAdapters for third-party APIs and controlled outbound calls.
↓
StatePostgreSQL / SQLite • object storage • history • audit • metrics.
External servicesHotel APIs • model APIs • Stripe • Resend • SSO • data providers.
OperationsHealth • logs • traces • metrics • backups • alerts • release gates.
Service inventory
Current, documented, and considered services
This section separates operational systems from ideas. A concept appearing here does not imply it is currently sold or deployed.
Hotel Price Checks
Hotel/destination watches, provider adapters, recurring checks, observations, deal detection, email alerts, shared caching, and adaptive scheduling direction.
deep diveAI/Core inference gateway
Private AI/Core role for model access, worker/inference separation, authorization, and centralized model-provider access.
AI architecturePlatform Ops
Cross-system health, recovery awareness, operations checks, and infrastructure supervision.
operationsDeveloper API families
Translation, text/dictionary, data/search, media tools, automation, AI-enhanced utilities, and unified developer APIs.
service designGuidingStar.life
A documented workload on the general web host. Keep architecture, hosting, domain boundaries, and service dependencies distinct from unrelated products.
host mapMediaOps AI
Agent-driven media ingest, FFmpeg pipelines, thumbnails, metadata, storage, publishing, queue orchestration, and processing automation.
agentsworkersAgent infrastructure
Agent hosting, tool/API access, MCP-compatible services, secure execution, workspaces, task coordination, and audit.
MCP/toolsAEO / GEO services
Technical content, structured data, source discoverability, entity clarity, citation-friendly pages, and visibility in AI/search answer systems.
trafficfuture webAI coding & systems agent
Secure coding/workspace automation, deployment assistance, system inspection, task execution, testing, and release validation with strict tool boundaries.
agent runtimerelease engineeringAPI marketplace / agentic commerce
Discoverable APIs/tools with machine-readable capabilities, metering, provider cost, customer price, permissions, and potentially agent-driven purchasing or invocation.
API contractsunit economicsAPI systems
Design APIs as contracts, not just URLs
| Layer | Responsibility | Failure to avoid |
|---|---|---|
| Edge | TLS, routing, WAF, compression, connection handling. | Putting business logic at the edge with no clear ownership. |
| Gateway | Authentication, API keys, quotas, rate limiting, request IDs, routing. | Becoming a giant monolith of product logic. |
| Internal API | Stable business/domain contract owned by your system. | Leaking provider-specific fields directly to clients. |
| Provider adapter | Remote auth, schemas, normalization, retries, limits, response validation. | Letting each product implement the same third-party integration differently. |
| Async interface | Events, queues, jobs, webhooks, long-running work. | Blocking HTTP requests on slow/nonessential work. |
Contract-first approach
Use OpenAPI for HTTP interfaces and AsyncAPI for event/message contracts. Keep schemas versioned, reviewable, and testable.
OpenAPI specification ↗ · AsyncAPI docs ↗request → gateway → internal API
↘ cache
↘ queue/job
↘ provider adapter → remote API
response ← normalize ← policy ← result
Core API design topics
Versioning
Prefer additive evolution; define deprecation windows; test compatibility; publish changelogs.
Idempotency
Critical for payments, webhook processing, retries, jobs, and any mutation that can be delivered twice.
Rate limiting
Enforce per-IP, account, key, endpoint, provider, and plan limits without confusing quotas with burst limits.
Backpressure
When downstream systems are full, slow admission instead of letting queues and memory grow without bound.
API service design
How to think about API service families
Language & text
Translation, normalization, definitions, summarization, classification, extraction, rewrite, language detection, embeddings.
Key concerns: model/provider cost, latency, accuracy, prompt injection, user privacy, language coverage.
Media
Image resize/convert, video transcode, thumbnails, OCR, speech, metadata extraction, moderation, generation.
Key concerns: object storage, queueing, CPU/GPU limits, large uploads, timeouts, content safety.
Data & search
Search, enrichment, public datasets, parsing, geocoding, company/market/reference data, research metadata.
Key concerns: data licensing, freshness, attribution, provider quotas, caching rights.
Automation
Scheduled jobs, triggers, webhooks, workflows, external actions, batch operations, event processing.
Key concerns: idempotency, retries, audit, user approval, secret handling, failure recovery.
AI agents
Tool-using assistants, research agents, coding agents, operations agents, support agents, orchestrators.
Key concerns: permissions, sandboxing, tool policies, verification, cost ceilings, traceability.
Vertical APIs
Hotel pricing, travel search, media pipelines, system monitoring, niche business workflows.
Key concerns: domain-specific normalization, provider diversity, business rules, customer value.
AI architecture
AI is a system of components, not a single model
The model is only one dependency. Production AI also needs routing, context, tools, retrieval, memory, evaluation, safety, observability, and cost controls.
Inference gateway
One internal entry point for model providers, auth, quotas, model selection, failover, logging, and cost accounting.
Model router
Chooses model by task, cost, latency, privacy, context size, reasoning need, modality, and availability.
Prompt/context layer
System instructions, templates, tool schemas, user context, policy context, output constraints.
Retrieval (RAG)
Searches trusted files/data and inserts relevant evidence into the model context.
Embeddings/vector search
Turns content into vectors for semantic search, clustering, deduplication, retrieval, and recommendations.
Tool layer / MCP
Lets models access APIs and applications through explicit tool contracts rather than unrestricted network access.
Agent runtime
Plans tasks, calls tools, maintains state, retries steps, applies policies, and stops on budget/permission boundaries.
Memory/state
Short-term conversation state, long-term user/project facts, task state, vector memory, and durable job state.
Evaluation
Golden tasks, regression suites, judge models, human review, groundedness, safety, latency, and cost benchmarks.
Guardrails & policy
Input filtering, tool allowlists, output checks, secret isolation, authorization, content rules, data boundaries.
Observability
Trace prompts/tools/models, token usage, latency, errors, retrieval quality, tool failure, and user-visible outcomes.
Serving infrastructure
Provider APIs, serverless inference, self-hosted GPU/CPU models, batching, KV cache, quantization, autoscaling.
MCP direction
Model Context Protocol standardizes how applications expose context and tools to models. Treat it as an interface layer—not permission to bypass authentication or security boundaries.
Model Context Protocol ↗Creating AI from scratch
There are three very different meanings of “build AI”
1. Build an AI application
Use existing models, then build context, tools, APIs, retrieval, permissions, evaluation, and UX. This is usually the fastest path to useful software.
2. Fine-tune / adapt a model
Start with pretrained weights and train on task/domain examples. Lower cost than pretraining and often enough for style or specialized behavior.
3. Pretrain a foundation model
Build tokenizer, dataset pipeline, transformer/model architecture, distributed training, checkpoints, evaluation, post-training, and inference stack.
Foundation-model path
| Stage | What you build | Common hidden problem |
|---|---|---|
| Dataset | Collection, licensing, deduplication, filtering, PII removal, balancing, quality scoring. | Bad data dominates model quality and legal risk. |
| Tokenizer | BPE/SentencePiece-like vocabulary, special tokens, multilingual strategy. | Vocabulary choices affect context efficiency and language performance. |
| Architecture | Transformer blocks, attention, MLPs, normalization, position encoding, parameter count. | Scaling model size without matching data/compute wastes resources. |
| Training | Distributed data/model parallelism, optimizer, scheduler, mixed precision, checkpointing. | Instability, hardware failures, data bugs, reproducibility. |
| Post-training | Instruction tuning, preference optimization, tool use, reasoning behaviors, safety training. | Capability and alignment can regress independently. |
| Evaluation | Benchmarks + private task suites + red-team + factuality + calibration + cost. | Optimizing public benchmarks instead of real use cases. |
| Serving | Quantization, batching, KV cache, tensor parallelism, autoscaling, routing. | Inference economics can dwarf application logic. |
Practical rule:
Do not pretrain a model just because you want “your own AI.” First prove that existing models + retrieval + tools + private data + evaluation cannot satisfy the requirement.
Resource management
Every service needs explicit budgets
CPU
Request workers, compression, parsing, encryption, video/image processing, model inference. Control with process limits, cgroups/systemd, container limits, job concurrency.
Memory
Caches, model weights, in-flight requests, response buffers, Python/Node heaps. Use caps, OOM planning, streaming, bounded queues.
Disk / object storage
Databases, logs, uploads, media, backups, model artifacts. Track growth, retention, snapshots, restore cost, IO saturation.
Network
Provider calls, media transfer, database traffic, user downloads. Apply timeouts, compression, CDN, connection pooling, egress awareness.
Database connections
Finite and often more important than CPU. Pool connections, bound worker counts, use replicas/caches, avoid N+1 patterns.
Third-party quotas
Treat API calls as a scarce resource. Budget by provider, tenant, priority, endpoint, and time window.
AI tokens / GPU
Use model routing, token ceilings, context trimming, caching, batching, quantization, and task-specific model choices.
Human attention
Operational complexity is also a resource. Prefer systems that are observable, documented, reversible, and boring under failure.
budget = { cpu, ram, disk, db_connections, network, provider_calls, tokens, dollars, time }
admission(request) only if downstream capacity + quota + cost budget permit it
Scaling
Scale the bottleneck, not everything
Load-spike strategy
- Absorb: CDN, caches, queues, browser caching, connection reuse.
- Protect: rate limits, concurrency limits, request size limits, backpressure.
- Scale: stateless web/API workers horizontally.
- Separate: background workers scale independently from request servers.
- Degrade: disable expensive nonessential features before core flows fail.
- Recover: circuit breakers, retry queues, dead letters, replay, idempotency.
| Bottleneck | Typical response |
|---|---|
| Web CPU | Add stateless replicas; optimize hot paths. |
| Database reads | Cache, index, query tune, read replicas. |
| Database writes | Batch, partition, reduce write amplification, queue noncritical writes. |
| Provider API | Deduplicate, cache, schedule, budget calls, diversify sources. |
| AI inference | Route models, batch, cache, queue, scale GPU/remote provider capacity. |
| Media jobs | Dedicated worker pools, object storage, job priorities. |
Data & caching
Freshness is a policy, not a number
| Pattern | Use | Risk |
|---|---|---|
| Cache-aside | App checks cache then source on miss. | Stampedes without locking/collapsing. |
| Stale-while-revalidate | Serve acceptable old value while refreshing. | Must define maximum staleness. |
| Request collapsing | One refresh serves many identical callers. | Lock expiry and failure handling matter. |
| Negative cache | Remember “not found” briefly. | Can hide newly available data if TTL too long. |
| Materialized snapshot | Precompute expensive query/aggregation. | Refresh lag and rebuild logic. |
| Event invalidation | Invalidate on known changes. | Missed events cause stale state. |
Adaptive refresh score
priority ≈
(demand × volatility × urgency × stale_age × provider_value × info_gain)
÷ provider_cost
For hotel pricing, provider-specific next-check times are better than one global hourly job. For mostly static metadata, days or weeks may be correct. For authentication/security state, cache rules are entirely different.
Security
Think in trust boundaries
Public edge
TLS, WAF, DDoS, bot controls, secure headers, input sizes, origin policy, rate limits.
Identity
Passwords/Argon2, MFA, passkeys/WebAuthn, SSO/OIDC/SAML, session assurance, API keys.
Internal services
Private networks, service identity, least privilege, mTLS where justified, scoped secrets, deny-by-default egress.
Third-party calls
Allowlisted hosts, HTTPS only, SSRF protection, no arbitrary redirects, bounded response sizes, response validation.
Secrets
Root/restricted env files, KMS/HSM where appropriate, rotation, no frontend exposure, no logs, no prompts.
Data
Encryption at rest/in transit, retention, deletion, access logs, privacy controls, backup protection.
AI tools
Tool allowlists, sandbox execution, approval gates, untrusted-content handling, prompt-injection resistance, budget limits.
Operations
SSH keys, patching, service sandboxing, audit logs, alerting, backups, restore drills, incident response.
Passkeys / WebAuthn
Passkeys use public-key cryptography and avoid sending reusable passwords to the site. They are worth understanding for high-assurance customer and administrator access.
MDN Passkeys guide ↗Observability
Measure the system in layers
Request metrics
Rate, errors, duration (RED), status codes, endpoint, account/key, request ID, response size.
Resource metrics
CPU, memory, disk, IO, network, DB connections, queue depth, worker saturation, Redis memory.
Provider metrics
Latency, 429s, error rate, cost/call, unique-value rate, freshness, quota remaining, timeout rate.
AI metrics
Tokens, cost, model, latency, tool calls, retrieval hits, groundedness, refusal/failure rate, task outcome.
Business metrics
Activation, retention, conversion, MRR, attributable provider cost, margin, churn, support load.
Audit metrics
Admin actions, key changes, security events, billing actions, support grants, privacy requests, webhook replays.
Trace across boundaries
A request ID should follow the user request through gateway → service → queue/job → provider → notification, so incidents can be reconstructed without guessing.
Release engineering
A release is a system change, not a file copy
Release gate
build → unit/integration tests → security checks → migration check → deploy → health/readiness → smoke test → metrics → rollback readiness
- Database migrations must be forward-compatible where possible.
- Backups are only useful if restores are tested.
- Feature flags/canaries reduce blast radius.
- Deploy application and workers with compatible contracts.
Recursive validation areas
Backend/API
Frontend/UI
Workers/scheduler
Database/migrations
Security/logging
Install/upgrade
Docs/links
Git/release state
Mail delivery
Email is a distributed delivery system
Transactional mail
Password resets, verification, receipts, alerts, price drops, security notifications. Must be reliable and auditable.
Marketing / lifecycle
Newsletters, onboarding, education, reactivation. Requires consent, segmentation, unsubscribe handling, reputation care.
Inbound mail
Provider webhook receives mail; validate signature, parse safely, render plain text/sanitized content, route to inbox/workflow.
Delivery events
Sent, delivered, deferred, bounced, complained, opened/clicked where used. Feed suppression and health logic.
DNS/authentication
- SPF: who may send for the domain.
- DKIM: cryptographic signature proving message integrity/domain authorization.
- DMARC: policy/reporting using SPF/DKIM alignment.
- Reverse DNS / HELO: important for self-hosted SMTP; managed providers handle more of this.
Reliable send pipeline
event → notification policy → queue → template/render → provider API → delivery webhook → status/suppression
Use idempotency keys so retries do not send duplicates. Bounce/complaint handling must update suppression state.
Current provider note
Resend is documented in the current platform for transactional/inbound mail workflows and in Hotel Price Checks for production alert delivery.
Webhooks & events
Push changes instead of polling when possible
Webhook checklist
- HTTPS destination required.
- Sign payload (HMAC or provider-supported scheme).
- Include delivery ID + timestamp.
- Reject old/replayed deliveries.
- Retry with exponential backoff + jitter.
- Dead-letter after bounded attempts.
- Make receiver idempotent.
- Provide delivery history and manual replay.
Internal event bus
Use events when multiple components need to react independently: user.created, subscription.changed, price.observed, deal.detected, mail.bounced, provider.degraded, job.failed.
Contract them.
AsyncAPI can document message/event interfaces in the same spirit OpenAPI documents HTTP APIs.
Hotel Price Watch
A vertical example that connects nearly every topic
Customer watchHotel or destination + dates + guests + rooms + rate preferences.
↓
Canonical search keyDeduplicates identical searches across customers.
Refresh intelligenceDemand, volatility, check-in urgency, provider cost, quota, usefulness.
↓
Provider workersCall owned integration API + remote hotel/deal APIs through adapters.
NormalizerTotal expected cost: room + mandatory taxes + mandatory fees, normalized currency.
↓
HistoryPrice observations and availability over time.
Deal detectorNew low, target reached, significant drop, better terms.
ForecastChance a lower comparable rate appears before check-in.
↓
Notification policyEmail/SMS/push with anti-spam rules, quiet hours, dedupe, booking/deep links if used.
Signals for adaptive refresh
- Provider per-call price and quota.
- Provider cache/storage contract rules.
- Hotel + destination popularity.
- Number of active watches sharing the same query.
- Days until check-in.
- Price and inventory volatility.
- Recent historic-low event.
- Market events, holidays, conferences, sports.
- Provider reliability and cheapest-price win rate.
- Expected information gain per call.
Technology inventory
Components already in or around the current systems
This is a learning map of technologies documented in the current platform and adjacent workloads. Each tool is useful because of the role it fills, not because every project needs the same stack.
Current
Caddy
Public HTTP/TLS edge and reverse proxy: redirects, compression, routing, access logs, and response security headers.
edge securityCurrent
FastAPI + Uvicorn
Python application/API layer and ASGI serving for API Stackr and related services.
API architectureworker limitsCurrent
PostgreSQL
Primary relational store for the platform: accounts, usage, billing, plugins, security/audit data, jobs, privacy, and operations state.
datamigrationsCurrent
Redis
Local cache/coordination layer used for fast state, rate limiting, and workload coordination.
cachingmemory budgetsCurrent
SQLite (Hotel workload)
Documented authoritative store in the Hotel Price Checks architecture, with WAL/foreign keys/busy-timeout considerations.
hotel architectureCurrent
systemd
Process lifecycle, restart behavior, dependency ordering, sandboxing, resource caps, timers, and backup jobs.
resource controlsoperationsCurrent
Alembic
Database migration system with migration checks and deployment compatibility gates.
release gatesCurrent
Resend
Transactional delivery and inbound-email/webhook integration documented in the platform and Hotel Price Checks.
mail architectureCurrent
SSO / identity providers
Google, Microsoft/Entra, GitHub, GitLab and broader identity-linking/SAML support are represented in current platform capabilities/configuration.
identity securityCurrent
AI model/provider access
Platform AI advisor support plus private AI/Core gateway and serverless/provider inference path.
AI stackmodel buildingCurrent
SentinelX + Fail2ban
Remote-management/operations agent and host-level abuse/SSH defense as part of infrastructure operations.
host securityoperationsCurrent
Bootstrap 5
This handbook uses Bootstrap 5 for responsive layout; framework use is presentation-layer only and separate from backend architecture.
frontend evolutionCurrent topology reference
Known host roles
Role-level topology is shown here; private addresses and credentials are intentionally omitted from this educational page.
Current
www.apistackr.com
Primary public API Stackr origin; apex domain redirects to the www host.
Current
admin.apistackr.com
Administrative interface routed at the production edge.
Current
apistackr-prod
Primary API Stackr host: edge proxy, platform/API processes, PostgreSQL, Redis, backups, service sandboxing.
Current
gen-web
General web/workload host including Hotel Price Checks and GuidingStar-related workloads.
Current
ai-model
Internal AI/Core and worker compute role; centralizes private model access and inference duties.
Current
platform-ops
Cross-system operations, health evaluation, recovery verification, and infrastructure supervision.
Building traffic
Traffic comes from distribution systems, not one trick
Search / SEO
Technical crawlability, useful pages, internal linking, canonical URLs, structured data, sitemaps, page experience, topical depth.
Developer acquisition
Docs, code examples, SDKs, public schemas, changelogs, status pages, sample projects, GitHub, tutorials, comparison pages.
Content engine
Reference pages, how-to guides, architecture explainers, use cases, benchmarks, problem/solution pages, glossary pages.
AEO / GEO
Clear entities, source-backed facts, answerable sections, structured data, stable URLs, machine-readable APIs/docs, original data.
Email audience
Permission-based updates, alerts, educational sequences, release notes, digests, lifecycle messages—not purchased lists.
Partnerships & referrals
Affiliate relationships, integration directories, partner docs, marketplaces, co-marketing, communities, ecosystem listings.
Core Web Vitals
Google continues to recommend good LCP, INP, and CLS as part of strong page experience. Treat performance as UX first and search support second.
- LCP: loading performance.
- INP: responsiveness.
- CLS: visual stability.
Traffic measurement funnel
impression → visit → engaged visit → signup/lead → activation → paid → retained → expansion/referral
Measure by channel and landing page. “More traffic” is not valuable if it does not increase qualified activation or durable revenue.
Making sales
Sales is matching a costly problem to a credible outcome
Offer design
- Who has the problem?
- How often does it happen?
- What does it cost them now?
- What measurable outcome do you create?
- How quickly can they see value?
- What proof reduces perceived risk?
- What is the easiest next step?
API/software funnel
education/docs → sample/try → account/key → first successful request → repeated use → paid threshold → expansion
For API products, activation is often “made first successful API call,” not “created account.” Instrument that event explicitly.