ai_services app

Multi-provider AI: chatbot, embeddings for RAG, AI-generated reports. Also hosts onboarding checklists — a plain step-by-step feature (template → steps → per-user progress), no AI/chat involved; it rides this module purely for licensing/gating.

Models (12)

• AIConfig — per-org provider config (chat + embedding); encrypted api_key; use_platform_default fallback flag; is_active / is_verified; per-content-type ai_scope_* toggles (members, forum_posts, compliance, documents, events, mentorship, notifications, onboarding, reports, learning)
• PlatformAIConfig — singleton platform-wide default AI provider (used when an org doesn't configure its own, or explicitly opts into the default). Carries its own temperature and max_tokens, which govern every org on the platform default — see Who owns the temperature and token budget
• AvailableAIModel — curated per-provider model catalog row, synced from the vendor's live model-listing API; organization=None for the platform-wide catalog, or a specific org for its own BYOM catalog; deprecated flags a model that should no longer be offered — either it disappeared from a later sync, or a call to it 404'd — without deleting it (orgs still configured with it keep working); supports_temperature and not_found_at_provider are learned at call time rather than synced (see Temperature capability and Models that 404)
• ContentEmbedding — one embedding chunk of indexable content (member/forum_post/document/compliance/learning); embedding is a JSONField (not a native pgvector column) holding a provider-generated vector; carries visibility/chapter_id mirrors from the source record so RAG search can enforce document-level access
• EmbeddingJob — long-running batch indexing job state (pending → running → completed/failed)
• ReportRequest — AI-assisted admin report request (quarterly chapter, annual org, compliance summary, membership overview, financial summary); result stores the generated markdown directly on the row (no file upload)
• OnboardingTemplate — admin-authored checklist definition; org-scoped, is_active flag, order
• OnboardingStep — individual step in a template
• OnboardingProgress — per-user completion state for a single step (completed, completed_at)
• ChatFeedback — thumbs up/down feedback on a single AI chat response; unique per (user, message_id) so re-submitting revises the same row
• ChatThread — a persisted AI conversation for one user (title, message_count, archived, last_message_at)
• ChatMessage — a single message within a ChatThread (role, content, tool_calls, sources citations, sequence_number)

Key endpoints

Mounted at /api/ai-services/.

URL	Purpose
GET/PATCH /api/ai-services/organizations/<id>/ai-config/	Org AI config
POST /api/ai-services/organizations/<id>/ai-config/test/	Validate configured credentials
GET /api/ai-services/organizations/<id>/ai-config/providers/	Available providers
GET /api/ai-services/organizations/<id>/ai-config/providers/<provider>/models/	Synced model catalog for a provider (org-specific → platform-wide → static fallback)
POST /api/ai-services/organizations/<id>/ai-config/providers/<provider>/sync/	Manually refresh this org's live model catalog from the vendor now
GET /api/ai-services/organizations/<id>/embedding-jobs/, POST .../embedding-jobs/trigger/	List embedding jobs / trigger a reindex
GET/POST /api/ai-services/organizations/<id>/reports/, GET .../reports/<report_id>/	Request an AI report / check status
POST /api/ai-services/resume-parse/, POST .../resume-parse/confirm/	Parse and confirm a resume upload
GET/POST /api/ai-services/organizations/<id>/onboarding/templates/, .../steps/; GET/POST .../onboarding/progress/	Onboarding checklist admin + per-user progress
POST/GET /api/ai-services/chat/feedback/	Submit or fetch the caller's own thumbs up/down feedback on a message
GET /api/ai-services/platform/chat/feedback/	Platform-admin feed of recent chat feedback (defaults to last 30 days, negative sentiment)
GET/POST /api/ai-services/chat/threads/, GET/PATCH .../chat/threads/<id>/	List/rename/archive the caller's chat threads. POST is a get-or-create against the caller's existing active (non-archived) thread for the org, not an unconditional insert — GreekManageAI is one ongoing thread per (user, org), so this can't be used to spawn duplicates
WS ws/chat/	Chatbot WebSocket connection (ChatConsumer). With no ?thread_id=, connect() resolves and resumes the caller's existing active thread for the org (soft one-thread-per-org) rather than leaving a blank slate that would create a new one

Permissions

• IsNationalAdmin — AI config + report management
• IsAuthenticated — chat (WS + feedback + threads) and onboarding (per-user)
• IsPlatformAdminPerm — platform chat-feedback review queue

Background tasks

• generate_member_embeddings(org_id) / generate_forum_post_embeddings(org_id) / generate_compliance_embeddings(org_id) / generate_document_embeddings(org_id) — per-content-type embedding generation for one org
• generate_org_embeddings(org_id) — fans out the four tasks above for a single org; gated by is_module_enabled(org_id, "ai_services")
• generate_all_embeddings() — Celery beat nightly at 4:00 UTC; regenerates embeddings across all orgs
• trigger_full_reindex(org_id) — on-demand full reindex, triggered via POST .../embedding-jobs/trigger/
• generate_report_task(report_request_id) — runs the report job (services/report_generator.py), writes markdown to ReportRequest.result
• sync_all_available_models() — Celery beat daily at 2:00 UTC; refreshes the platform-wide AvailableAIModel catalog plus every org's own verified BYOM catalog
• sync_provider_models_task(provider, api_key, org_id=None) — syncs one provider's catalog (platform-wide if org_id is None); also invoked synchronously by the "sync now" endpoints for an immediate refresh

External integrations

• Anthropic (Claude — chat only; does not support embeddings)
• OpenAI (GPT chat; text-embedding-3-small, 1536-dim)
• Google (Gemini chat; text-embedding-004, 768-dim)

Notable patterns

Provider abstraction

apps/ai_services/providers/:

• base.py — AIProvider ABC (stream_chat(...), chat(...), validate_credentials(temperature=None), supports_embeddings(), generate_embedding(text), get_embedding_dimensions(), get_embedding_model(), get_available_models(), list_live_models(api_key)); the constructor takes supports_temperature alongside the key and model name
• anthropic_provider.py, openai_provider.py, google_provider.py — concrete implementations

Who owns the temperature and token budget

Whoever picks the model picks its sampling temperature — and its completion-token budget.

get_ai_provider() decides which config supplies the model — the org's own AIConfig, or PlatformAIConfig. The effective temperature and max_tokens are attached to the provider it returns, so they always come from that same row, and the chat turn reads provider.temperature / provider.max_tokens rather than looking either value up again.

org state	model from	temperature + max_tokens from
use_platform_default = true	platform	platform
own key, active + verified	org	org
no usable config (fallback)	platform	platform

Before #879 (temperature) and #880 (max_tokens) both values were read from the org's AIConfig at the point of the turn, regardless of which config had supplied the model. An org on the platform default therefore governed the sampling of a model the platform had chosen, and the platform had no way to set one for its own model — which is also what made #862 hard to diagnose, since the offending value lived in an org row nobody had touched.

Resolution happens in the factory rather than at the call site deliberately: a second lookup would need its own copy of the priority ladder above, and two copies drift.

Two things sit on top of this and still win:

• AvailableAIModel.supports_temperature (#862) — a model that accepts only its default temperature has the parameter suppressed entirely, whichever config supplied the value.
• The org-level controls (both of them) are disabled, with an explanation, while the org is on the platform default, because their values are not used in that state.

The budget matters more than the sampling: models differ in maximum completion length, and some reject values others accept — so an org row's budget applied to a platform-chosen model can truncate responses or error outright (#880). The shared 100–8192 bound is knowingly not right for every model in the catalog; if that is ever tightened it must become model-aware in both serializers at once.

The task-specific internal callers (services/resume_parser.py, services/report_generator.py) deliberately keep their own hardcoded temperatures. Those are tuned for extraction accuracy, not user preference, and are not meant to follow a chat setting.

• factory.py — get_ai_provider(org_id) resolves the provider: org's own AIConfig first (unless use_platform_default is set), falling back to PlatformAIConfig, then erroring if neither is usable; get_provider_models(provider, org_id=None) resolves the model list the same way (org catalog → platform catalog → static get_available_models() fallback)

Live model catalog sync

get_available_models() is a small hardcoded fallback list per provider — it only exists for the rare case a sync has never run. The real catalog is AvailableAIModel, kept current by:

1. AIProvider.list_live_models(api_key) — calls the vendor's actual model-listing API (client.models.list() for all three providers), returning raw entries.
2. services/model_sync.py's curate() — groups raw ids into "families" (dated/numbered snapshot suffixes stripped, e.g. gpt-4o-2024-08-06 and gpt-4o collapse to one gpt-4o entry), drops non-chat/preview/experimental noise, and keeps the newest or bare-alias entry per family.
3. sync_provider_catalog(provider, api_key, organization=None) — filters curate()'s output down to the newest AI_MODEL_CATALOG_MAX_GENERATIONS (env-overridable, default 2) generations via _limit_to_latest_generations(), ranking the distinct major_version values numerically (e.g. "5" > "4.1" > "4"), then upserts the survivors into AvailableAIModel; anything missing from the fresh sync (aged out of the cap, or genuinely removed by the vendor) is marked deprecated=True, never deleted, so an org still configured with a since-removed model keeps working. A model already known to 404 stays deprecated even though the listing still contains it — see Models that 404.

Temperature capability

Some chat models accept only their default sampling temperature and answer any explicit value with a 400 (code='unsupported_value', param='temperature'). This is not derivable from the model id. Measured against OpenAI's live catalog: gpt-5, gpt-5.6-luna and gpt-5.6-sol reject an explicit temperature, while gpt-5.1, gpt-5.4-mini and gpt-5.4-nano accept one. No version prefix or family name separates the two groups, and no vendor catalog advertises the property — so a model-name rule would be wrong on arrival and wrong again with every release.

It is therefore learned, not predicted:

1. OpenAIProvider matches that 400 on its structured fields (_is_unsupported_temperature), drops the parameter and reissues. The retry happens before any token is streamed, so the turn succeeds rather than surfacing a provider error.
2. services/model_capabilities.record_temperature_unsupported() writes supports_temperature=False onto every catalog row for that (provider, model_id) — the constraint belongs to the vendor's model, not to one tenant.
3. get_ai_provider() seeds each provider from temperature_supported() (org catalog → platform catalog → optimistic True), so later calls omit the parameter without paying for the rejected round trip.
4. sync_provider_catalog() deliberately leaves supports_temperature out of its defaults, so a re-sync cannot reset a learned False.

validate_credentials(temperature=...) sends the temperature a real turn would, making verification both honest and the cheapest place for the discovery to happen — before an org's first chat turn. get_provider_models() surfaces the flag so the settings UI can disable a control the model would ignore.

Models that 404

A vendor can keep listing a model id it will no longer serve. gpt-5.1-chat-latest and gpt-5.3-chat-latest were still returned by client.models.list() while every call to them answered 404 - The model \X` has been .... Because sync_provider_catalog()` deprecates a row only when the id disappears from the listing, that path never fired: the dropdown kept offering both, and picking either failed Verify and every chat turn.

The 404 is learned the same way the temperature 400 is:

1. Each provider matches its SDK's own not-found exception — openai.NotFoundError, anthropic.NotFoundError, google.genai.errors.ClientError with code == 404 — and then asks services/model_capabilities.looks_like_model_not_found() whether that 404 is about this model (code='model_not_found', or an error body naming the id we asked for). A 404 about anything else (unknown project, wrong path) is left alone. Detection is never a rule about how the id is spelled — nothing in -chat-latest distinguishes a live alias from a retired one, the same lesson the temperature capability taught.
2. record_model_not_found() sets deprecated=True and not_found_at_provider=True on every catalog row for that (provider, model_id), so get_provider_models() — which filters deprecated=False — stops offering it immediately, for every org.
3. sync_provider_catalog() re-applies deprecated=True to not_found_at_provider rows after its upsert. Without that step the next sync, which still sees the id in the vendor's listing, would put an uncallable model straight back in the dropdown. Its return value (the count of usable models) excludes them.
4. stream_chat() replaces the raw provider 404 with model_unavailable_message() — the model was offered by our own dropdown, so the error says it has been withdrawn and where to pick another.

The catalog write is guarded at both layers (_record_model_not_found / _arecord_model_not_found on AIProvider): it is bookkeeping piggybacking on a failing call, and it legitimately fails when code is deployed ahead of its migration, when the caller still owes the user a real error.

Platform and BYOM catalogs are synced independently (organization=None vs. a specific org) because vendor account tiers can see different models than the platform's own key. AIConfigSerializer/PlatformAIConfigSerializer expose model_deprecated (true once a sync has run for that provider but no longer lists the configured model), and the frontend shows a warning banner plus an "advanced: custom model ID" text input so admins aren't blocked by the curated dropdown.

Variable-dim embeddings

Only OpenAI (text-embedding-3-small, 1536-dim) and Google (text-embedding-004, 768-dim) support embeddings; Anthropic's generate_embedding() raises immediately (error_type="embeddings_not_supported") — chat-only orgs fall back to keyword/trigram search for all content types. ContentEmbedding.embedding is a plain JSONField, not a native pgvector column, so it stores whichever dimension the org's provider produced without a fixed-width constraint.

RAG pipeline

→ AI / RAG pipeline

Chat consumer

apps/ai_services/consumers.py — ChatConsumer(AsyncWebsocketConsumer):

1. Authenticate user from scope["user"]; close with a specific code if unauthenticated (4001), no resolvable org (4002), module disabled (4003), or no active AI config (4004)
2. Resume an existing ChatThread if thread_id is supplied (connect query string or per-message), else create one on the first user message
3. Persist the user message to ChatMessage
4. Route the query through query_router (intent classification → the matching search service — member/forum/document/compliance/learning/event/onboarding/retention) to get rendered context + raw results for citations
5. Build the system prompt from that context and stream the response via the provider's stream_chat(...)
6. Emit a sources frame after the response, built from the raw search results
7. Persist the assistant response to ChatMessage

Each turn carries a stable message_id so the frontend can attach ChatFeedback and sources to the right assistant message.

Socket lifecycle

The consumer does not itself enforce one thread per connection — as above, a message can carry a thread_id different from the one the socket connected with, and receive() simply switches to serving it. The one-thread-per-socket property is a client behavior: use-chat.ts closes and reopens the socket only on a genuine thread switch, not on every server frame that happens to name a thread. Both connected (an existing thread resolved during the handshake) and thread_created (a new thread created on this socket) record the thread they name on connectedThreadRef before updating the store, so the thread-change effect sees no change and leaves a healthy socket alone — without that, a frame naming the socket's own thread looked like a switch and tore the connection down mid-turn. Only the socket instance that still owns wsRef.current is allowed to clear it on close — a superseded socket's own close event can arrive after its replacement has already been stored, and clearing the ref there would strand the live connection.

Independent of the client's reconnect behavior, the backend holds one invariant defensively: a turn owns its thread id for its whole lifetime. receive() pins _turn_thread_id when the message arrives and the trailing assistant persist uses that, not self.thread_id, which disconnect() nulls. Channels dispatches consumer events sequentially, so disconnect() is not expected to interleave with a streaming receive() — the pin means nothing has to depend on that, since addressing a persist to thread None fails silently (_persist_message swallows ChatThread.DoesNotExist) and would lose a finished answer with no trace. Note this does not make a turn survive the application task being cancelled — after Daphne's application_close_timeout the trailing persist does not run at all (#868).

BYOM (Bring Your Own Model)

AIConfig has a single encrypted api_key field (shared by chat and embedding calls, since a provider account covers both) plus a separate use_platform_default flag. Org admins can uncheck platform default and supply their own provider + key; if they don't, get_ai_provider falls back to PlatformAIConfig.

Code paths

• Models: backend/apps/ai_services/models.py
• Providers: backend/apps/ai_services/providers/
• Consumers: backend/apps/ai_services/consumers.py
• Services (search, routing, embeddings, reports): backend/apps/ai_services/services/
• Tasks: backend/apps/ai_services/tasks.py

Related

• AI / RAG pipeline
• AI Services module (user-facing)
• AI chatbot (member)