*************************************** AI Gateway & LLM Integration *************************************** Overview ======== Libre Workspace provides a centralized AI integration layer enabling managed AI features across the workspace ecosystem (such as Nextcloud Assistant, internal bots, and AI modules). The system features a **Dual-Mode** architecture designed to support both multi-tenant SaaS environments and self-hosted, private offline deployments: 1. **SaaS Gateway Mode (Anindus AI Gateway):** Designed for managed environments using ``https://ai.anindus.com`` (or an enterprise gateway instance). Enforces seat licensing, tiers (e.g. *Standard*, *Pro*, *CheatUnlimited*), usage metrics, and quota allocations. 2. **Local / Direct LLM Mode (e.g., Ollama, vLLM, LiteLLM):** Designed for self-hosted instances running local models (e.g., ``http://localhost:11434`` or internal cluster nodes). Does not require a Master Key and provides an unmetered virtual ``Unlimited`` seat allocation to all eligible users. .. contents:: Table of Contents :local: :depth: 2 Configuration ============= Administrators configure the AI integration either through the web portal or via configuration files. Web Portal ---------- Navigate to **Server Administration** -> **System Configuration** (**Miscellaneous Settings**). * **AI Gateway URL** (``ai_gateway_url``): The base HTTP(S) URL of the AI Gateway or local LLM server. Examples: ``https://ai.anindus.com`` or ``http://localhost:11434`` or ``http://10.1.0.50:11434``. * **AI Gateway Master Key** (``ai_gateway_master_key``): The administrative Bearer token. * **Mandatory** when connecting to the Anindus AI Gateway. * **Optional** when connecting to local, unauthenticated Ollama or open OpenAI-compatible servers. URL Normalization ----------------- Administrators frequently paste URLs ending in trailing slashes or ``/v1`` (such as ``https://ai.anindus.com/v1/``). The portal automatically sanitizes inputs by stripping trailing slashes and the ``/v1`` suffix internally. Both the base URL and the OpenAI-compatible endpoints are resolved cleanly without duplicate path segments. Configuration Storage --------------------- Settings are persisted in uppercase format in ``/etc/libre-workspace/libre-workspace.conf``: .. code-block:: ini AI_GATEWAY_URL="https://ai.anindus.com" AI_GATEWAY_MASTER_KEY="sk-live-..." The configuration parser clears internal worker caches upon write, guaranteeing immediate synchronization across all Gunicorn worker processes. Connection Testing & Health Probing =================================== The **Test connection** button in the Web Portal dynamically probes the configured endpoint to detect the provider type and health status. Probing Flow ------------ The probing logic follows this sequence: 1. **Gateway Subscription Probe:** If a Master Key is provided, the portal issues a request to: .. code-block:: text GET /v1/instance/subscription Authorization: Bearer Accept: application/json If this endpoint returns HTTP 200, the system recognizes the service as an **Anindus AI Gateway**. It loads customer metadata, available model lists, and tier quotas (e.g., ``{"standard": {"total": 5, "allocated": 3}}``). 2. **Local Model Probing (Ollama / OpenAI Standard):** If no Master Key was specified, or if the subscription endpoint returned 404/405 (endpoint not found): The portal probes standard model endpoints: * ``GET /v1/models`` (Standard OpenAI model list, supported by Ollama, vLLM, LiteLLM) * ``GET /api/tags`` (Native Ollama model list) If either returns HTTP 200, the service is recognized as **Local / Self-Hosted LLM**. The response extracts available model names and initializes the instance in unmetered mode. 3. **Authentication Feedback:** If an endpoint returns HTTP 401 or 403 (e.g. an authenticating proxy without a key), the test returns the exact server response (e.g. *"Missing or invalid Authorization header"*) to guide the administrator. Live UI Feedback ---------------- Upon a successful test, the portal displays: * Measured round-trip latency in milliseconds. * Available seat allocation counts. * In Local Mode: Virtual unmetered status (``Unlimited: Unlimited (X active)``) and available model names (e.g., ``Models: llama3.2, mistral``). User Seat & Tier Management =========================== Access to AI features is controlled per user in the Identity Management (IDM) app. User assignments are stored in the database model ``UserAITier`` (mapping ``username`` to ``tier``). Tier Options ------------ * **None** (``none``): The user has no access to AI services. This is the default for new users unless auto-assigned. * **Subscription Tiers** (e.g., ``standard``, ``pro``, ``cheatunlimited``): Provided dynamically by the AI Gateway subscription. * **Unlimited** (``unlimited``): Virtual tier active in Local Mode representing unmetered local inference. User Creation Form ------------------ When creating a new user in the portal (**User Management** -> **Add User**): * The **AI Access / AI Tier** dropdown is available at the bottom of the form. * If the AI Gateway is unconfigured, the field is automatically omitted. * If the Gateway is offline, the field is disabled with an explanatory badge. * Options showing full capacity indicate exhausted quotas and cannot be selected. * When submitted, the LDAP account is created and the chosen AI Tier is recorded in ``UserAITier``. User Edit Form -------------- When editing an existing user (**User Management** -> **Edit User**): * The dropdown reflects the user's current tier. * **Quota Exhaustion vs Display Count:** The badge display shows the true total allocation (e.g. ``Standard: 3/5 allocated, 2 free``). During validation, the user currently being edited is excluded from quota exhaustion checks. This ensures that an administrator editing an existing user who already holds a seat can safely save the user without receiving a false "Quota exhausted" error. Automatic Tier Assignment ========================= To simplify administrative onboarding, Libre Workspace includes an automatic tier assignment mechanism (``auto_assign_single_tier_to_all_users``): * **Condition:** Triggered when the administrator saves System Configuration, and exactly **one** non-zero tier is available. * **Single-Tier Gateway:** If an AI Gateway subscription provides only one tier (e.g., 20 Standard seats) and the seat count is greater than or equal to the number of non-system users in LDAP, the portal automatically assigns this tier to all eligible users upon save. * **Local Mode:** In local Ollama / unmetered mode, the single available tier is ``unlimited``. Saving the configuration automatically assigns ``Unlimited`` access to all users across the domain. Internal Service Consumption ============================ Internal components (such as chatbots, Nextcloud apps, or automation modules) resolve credentials using the service helper: .. code-block:: python import services.ai_service as ai_service try: creds = ai_service.get_user_ai_credentials(username) # Returns: # { # "gateway_url": "https://...", # "master_key": "...", # "user_identifier": username, # "tier": "standard" # or "unlimited" # } except PermissionDenied: # User has tier 'none' - AI access blocked pass Troubleshooting & FAQs ====================== "Test Connection reports: URL must not be empty" ------------------------------------------------ Ensure a valid HTTP or HTTPS URL is entered in the AI Gateway URL field before clicking Test Connection. "Test Connection reports 401 Unauthorized against a local URL" -------------------------------------------------------------- The remote endpoint (such as an OpenAI reverse proxy or LiteLLM gateway) requires authentication. Enter the corresponding token in the **AI Gateway Master Key** field and test again. "Why does the seat count badge show 'Unlimited'?" -------------------------------------------------- When connected to an Ollama or local OpenAI model server without subscription limits, the system operates in Local Mode with an unmetered pool of seats. "Can AI access be revoked for individual users in Local Mode?" -------------------------------------------------------------- Yes. Edit the user in **User Management** and set **AI Access / AI Tier** to **No Access (none)**. Even in unmetered Local Mode, users with tier ``none`` are denied access.