AI Gateway & LLM Integration

Overview

Libre Workspace provides a centralized AI integration layer enabling managed AI features across the workspace ecosystem (such as Nextcloud Assistant, internal bots, and AI modules).

The system features a Dual-Mode architecture designed to support both multi-tenant SaaS environments and self-hosted, private offline deployments:

SaaS Gateway Mode (Anindus AI Gateway): Designed for managed environments using https://ai.anindus.com (or an enterprise gateway instance). Enforces seat licensing, tiers (e.g. Standard, Pro, CheatUnlimited), usage metrics, and quota allocations.
Local / Direct LLM Mode (e.g., Ollama, vLLM, LiteLLM): Designed for self-hosted instances running local models (e.g., http://localhost:11434 or internal cluster nodes). Does not require a Master Key and provides an unmetered virtual Unlimited seat allocation to all eligible users.

Configuration

Administrators configure the AI integration either through the web portal or via configuration files.

Web Portal

Navigate to Server Administration -> System Configuration (Miscellaneous Settings).

AI Gateway URL (ai_gateway_url): The base HTTP(S) URL of the AI Gateway or local LLM server. Examples: https://ai.anindus.com or http://localhost:11434 or http://10.1.0.50:11434.
AI Gateway Master Key (ai_gateway_master_key): The administrative Bearer token.
- Mandatory when connecting to the Anindus AI Gateway.
- Optional when connecting to local, unauthenticated Ollama or open OpenAI-compatible servers.

URL Normalization

Administrators frequently paste URLs ending in trailing slashes or /v1 (such as https://ai.anindus.com/v1/). The portal automatically sanitizes inputs by stripping trailing slashes and the /v1 suffix internally. Both the base URL and the OpenAI-compatible endpoints are resolved cleanly without duplicate path segments.

Configuration Storage

Settings are persisted in uppercase format in /etc/libre-workspace/libre-workspace.conf:

AI_GATEWAY_URL="https://ai.anindus.com"
AI_GATEWAY_MASTER_KEY="sk-live-..."

The configuration parser clears internal worker caches upon write, guaranteeing immediate synchronization across all Gunicorn worker processes.

Connection Testing & Health Probing

The Test connection button in the Web Portal dynamically probes the configured endpoint to detect the provider type and health status.

Probing Flow

The probing logic follows this sequence:

Gateway Subscription Probe: If a Master Key is provided, the portal issues a request to:
```
GET <URL>/v1/instance/subscription
Authorization: Bearer <master_key>
Accept: application/json
```
If this endpoint returns HTTP 200, the system recognizes the service as an Anindus AI Gateway. It loads customer metadata, available model lists, and tier quotas (e.g., {"standard": {"total": 5, "allocated": 3}}).
Local Model Probing (Ollama / OpenAI Standard): If no Master Key was specified, or if the subscription endpoint returned 404/405 (endpoint not found): The portal probes standard model endpoints:
- GET <URL>/v1/models (Standard OpenAI model list, supported by Ollama, vLLM, LiteLLM)
- GET <URL>/api/tags (Native Ollama model list)
If either returns HTTP 200, the service is recognized as Local / Self-Hosted LLM. The response extracts available model names and initializes the instance in unmetered mode.
Authentication Feedback: If an endpoint returns HTTP 401 or 403 (e.g. an authenticating proxy without a key), the test returns the exact server response (e.g. “Missing or invalid Authorization header”) to guide the administrator.

Live UI Feedback

Upon a successful test, the portal displays:

Measured round-trip latency in milliseconds.
Available seat allocation counts.
In Local Mode: Virtual unmetered status (Unlimited: Unlimited (X active)) and available model names (e.g., Models: llama3.2, mistral).

User Seat & Tier Management

Access to AI features is controlled per user in the Identity Management (IDM) app. User assignments are stored in the database model UserAITier (mapping username to tier).

Tier Options

None (none): The user has no access to AI services. This is the default for new users unless auto-assigned.
Subscription Tiers (e.g., standard, pro, cheatunlimited): Provided dynamically by the AI Gateway subscription.
Unlimited (unlimited): Virtual tier active in Local Mode representing unmetered local inference.

User Creation Form

When creating a new user in the portal (User Management -> Add User):

The AI Access / AI Tier dropdown is available at the bottom of the form.
If the AI Gateway is unconfigured, the field is automatically omitted.
If the Gateway is offline, the field is disabled with an explanatory badge.
Options showing full capacity indicate exhausted quotas and cannot be selected.
When submitted, the LDAP account is created and the chosen AI Tier is recorded in UserAITier.

User Edit Form

When editing an existing user (User Management -> Edit User):

The dropdown reflects the user’s current tier.
Quota Exhaustion vs Display Count: The badge display shows the true total allocation (e.g. Standard: 3/5 allocated, 2 free). During validation, the user currently being edited is excluded from quota exhaustion checks. This ensures that an administrator editing an existing user who already holds a seat can safely save the user without receiving a false “Quota exhausted” error.

Automatic Tier Assignment

To simplify administrative onboarding, Libre Workspace includes an automatic tier assignment mechanism (auto_assign_single_tier_to_all_users):

Condition: Triggered when the administrator saves System Configuration, and exactly one non-zero tier is available.
Single-Tier Gateway: If an AI Gateway subscription provides only one tier (e.g., 20 Standard seats) and the seat count is greater than or equal to the number of non-system users in LDAP, the portal automatically assigns this tier to all eligible users upon save.
Local Mode: In local Ollama / unmetered mode, the single available tier is unlimited. Saving the configuration automatically assigns Unlimited access to all users across the domain.

Internal Service Consumption

Internal components (such as chatbots, Nextcloud apps, or automation modules) resolve credentials using the service helper:

import services.ai_service as ai_service

try:
    creds = ai_service.get_user_ai_credentials(username)
    # Returns:
    # {
    #     "gateway_url": "https://...",
    #     "master_key": "...",
    #     "user_identifier": username,
    #     "tier": "standard"  # or "unlimited"
    # }
except PermissionDenied:
    # User has tier 'none' - AI access blocked
    pass

Troubleshooting & FAQs

“Test Connection reports: URL must not be empty”

Ensure a valid HTTP or HTTPS URL is entered in the AI Gateway URL field before clicking Test Connection.

“Test Connection reports 401 Unauthorized against a local URL”

The remote endpoint (such as an OpenAI reverse proxy or LiteLLM gateway) requires authentication. Enter the corresponding token in the AI Gateway Master Key field and test again.

“Why does the seat count badge show ‘Unlimited’?”

When connected to an Ollama or local OpenAI model server without subscription limits, the system operates in Local Mode with an unmetered pool of seats.

“Can AI access be revoked for individual users in Local Mode?”

Yes. Edit the user in User Management and set AI Access / AI Tier to No Access (none). Even in unmetered Local Mode, users with tier none are denied access.