Skip to main content

Admin Workflow

What it is​

The Admin workflow covers platform configuration, monitoring, and management tasks performed by C_AI Engineers and Admin Viewers. This includes tenant management, corpus configuration, criteria import, and run monitoring.

When to use​

Use this workflow when:

  • Setting up a new tenant
  • Importing or updating assessment criteria
  • Managing the regulatory corpus
  • Monitoring assessment runs and system health
  • Reviewing audit logs for compliance

Do not use when:

  • Submitting evidence documents (see Portal User Workflow)
  • Running assessments (Portal User function)
  • Debugging application code (developer function)

Prerequisites​

Before starting, ensure you have:

  • Admin credentials with appropriate role (C_AI_ENGINEER or ADMIN_VIEWER)
  • Understanding of the tenant you're configuring
  • Any configuration files ready (criteria CSV, corpus documents)

Step-by-step​

Step 1: Access the admin console​

  1. Navigate to /admin/login
  2. Select your role:
    • C_AI Engineer: Full admin access including Engine Studio
    • Admin Viewer: Read-only access to monitoring and audit logs
  3. Select the tenant you want to manage
  4. Click Login

You will be redirected to the Admin Observability dashboard.

Step 2: Navigate the admin interface​

The admin console has several sections accessible from the left sidebar:

SectionPurpose
Run HistoryView past assessment runs and their status
MetricsMonitor system performance and throughput
Audit LogsReview all platform actions for compliance
AI PerformanceEvaluate assessment quality metrics
Corpus ManagerManage regulatory document corpus
Engine StudioConfigure retrieval, chunking, and prompts
Criteria ManagerImport and manage assessment criteria

Step 3: Import assessment criteria​

To set up requirements for a new pack:

  1. Navigate to Criteria Manager
  2. Click Import Criteria
  3. Select your criteria CSV file
  4. Review the import preview showing criteria and requirements
  5. Confirm the import

CSV format requirements:

  • Headers: S/No, Criterion, Description, Check items
  • Check items can span multiple rows under the same S/No
  • Roman numerals (i, ii, iii) are parsed as sub-items

Step 4: Manage the regulatory corpus​

To add regulatory documents that provide context for assessments:

  1. Navigate to Corpus Manager
  2. Click Create Document or Upload Document
  3. Provide metadata (title, effective date, version)
  4. Upload the document file
  5. Create an Edition to group related versions
  6. Activate the edition to make it available for assessments

Step 5: Monitor assessment runs​

To review run health and results:

  1. Navigate to Run History
  2. View the list of recent runs with status indicators
  3. Click a run to see details:
    • Timeline showing processing stages
    • Assessment summary (COMPLETE, PARTIAL, MISSING counts)
    • Evaluation metrics if available
  4. For failed runs, review the failure reason

Step 6: Review audit logs​

To verify platform activity:

  1. Navigate to Audit Logs
  2. Use filters to narrow by:
    • Date range
    • Action type (RUN_STARTED, DOCUMENT_UPLOADED, etc.)
    • User or session
  3. Click an entry to see full details including metadata

Step 7: Configure engine settings (C_AI Engineer only)​

To adjust retrieval and assessment behavior:

  1. Navigate to Engine Studio
  2. Select the configuration type:
    • Retrieval: Vector search parameters
    • Chunking: Document splitting strategy
    • Prompt: LLM prompt templates
    • Agent: Assessment workflow configuration
  3. Create a new version or edit an existing draft
  4. Test changes in the Test Console
  5. Publish when ready

Step 8: Activate configurations​

After publishing engine configurations:

  1. Navigate to the Releases section in Engine Studio
  2. Find the release you want to activate
  3. Click Activate for the target environment (DEV, STAGING, or PROD)
Pack immutability

Once a pack has any ACTIVE bindings (corpus, criteria, or engine), it becomes immutable. You cannot modify active configurations—instead, create new versions.

Example​

Scenario: Setting up a new project pack with custom criteria.

Input:

  • Tenant: "project-alpha"
  • Criteria CSV: 20 criteria, 70 requirement items
  • Corpus: 3 regulatory documents

Process:

  1. Login as C_AI Engineer → Select "project-alpha" tenant
  2. Import criteria CSV → 70 requirements created
  3. Upload 3 corpus documents → Create edition "v1.0"
  4. Activate corpus edition → Ready for assessments
  5. Verify in Run History → No runs yet (as expected)

Result:

  • Tenant configured with 70 requirements across 20 criteria
  • Regulatory corpus active for assessment context
  • Portal Users can now upload evidence and run assessments

Troubleshooting​

Cannot access admin console​

Symptom: Redirected to login or shown access denied.

Causes and fixes:

  • Session expired: Re-authenticate at /admin/login
  • Insufficient role: Request C_AI_ENGINEER or ADMIN_VIEWER role
  • Wrong tenant selected: Ensure you're in the correct tenant context

Criteria import fails​

Symptom: Import shows errors or creates fewer items than expected.

Causes and fixes:

  • CSV format incorrect: Verify headers match expected format
  • Duplicate S/No values: Ensure unique section numbers
  • Empty check items: Every S/No must have at least one check item

Corpus activation fails​

Symptom: Cannot activate a corpus edition.

Causes and fixes:

  • Pack already has active corpus: Deactivate existing first or create new pack
  • Immutability constraint: If pack has active bindings, you cannot change them
  • Missing edition: Ensure edition exists before activation

Run history shows no runs​

Symptom: Run History is empty even after Portal Users run assessments.

Causes and fixes:

  • Wrong tenant context: Switch to the correct tenant
  • Runs in different environment: Check DEV vs STAGING vs PROD
  • No runs actually started: Verify with Portal Users

Audit logs missing expected entries​

Symptom: Cannot find audit entries for known actions.

Causes and fixes:

  • Date filter too narrow: Expand the date range
  • Action not audited: Not all actions generate audit entries
  • Wrong tenant: Audit logs are tenant-scoped

Engine configuration won't publish​

Symptom: Publish button fails or shows error.

Causes and fixes:

  • Validation errors: Fix all schema validation issues first
  • Version conflict: Another version may already be published
  • Permission denied: Requires C_AI_ENGINEER role

Trace viewer shows no data​

Symptom: Trace details are empty or fail to load.

Causes and fixes:

  • Run has no traces: Older runs may lack trace data
  • Tracing service down: Check Phoenix/OTEL connectivity
  • Run still in progress: Wait for completion

Metrics not updating​

Symptom: Dashboard metrics appear stale.

Causes and fixes:

  • Auto-refresh disabled: Click refresh manually
  • No recent activity: Metrics reflect actual usage
  • Data aggregation delay: Wait a few minutes

Gotchas and edge cases​

  1. Tenant context persists: Your selected tenant remains in your session. Always verify you're in the correct tenant before making changes.

  2. Admin Viewer is read-only: This role can monitor but cannot make changes. Engine Studio write operations require C_AI_ENGINEER.

  3. Environment separation: DEV, STAGING, and PROD are isolated. Configurations activated in DEV don't affect PROD.

  4. Pack immutability after activation: Once any configuration is ACTIVE for a pack+environment, that combination is locked. Create new versions instead of modifying.

  5. Criteria versions are managed: Imported criteria go through a lifecycle (DRAFT → EVALUATING → ACTIVE → SUPERSEDED). Only one version can be ACTIVE per pack.

  6. Audit logs are immutable: Audit entries cannot be modified or deleted. They provide a complete compliance trail.