Customer story How SEGA sped up live-ops decisions for P5X

Read the story
ThinkingAI Logo
Back to Product Updates
AgentsOctober 30, 2026

Test Agents in batches before launch, and review knowledge base updates before they go live

With the AI Evaluation Platform, you can test Agents and models in batches against your own criteria. Knowledge base administrators can also require approval before updates to a company knowledge base go live. The AE CLI adds community evidence tools, version-pinned retrieval and Parquet ingestion.

An evaluation board of result tiles with two failed tiles marked and a repeat arrow for scheduled runs, joined to a knowledge base book where an update waits at a checkpoint with an approval check.

You can now batch-test Agent quality before you ship a change, and review knowledge base updates before they affect answers. A new preset Agent analyzes community feedback and provides evidence from the original sources. The AE CLI adds community evidence tools, version-pinned knowledge retrieval, Parquet ingestion, and after a self-update it runs your original command once.

ModuleChangeWhere you'll see it
Agent managementAI Evaluation PlatformAgent evaluation
Community Insights Agent presetAE Agent
Thinking depth per conversation or automationConversation and automation settings
Lark Docs, Sheets and Bitable as sourcesKnowledge base sources
Version-pinned knowledge base retrievalAE CLI
Project knowledge base targeting and export versionAE CLI
Commands and Skills for your enabled business domainsAE CLI
AnalyticsAnalysis Q&A reviewAE CLI
Community spaces, sources and evidence in batchesAE CLI
Topic and tag filters for post searchesAE CLI
Data foundationParquet file ingestionAE CLI
GovernanceApproval for company knowledge base updatesKnowledge base settings
Knowledge base publication previewAE CLI
Asset recommendation trackingAE CLI

Agent management

AI Evaluation Platform. You can batch-test Agents, large language models or existing answers with your own datasets and scoring criteria, on demand or hourly, daily, weekly or monthly. Results show scores with explanations, low-scoring cases and failure reasons, and run snapshots are kept so you can review and export the results. Use it to test customer service, knowledge Q&A and business assistant Agents with typical questions and edge cases, then retest with the same dataset and criteria after a change. You can also import past Q&A and score it consistently for accuracy, completeness and compliance.

A dataset, a model and an Agent feed one scoring step with a scheduled run below it, and the results fan out with one low-scoring case highlighted for review.

Community Insights Agent. You can use a new preset Agent to query and analyze community, social media and chat data. It provides original-source evidence, a sentiment assessment and comparisons with competitors. It is available when the community insight service is enabled and you have been granted access.

Thinking depth. You can choose how deeply the model reasons, from simple Q&A to complex analysis. In Standard Claude conversations and automations, you can pick from five levels, Adaptive, Low, Medium, High and Max. The setting is saved for each conversation and separately for each automation task.

Lark Docs, Sheets and Bitable as knowledge sources. Knowledge bases can now sync content your team can access in Lark Docs, Sheets and Bitable. You can run full or incremental updates, including additions, edits, deletions and renames.

Version-pinned knowledge base retrieval. The AE CLI is the command-line client for Agentic Engine. In it, you can pin a published knowledge base version for a series of retrievals and check which version an answer originally cited. The CLI always applies your current permissions to these retrievals.

Project knowledge base targeting and export version. kb project-target get finds and reuses your project's semantic knowledge base, which helps you avoid maintaining another project's knowledge base by mistake. Resource package export now takes --view working or --view published. You can migrate a knowledge base that is still in progress or deliver the published version. The default is working.

Commands and Skills for your enabled business domains. Skills are reusable task methods and workflows that Agents can use. The local CLI shows the commands and companion Skills for the business domains enabled in your environment. After a new domain is enabled, run ae-cli config refresh and start a new Agent conversation.

Analytics

Analysis Q&A review. Analysis leads with project configuration permissions can view collected analysis Q&A, associated feedback, related command details and response times. An Agent can then review those Q&A records for repeated questions, errors and omissions.

Community spaces, sources and evidence in batches. The CLI lists your available community spaces and data sources, so you can confirm the right space, channel and source before you start an analysis. You can also retrieve source text from up to 20 posts or videos per batch, with deduplication, length limits and a status for each item.

Topic and tag filters for post searches. You can narrow post searches to specific topics. You can also filter them by user tags.

Data foundation

Parquet file ingestion. The CLI loads Parquet files locally and preserves long integers, decimals, nested fields and time semantics. Warehouse exports need less conversion as a result.

Governance

Approval for company knowledge base updates. Knowledge base administrators can review updates before they go live, so unconfirmed content stays out of answers. Administrators turn on approval separately for each knowledge base, and it is off by default. Once it is on, updates take effect only after approval, and the previous version stays available while a review is pending. Administrators can review and filter changes to sources, Wiki pages and indexes, record a reason when they reject a change, and compare content between historical versions.

Knowledge base publication preview. From the CLI you can check a knowledge base's publication status and candidate versions. You can filter changes to sources, Wiki pages and indexes, and page through the results.

Asset recommendation tracking. In the CLI, you can track each asset recommendation from material export through submission for review, with completed, failed and canceled statuses. Recommendations are grouped by business domain and subtopic, each with a short explanation of its method and risk.

Other improvements

  • Administrators can see cache prices for company models. Prices are shown per unit for cache reads and cache writes.
  • When you leave out a knowledge base name, the Agent searches across your business scope, your space and the sources you use most. There is no limit on how many knowledge bases it uses.
  • Project knowledge bases are matched by permission, with clear messages about conflicts. Skills you turned off stay off.
  • You can review the affected tasks before you enable or disable a Team. You can also pause linked automations and notify the task creators.
  • In shared spaces, creators manage their own automations. Owners and administrators manage everyone's automations.
  • Slack receives final results only. Long reports are split into readable messages that keep headings, tables and code where possible.
  • After the CLI updates itself, your original command runs once and returns its result and exit code. If your scripts rerun commands after a sync, remove that step. Retry only when AE_CLI_VERSION_SYNCED says the command has yet to run.
  • When you install Skills, your own changes and unknown files are preserved. Existing Skills are also preserved if the network fails.
  • The CLI keeps every digit of long community IDs. It also validates inputs before running a community query.
  • Before data ingestion, the assistant asks you to confirm direct upload or file delivery through LogBus2, a log synchronization tool, to help you avoid duplicate data.

Questions

Questions about anything in this update? Contact your account team. This update covers the release notes dated October 30, 2026.