Here is a piece of player feedback we wrote to illustrate a point: "The event rewards this time are great, but since the update I keep failing to log in. Can you help me with that first?"
A single sentiment label has to call that message positive or negative, and either answer throws most of it away. The player likes the rewards, has hit a login bug, and wants help now. Praise, a product problem and a request for help each need their own label, because each one goes to a different team.
Jev, a new model from TypeSafe AI, got us asking which judgments inside an agent workflow are small, bounded calls like these, and whether a general-purpose LLM should be the one making them.
What Jev returns
TypeSafe AI describes Jev as a System One model, its term for a model built to make structured decisions that software can use. You give it business context and a clearly defined question, and it answers with one of the values you allowed: an option from a list, a yes or no on whether a condition holds, or a point on a scale you set. Each answer comes with a probability, which is the model's own level of support for that answer.
Because the answer is always one of the values you defined, code can act on it without parsing text. For one piece of feedback you can ask three separate questions (what is the player talking about, what are they unhappy with, are they asking for help) and let code combine the answers. When a message touches several topics, you ask about each one in turn and include "other" or "unsure" among the allowed answers.
How the work splits
In the setup we tested, general-purpose LLMs handle the open-ended work: understanding a complex request, planning an analysis, drafting content and explaining results. Jev handles the narrow judgments that recur and have clearly defined answers. Agentic Engine, the ThinkingAI platform where these agents run, supplies the company's data, business knowledge, Skills (our packaged, reusable agent tasks) and workflow rules, and passes results between the models.
Agentic Engine and deterministic code handle everything that has to be exactly right: calculations, permissions, rate limits and approvals.
The three tests below put that split to work on three different tasks. The samples are small, so each one states its size up front.
Job 1: Tag the feedback without flattening it
Sample: 6 pieces of feedback we prepared, 4 questions each, compared against labels we set in advance.
We asked Jev four yes-or-no questions about each piece of feedback. Does it contain praise? Does it report a product problem? Is the person explicitly asking for help? Does it involve payment? One message can say yes to several.
On the login message above, Jev gave higher scores to the first three and a lower score to "involves payment." That matches the message: the player likes the rewards, hit a login failure, wants help, and never mentions paying. Each score is the model's level of support for that label. To see whether the labels were right, we compared them with the labels we had set in advance.
21 of the 24 judgments matched. The other 3 disagreed, and working through them showed us that some of our label definitions need to be tighter.
We then ran Jev on 2 longer posts from a real player community. One was a player looking for teammates, mostly describing how they like to play and when they are free. Jev tagged it "team-up and social" and returned separate signals for recruiting a squad and a preference for casual play. The other was about an exhibition esports event. The player found the format "kind of fun" but wanted bigger-name players on the roster. Jev returned support scores of 0.99 for mixed sentiment, 0.91 for interest in the format and 0.89 for a roster request, plus 0.75 on a label about team timing and position, which it marked for human review. Treat these 2 posts as examples. They sit outside the 24-judgment comparison above.
The reason to split feedback this way is what an operations team can do next. They can track one topic over time, separate what players like from what they want changed, and combine the labels with the game version, the live events running at the time and what players actually did before deciding whether to follow up.
Job 2: Route players inside a flow
Sample: 8 player messages, 4 possible paths, compared against the path we expected for each.
Segmentation rules can find the players whose activity is dropping or who never finished a key step. Two players in the same segment can still need very different help.
In a test on a win-back flow for a mobile game, we added a step that reads each player's most recent feedback and picks one of four paths we had already defined. A player who needs instructions goes to how-to guidance. A player who hit an error goes to support, where business rules decide whether to compensate. A player who has asked us to stop messaging goes to do-not-disturb. Anything that does not fit the purpose of the flow goes to human review.
We gave Jev the 8 messages and asked for one path each. "I still can't figure out how to use this feature" went to how-to guidance. "It errors out as soon as I get to this step" went to support. "I don't need anything right now, stop sending me messages" went to do-not-disturb. All 8 matched the path we expected. Eight messages is enough to see the routing step behave as designed. How it holds up inside a live flow is part of what we test next.
One message was worth a closer look: "When does next season start? I'm max level and there's nothing to do." It asks about upcoming content, which none of the three automated paths covers. Jev sent it to human review, the path this rule set keeps for requests it does not cover.
In this setup, Agentic Engine's rules decide which players enter the flow, Jev helps decide which path a message fits, and a general-purpose LLM helps explain the strategy and draft the message. Agentic Engine then applies the permissions, rate limits and approvals the company has configured before anything is sent or any compensation is granted. A classification on its own authorizes nothing.
Whether the strategy works has to be measured separately, in clicks, conversions, retention and complaints about too many messages, compared in an experiment. The probability that a message belongs on a path describes the message. Whether sending it down that path lifts conversion is a different number, and only the experiment produces it.
Job 3: Check the question before and after the query
Sample: one question, 4 candidate events, and one check of a written conclusion.
A plain-language data question hides a lot of judgment calls. "How is the paying rate for recent new users?" sounds simple. Does a new user mean someone who signed up, or someone who logged in for the first time? What counts as recent? Is the paying rate measured on day one, or within seven days of sign-up? Each version produces a number, and the numbers mean different things.
We are building a project semantic layer that turns metric definitions, business terms, data sources and usage rules into context an agent can reuse. The test around it ran on inputs we supplied, in two steps: one before the query and one after.
Before the query, we gave Jev 4 candidate events from a real project: account sign-up, app install, onboarding completed and login. It chose account sign-up as the new-user definition for this example, and it flagged that "recent" had no time range and needed to be confirmed with the user. In this example it narrowed the candidates and caught a missing condition. The company's official definition of a new user still has to come from the semantic layer.
After the query, we supplied two weekly results as inputs, 4.2% this week and 4.4% last week, and asked Jev to check one sentence: "The paying rate is about 4.2% and has declined for four straight weeks." Jev kept the 4.2%, which matched the input, and marked the four-week claim as unsupported, because two weeks of data cannot show a four-week trend.
That points to a concrete use. Before the query, help choose among the governed metrics and data sources a user is allowed to see, and ask a follow-up when a condition is missing. After the query, check that the time range, the metric and the written conclusion agree with each other.
One example cannot show that a model checks numbers reliably. The data engine and deterministic code run the query, do the calculations, compare dates and check numeric consistency. Jev checks whether the definition and the wording of the conclusion match what it was given. We want fewer wrong metrics, fewer missing conditions and fewer conclusions that go past the evidence, and it will take many more real questions to know whether this setup delivers that.
What we are taking from this
We tested Jev to see whether its answers could plug into the data, knowledge and workflows a company already runs, with every key step open to inspection and review. The small samples helped us pick the scenarios worth testing at larger scale. They also showed us where our label definitions and edge cases need work. Next we need larger samples from real business work, a better review process, and tests inside live workflows.
Jev answers a defined question. Agentic Engine applies the workflow rules the company configured, and a person reviews any action that requires approval.
If your team is sorting a lot of player feedback, building finer-grained engagement branches, or fighting ambiguous definitions in plain-language data questions, bring one concrete case and we will work through it with you.
