Relational Transformers can play Doom too.

Try it yourself: code and tuning recipes. The minimal repo automatically downloads the pretrained model and includes the original demonstration data for reproducing the tuning. No trained weights are bundled.

Could a model built for prediction over relational data control a game? We adapted the Dasein Labs Doom demo, replaced its action scorer with RT-J, and fine-tuned the existing model. It learned to aim, shoot, and move. Some of that behavior carried over to scenarios it had never seen during our Doom fine-tuning.

The most interesting result came from training on one full level and testing on another. On five matched MAP02 runs, fine-tuning only on MAP01 increased average kills from 4.0 to 6.4 and average spatial coverage from 4.4 to 32 cells. It still did not finish the level. Both facts matter.

What does this have to do with Jev?

TypeSafe describes Jev as a System One model: send a state and typed questions, receive decisions and probability distributions. Its API offers Choice, Score, and Noul, a yes/no probability. Questions are evaluated in parallel against the shared state. TypeSafe documentation.

TypeSafe says Jev uses a new architecture, parallel sampling, and Reinforcement Learning for Calibrated Decisions (RLCD). Its launch post also explicitly says the Doom demonstration reads structured state with text, rather than images. Those are TypeSafe’s descriptions, not properties we independently evaluated. Jev launch post.

The similarly named daseinlabs/open-jev repository is a separate implementation. Its default scorer uses Gemma 3 4B through MLX: encode the context once, batch the supplied options, and rank their conditional token log-probabilities without generating a response. Its Doom adapter turns visible-object labels and depth into text. We reused that game-wrapper approach. open-jev repository.

The common idea is useful: put the available actions in the request and score them. Our distinguishing choice is to keep game facts as typed, related records and use the relational model’s existing prediction machinery. The comparison above describes interfaces and implementations; it is not a gameplay leaderboard. We did not run TypeSafe Jev or Gemma against RT-J on matched seeds, and cannot claim a win on skill, latency, cost, or calibration.

There is also no universal “Jev architecture” established by these open recreations. Dasein’s repository separately documents a frozen-Gemma feature extractor with a trainable scoring head. That is one adaptation route; our experiment updates the existing RT model and adds no head. Dasein’s fine-tuning design.

A game state becomes a small relational context

We start from RelativeDB’s fp16 conversion of Stanford STAR’s pretrained RT-J. At each decision, the adapter builds three kinds of records: an observation, the visible actors, and a candidate action. Actor and action records point to the observation. Health, ammunition, depth, actor position, and actor size remain numeric; names and action descriptions use a frozen MiniLM-L12-v2 text encoder.

Observation: health, ammo, depth, previous action
    ↑ actor → name, kind, horizontal position, size
    ↑ candidate → action text, appropriate action = ?

Native RT-J score for each candidate → argmax → game input

The target is a masked boolean field: is this an appropriate action? We evaluate each legal candidate with the original decoder and choose the highest score. A different action menu does not require a new output layer. During normal gameplay, the model chooses every action; there is no scripted policy taking over when it gets stuck.

The model reads engine-provided labels and depth, not raw pixels. The videos show the world the agent acts in, but they are not its visual input. It sees up to six visible actors and the previous action, without a persistent map or a long observation history. The displayed softmax values are relative action scores; we have not established that they are calibrated probabilities of success.

Fine-tuning the model we already had

Every round continued from a pretrained or previously fine-tuned checkpoint. The lineage was RT-J → arena training → Deadly Corridor → MAP01. We never restarted from random weights. We also never attached a new policy head: the existing RT parameters, including its native output machinery, were updated while the separate text encoder stayed frozen.

Training used behavior cloning. A small scripted demonstrator labeled game observations, including observations collected while the model itself played. We trained each candidate against a positive or negative “appropriate action” label using binary cross-entropy. Left/right mirroring supplied additional examples. This teaches the demonstrator’s local preferences; it does not directly optimize long-term game reward.

The final MAP01 round used 1,200 observations, expanded to 2,400 by mirroring, for two epochs at a learning rate of 10⁻⁵. It started from corridor-v1 and saved map01-v1 separately. The parameter names and shapes matched before and after; 394 native tensors changed and zero heads were added.

Combat improved, but the corridor was not solved

Moving from the arena to Deadly Corridor exposed a weakness: the arena policy barely advanced. After fine-tuning on 1,800 corridor observations, the same ten fresh seeds produced a large improvement in native scenario reward and more kills.

Reward is not a win rate. Across those ten tuned runs there were six deaths, four timeouts, and zero completions. In the first two runs the agent killed six enemies and approached the final armor, then oscillated nearby until time expired. Another run kept firing with no ammunition. The policy had learned useful combat and movement, but not a reliable way to finish.

We also returned the unchanged corridor checkpoint to the original arena. Across 20 matched seeds it averaged 10.15 kills, compared with 10.30 for its arena parent and 1.90 for random actions. That shows similar retained combat performance. Since both checkpoints had arena training in their history, it is a retention test, not unseen-scenario generalization.

A different combat scenario

Next we froze the corridor checkpoint and tested basic, a scenario absent from our Doom training data. It requires strafing into alignment with a stationary enemy and shooting. The model succeeded on all 30 fresh seeds, versus 18 of 30 for random actions. It used both left and right strafes, even though those actions had not appeared in the arena or corridor training menus.

This is evidence of a reusable local aiming behavior, including a different movement action. It does not isolate an improvement caused by corridor tuning: the earlier arena checkpoint also succeeded on all 30 seeds. And a stationary-enemy task is much narrower than navigating a full level.

Train on MAP01, evaluate on MAP02

The stronger transfer check used two full maps from the bundled Freedoom II assets. We added USE to the level action menu for doors and switches, for both the before and after evaluations. All new training observations came from MAP01. MAP02 contributed no training examples, its name was not a model input, and we did no further tuning after seeing its results.

Each episode allowed up to 600 decisions of five game tics, about 86 seconds of game time, at skill 2. Coverage counts distinct 64 × 64 world-unit cells visited; these are not rooms or a percentage of the map. World coordinates were logged only for evaluation and never passed to RT-J. Parent coverage was recovered by replaying saved actions and verifying the original terminal outcomes.

The new checkpoint moved farther through the held-out map and killed more enemies. That is encouraging evidence of cross-level transfer. It also died once where the parent survived, and neither finished a level within the budget. Five seeds on one held-out map support a narrow conclusion: local navigation and combat behavior transferred to this map, while reliable level completion remains unproven.

Why this can work

The hypothesis is that many immediate Doom decisions fit the prediction problem well. Enemy left of the crosshair, ammunition available, wall ahead: those facts help determine whether turning, firing, or moving is useful. Representing the enemy as an actor linked to the current observation gives the model the information needed to score a candidate. Repeated supervised examples teach that mapping.

The transfer results are consistent with learning some of those local relationships rather than only a single route. They do not prove that relational structure itself caused the gain: we did not compare against an otherwise identical model with flattened inputs or removed relations. Nor do they show that the base checkpoint could play zero-shot. These are results from task-specific fine-tuning.

What we demonstrated is concrete: an existing relational prediction model can drive a game through its original output head, retain useful combat behavior, and transfer some of it to a new scenario and a different level. Longer-horizon planning, ammunition management, and reaching the exit are still open problems.

Experiment records and sources

The per-episode evaluation summaries include the corridor, arena-retention, basic, and MAP01/MAP02 comparisons. Videos show consecutive evaluation footage rather than selected successes. The MAP02 clip was reconstructed from the saved action trace; all 140 pre-action observations matched the original recording. Method and media provenance records seeds, training settings, and clip selection.

Our game wrapper and terminal renderer were adapted from Dasein Labs’ open-jev Doom demo. The model uses the Relational Transformers implementation and RT-J checkpoint. Jev comparisons use TypeSafe’s official documentation and launch post, reviewed September 20, 2026.