Eval bug: dflash: invalid reduced-logits token -1 in draft

[Riconec]

Name and Version

llama-cli --version
version: 9420 (22d66b5)
built with GNU 14.2.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

Tesla P40

Models

unsloth qwen3.6-27b and Ornstein-Hermes-27b

Problem description & steps to reproduce

When trying to use --spec-type dflash any request generates ? or / char in loop. llama-server log shows
=

First Bad Commit

tried week ago and latest - result the same

Relevant log output

llama-server.txt

Logs

[Anbeeld]

Thanks for the report. A few notes:

• The issue body says version: 9420 (22d66b567).
• The uploaded log appears to be from b9459-07ac3cec6.
• Both are behind current v0.3.0, and there have been significant DFlash changes after those builds around stale drafter sequence clearing, DFlash ring/capture fail-closed behavior, multi-slot isolation, tensor-split/meta capture guards, and scheduler sync before GPU hidden/tape peer copies.

So the first thing I need is a retest with the current v0.3.0 build/HEAD. Please retest with the exact same model pair and include:

1. llama-server --version
2. full server command line
3. full startup log through the DFlash contract / GPU ring lines
4. whether the loop happens on the first request or only after multiple requests
5. whether --parallel is greater than 1

Please also try this isolation matrix, one change at a time if possible:

# baseline: confirm target model is fine without DFlash
--spec-type none

# conservative DFlash path: removes adaptive draft-depth churn and avoids tiny draft ctx
--spec-type dflash \
--spec-draft-model <draft.gguf> \
--spec-draft-n-max 16 \
--spec-branch-budget 0 \
--spec-dflash-cross-ctx 1024 \
--no-spec-dm-adaptive \
-cd 1024 \
--parallel 1

Then, if it still reproduces:

--flash-attn off

--no-fused-gdn

GGML_DFLASH_GPU_RING=0

GGML_DFLASH_KV_CACHE_MODE=off

The P40 detail is useful because it may point at a CUDA/backend-specific path, especially if --flash-attn off or --no-fused-gdn changes behavior. But older reports of this same log line were also caused by DFlash state alignment issues, multi-slot/shared drafter state, stale drafter KV, and multi-GPU capture paths, so I do not want to pin this on P40 until we have a current-build repro.

[Riconec]

that's because I pulled 'main' branch and build on top of that. will try rebuild branch and test. any particular "no-noes" for llama-server run flags?

[Anbeeld]

Any apparent "no-no" would be a nice new report for fixing. :)

[Riconec]

git status
On branch v0.3.0
Your branch is up to date with 'origin/v0.3.0'.

nothing to commit, working tree clean

 llama-cli --version
version: 9420 (22d66b567)
built with GNU 14.2.0 for Linux x86_64

launch script a bit messy because I was trying different settings that might have impact

LLAMA_SERVER_BIN="$HOME/beellama.cpp/build/bin/"
LLAMA_MODEL="/mnt/data/inference/models/qwen3.6-27b-unsloth-q4.gguf"
DRAFT_MODEL="/mnt/data/inference/models/qwen-drafter-q5.gguf"
CTX_SIZE=80000
PARALLEL=1

echo "running llama.cpp"

GGML_DFLASH_KV_CACHE_MODE=${GGML_DFLASH_KV_CACHE_MODE:-off} \
GGML_DFLASH_GPU_RING=${GGML_DFLASH_GPU_RING:-0} \
GGML_DFLASH_FORCE_CPU_CROSS=${GGML_DFLASH_FORCE_CPU_CROSS:-1} \
$LLAMA_SERVER_BIN/llama-server \
  --model $LLAMA_MODEL \
  --alias Ornstein27B \
  --n-gpu-layers 999 \
  --host 0.0.0.0 \
  --port 1235 \
  -c $CTX_SIZE \
  -n -1 \
  -b 512 \
  -ub 128 \
  --no-context-shift \
  --temp 0.7 \
  --top-p 0.9 \
  --top-k 50 \
  --presence-penalty 0.3 \
  --repeat-penalty 1.1 \
  --repeat-last-n 128 \
  --fit on \
  --flash-attn on \
  --no-fused-gdn \
  --ctx-checkpoints 8 \
  --cache-ram 0 \
  --jinja \
  --parallel $PARALLEL \
  --spec-type dflash \
  --spec-dflash-cross-ctx 1024 \
  --no-spec-dm-adaptive \
  --spec-draft-n-max 16 \
  -cd 1024 \
  --spec-draft-model $DRAFT_MODEL \
  --spec-draft-ngl all \
  -ctk q4_0 \
  -ctv q4_0 \
  --reasoning on \
  --timeout 3000

llama-server.txt

[Riconec]

tried with --flash-attn off (and KV f16 instead of q4) and it produced sensible text but server crashed almost instantly:

llama-server.txt

[Anbeeld]

Before debugging the DFlash behavior further, we need to resolve the version mismatch.

Your source checkout says v0.3.0, but the binary still reports:

version: 9420 (22d66b567)

git status only proves the source tree state. It does not prove that server was rebuilt from that source. --version is compile-time build info, so if it still shows 22d66b567, either the script is running an old binary, the build directory reused stale generated build-info, or the build was not reconfigured/rebuilt after switching branches.

Please do a fresh build in a new build directory and run the server from correct path.

Also, please check llama-server --version, not only llama-cli --version, since the issue is from llama-server.

[Riconec]

build was fine, I just called wrong place for 'llama-cli --version'

./llama-cli --version
version: 9997 (fba22014f)
built with GNU 14.2.0 for Linux x86_64

./llama-server --version
version: 9997 (fba22014f)
built with GNU 14.2.0 for Linux x86_64

verified that server loads with 9997 version

[Anbeeld]

I pushed a fix that hardens the two concrete failure surfaces visible from the logs:

• CUDA reduced-logits argmax no longer dereferences rowx[-1] when a reduced-logits row has no valid candidate and returns token -1.
• DFlash now treats repeated invalid reduced-logits drafts as a request-local fail-closed condition and logs ring/cross-state context before disabling DFlash drafting for that request.
• DFlash recurrent replay now validates GDN state shape and tensor byte spans before building ggml_view_* tensors, so the ggml_view_1d assert path should no longer be reachable from bad/mismatched replay state.

Local smoke on RTX 3090 passed for no-spec, MTP, normal DFlash, and DFlash with the conservative issue-style toggles (GGML_DFLASH_FORCE_CPU_CROSS=1, GGML_DFLASH_GPU_RING=0, GGML_DFLASH_KV_CACHE_MODE=off, --no-fused-gdn).

Please retest from the new commit/build. If anything still reproduces, please attach one bundle with:

1. Exact binary path and output of ./llama-server --version from that same path.
2. Full llama-server command line.
3. All relevant env vars, especially GGML_DFLASH_*.
4. Target and draft model filenames, quantizations, and either sha256sum or file sizes.
5. Request body/prompt, n_predict, temperature, seed if fixed, --ctx-size, --parallel, and --ctx-checkpoints.
6. Full stderr from startup through the first failed request.
7. Whether each of these reproduces:
   • default DFlash
   • --flash-attn off with f16 KV
   • conservative flags: CPU cross on, GPU ring off, DFlash KV off, no fused GDN
   • each conservative toggle changed back one at a time, if you have time

The new diagnostic line is the important one if it still happens: it includes ring_filled, ring_write_pos, cpu_ring_valid, score, and invalid_reduced_logits_streak.