Guide

Running an LLM, and mining beside it

Serve a language model from the miner on an OpenAI-compatible API, and keep the card mining whenever nobody is asking it anything.

bzminer can serve a language model. It is not an algorithm and it needs no pool, no wallet and no account: point the miner at a checkpoint and it opens a chat page and an OpenAI-compatible API on a port of its own. The interesting part is that the same card can keep mining Pearl whenever nobody is asking the model anything.

Start it

One flag. --llm_model takes a folder under the models directory, a path, or a Hugging Face repo id — a repo id that is not there yet is downloaded, resumably.

./bzminer --dmon --llm_model pearl-ai/Llama-3.1-8B-Instruct-pearl
                                  # inference only. --dmon is device-monitor mode:
                                  # metrics and the web UI, nothing mining

./bzminer -a pearl -p <pool> -w <wallet> --llm_model Llama-3.1-8B-Instruct-pearl
                                  # mine Pearl AND serve the model, same card

./bzminer --dmon --llm_enabled    # open the port with no model chosen, then pick
                                  # or download one from the page

Open http://127.0.0.1:4020/. The page lists what is in the models folder, downloads new ones, and gives you somewhere to type.

The API

At the same origin: POST /v1/chat/completions, POST /v1/completions and GET /v1/models. Set "stream": true and replies arrive as server-sent events. Any API key is accepted, so anything that already speaks OpenAI — a client library, an editor plugin, a script — works by changing the base URL and nothing else.

curl http://127.0.0.1:4020/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"m","messages":[{"role":"user","content":"Say hello"}]}'

llm_chat in the archive is a terminal client for a model a rig is already serving, which saves writing that curl line twice.

The management endpoints are GET /api/llm/status, GET /api/llm/models, POST /api/llm/load, POST /api/llm/unload, POST /api/llm/download and POST /api/llm/download/cancel — the same calls the page makes.

Options

OptionWhat it does
--llm_model <x>a folder under the models dir, a path, or a Hugging Face repo id
--llm_enabledopen the port with no model chosen
--llm_models_dir <dir>where models live (default models)
--llm_port <n> / --llm_bind <addr>the page and API port (default 4020) and address (default: the dashboard’s)
--llm_device auto|cpu|<n>where to run. A comma-separated list (0,1) splits one model across cards
--llm_ctx <n>context length (default 4096); it sizes the KV cache
--llm_threads <n>CPU threads, for the CPU path
--llm_hf_token <t>a Hugging Face token, for gated repositories
--llm_mine <bool>mine the model’s own matmuls while it infers (default false; see below)

Every one is also a config.txt key — "llm_models_dir": "D:/models" — and the command line wins where both set it.

Which checkpoints run

safetensors weights with a tokenizer.json, a llama, qwen2, qwen3 or gemma3_text architecture, and compressed-tensors int7/int8 per channel (w8a8) or fp8 (channel, tensor or block). bf16 runs on the CPU backend only — on a GPU the layers have to be quantised. GGUF, CTranslate2 and AWQ are not read.

The four are the same block with additions: qwen2 puts a bias on q, k and v; qwen3 adds a per-head RMSNorm on q and k before RoPE; gemma3_text adds that same norm, GeGLU, an embedding scale and alternating sliding-window layers with their own RoPE base. A checkpoint whose weights disagree with its model_type is refused at load rather than run through the wrong block — the wrong block does not fail, it answers plausible nonsense.

Dense only. Anything else is refused by name when the config is parsed, before a single weight is read:

Checkpointmodel_type
Qwen3 0.6B – 32B, fp8 or w8a8qwen3runs
Qwen3-30B-A3B and other Qwen3 MoEqwen3_moerefused — no expert routing
Qwen3-Next (80B-A3B and kin)qwen3_nextrefused — MoE and hybrid linear attention
Gemma 3, text-onlygemma3_textruns
Gemma 3, multimodalgemma3refused, pointing at the text-only build

There is no expert routing in the engine at all, and one attention family: paged KV with grouped-query attention. Qwen3-Next interleaves Gated DeltaNet layers with standard attention, and a linear-attention layer has no KV cache in that sense, so it is a new layer type rather than a setting. Quantisation is not what stops either of them — fp8 per channel is the best-exercised path here.

Note
Only int7 layers mine. An int8 or fp8 checkpoint serves inference perfectly well and adds nothing at all to the hashrate. The pearl-ai models are quantised the mineable way; Llama-3.1-8B-Instruct-pearl has 144 such layers, about 67% of the model’s multiply-accumulates.

Serving more than one person

By default the server answers one request at a time. BZ_LLM_SLOTS=<n> sizes the KV cache for n independent sequences, which is the most the batch engine can step at once. It matters more than it looks: a decode step reads every weight in the model to produce one token, so n rows read them once for n tokens instead of once each. Pages are handed out per sequence as it is admitted rather than reserved per slot, so a large slot count does not cost the full context each.

BZ_WEB_WORKERS=<n> (default 32) is the HTTP server’s concurrency. A streaming completion holds a worker for the life of the request, so this is the ceiling on everything above it — slots do nothing if no worker is free to accept the connection.

Mining Pearl on the same card

A GPU serving a language model is idle most of the time on a normal workload, and Pearl can have those gaps. There are two arrangements and they are different work, not two settings of one knob.

Between requests — the default

With --llm_mine false (the default) Pearl mines its own work on the whole card and stands down for the life of each request, taking the card back the moment it ends. The LLM tells it when a request is in flight, so there is nothing to configure: run both and it happens.

Measured on an RTX PRO 6000 Blackwell serving Llama-3.1-8B int7, against a matched control with no Pearl running at every load. That card mines 419.64 TH/s with no model at all:

LoadTokens keptMedian requestHashrateOf Pearl alone
a model loaded, nothing asked of it––416.26 TH/s99.2%
8 streams, 3 s between turns95.8%+4.6%248.87 TH/s59.3%
24 streams, 1.5 s between turns98.9%+2.5%160.26 TH/s38.2%
saturated, 160 streams100.4%−0.5%0.00 H/s0%

Zero while saturated is the design, not a failure: there are no gaps to mine, and every mechanism tried for inventing them costs more than it earns. An interactive rig is the opposite case — it keeps nearly all of its tokens and still mines a third to three-fifths of the card.

Tip
Judge this by the median request time, not the token rate. Below saturation the token rate is set by how fast requests are offered, so “tokens unchanged” cannot tell a working gate from a client that quietly gave up.

During inference — --llm_mine true

The other mode mines the model’s own int7 matmuls: each linear layer is offered to Pearl before the backend computes it. It sounds like it should be free, and it is not — the product is multiplied once for the model and again, noised, for the search, so it is strictly extra work and costs tokens roughly in proportion to what it earns.

On the same rig it is worse at both ends. Saturated it buys 0.596 TH/s — 0.14% of that card — for 6.5% of the tokens. At 24 streams it mines nothing at all, because a 24-row batch is narrower than Pearl’s 105-row tile pattern and the service refuses every one. Leave it off unless you are measuring it.

Tuning the arrangement

VariableWhat it does
BZ_PEARL_GEMM_NO_BUSY_GATE=1keep mining through requests. For a rig that wants the hashes whatever they cost the tokens
BZ_PEARL_GEMM_BUSY_HOLDOFF_MS=<n>how long Pearl keeps standing down after the last request. Follows the measured search launch by default
BZ_LLM_SM_PARTITION=<n>give the model n streaming multiprocessors and Pearl the rest, so both run continuously. Worth it only on a rig that is saturated and wants hashrate

Problems

  • “no gemm service - inference only” in the log. The Pearl plugin is not loaded. That is only a problem if you wanted the card mining; inference is unaffected.
  • The model loads but the hashrate stays at zero. Check the load: a saturated server has no gaps, and zero is correct. If the card is idle and still not mining, the miner has no pool — --dmon serves the model without mining anything.
  • Tokens collapse when Pearl is mining. Check that BZ_PEARL_GEMM_NO_BUSY_GATE is not set. That variable exists to turn the protection off.
  • A bf16 checkpoint will not load on a GPU. It cannot — quantise it, or run it with --llm_device cpu.
  • A download stops partway. It resumes: files are written as <name>.partial until they are complete, so starting it again continues rather than restarts.