Guide
Running an LLM, and mining beside it
Serve a language model from the miner on an OpenAI-compatible API, and keep the card mining whenever nobody is asking it anything.
bzminer can serve a language model. It is not an algorithm and it needs no pool, no wallet and no account: point the miner at a checkpoint and it opens a chat page and an OpenAI-compatible API on a port of its own. The interesting part is that the same card can keep mining Pearl whenever nobody is asking the model anything.
Start it
One flag. --llm_model takes a folder under the models directory, a path, or a
Hugging Face repo id — a repo id that is not there yet is downloaded, resumably.
./bzminer --dmon --llm_model pearl-ai/Llama-3.1-8B-Instruct-pearl
# inference only. --dmon is device-monitor mode:
# metrics and the web UI, nothing mining
./bzminer -a pearl -p <pool> -w <wallet> --llm_model Llama-3.1-8B-Instruct-pearl
# mine Pearl AND serve the model, same card
./bzminer --dmon --llm_enabled # open the port with no model chosen, then pick
# or download one from the page
Open http://127.0.0.1:4020/. The page lists what is in the models folder,
downloads new ones, and gives you somewhere to type.
The API
At the same origin: POST /v1/chat/completions, POST /v1/completions
and GET /v1/models. Set "stream": true and replies arrive as
server-sent events. Any API key is accepted, so anything that already speaks OpenAI
— a client library, an editor plugin, a script — works by changing the base URL
and nothing else.
curl http://127.0.0.1:4020/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"m","messages":[{"role":"user","content":"Say hello"}]}' llm_chat in the archive is a terminal client for a model a rig is already
serving, which saves writing that curl line twice.
The management endpoints are GET /api/llm/status,
GET /api/llm/models, POST /api/llm/load,
POST /api/llm/unload, POST /api/llm/download and
POST /api/llm/download/cancel — the same calls the page makes.
Options
| Option | What it does |
|---|---|
--llm_model <x> | a folder under the models dir, a path, or a Hugging Face repo id |
--llm_enabled | open the port with no model chosen |
--llm_models_dir <dir> | where models live (default models) |
--llm_port <n> / --llm_bind <addr> | the page and API port (default 4020) and address (default: the dashboard’s) |
--llm_device auto|cpu|<n> | where to run. A comma-separated list (0,1) splits one model across cards |
--llm_ctx <n> | context length (default 4096); it sizes the KV cache |
--llm_threads <n> | CPU threads, for the CPU path |
--llm_hf_token <t> | a Hugging Face token, for gated repositories |
--llm_mine <bool> | mine the model’s own matmuls while it infers (default false; see below) |
Every one is also a config.txt key — "llm_models_dir": "D:/models"
— and the command line wins where both set it.
Which checkpoints run
safetensors weights with a tokenizer.json, a llama,
qwen2, qwen3 or gemma3_text architecture, and
compressed-tensors int7/int8 per channel (w8a8) or fp8 (channel, tensor or
block). bf16 runs on the CPU backend only — on a GPU the layers have to be quantised.
GGUF, CTranslate2 and AWQ are not read.
The four are the same block with additions: qwen2 puts a bias on q, k and v;
qwen3 adds a per-head RMSNorm on q and k before RoPE; gemma3_text
adds that same norm, GeGLU, an embedding scale and alternating sliding-window layers with
their own RoPE base. A checkpoint whose weights disagree with its model_type
is refused at load rather than run through the wrong block — the wrong block does not
fail, it answers plausible nonsense.
Dense only. Anything else is refused by name when the config is parsed, before a single weight is read:
| Checkpoint | model_type | |
|---|---|---|
| Qwen3 0.6B – 32B, fp8 or w8a8 | qwen3 | runs |
| Qwen3-30B-A3B and other Qwen3 MoE | qwen3_moe | refused — no expert routing |
| Qwen3-Next (80B-A3B and kin) | qwen3_next | refused — MoE and hybrid linear attention |
| Gemma 3, text-only | gemma3_text | runs |
| Gemma 3, multimodal | gemma3 | refused, pointing at the text-only build |
There is no expert routing in the engine at all, and one attention family: paged KV with grouped-query attention. Qwen3-Next interleaves Gated DeltaNet layers with standard attention, and a linear-attention layer has no KV cache in that sense, so it is a new layer type rather than a setting. Quantisation is not what stops either of them — fp8 per channel is the best-exercised path here.
pearl-ai models are quantised the mineable way; Llama-3.1-8B-Instruct-pearl has 144 such layers, about 67% of the model’s multiply-accumulates.Serving more than one person
By default the server answers one request at a time. BZ_LLM_SLOTS=<n> sizes
the KV cache for n independent sequences, which is the most the batch engine can
step at once. It matters more than it looks: a decode step reads every weight in the
model to produce one token, so n rows read them once for n
tokens instead of once each. Pages are handed out per sequence as it is admitted rather than
reserved per slot, so a large slot count does not cost the full context each.
BZ_WEB_WORKERS=<n> (default 32) is the HTTP server’s concurrency. A
streaming completion holds a worker for the life of the request, so this is the ceiling on
everything above it — slots do nothing if no worker is free to accept the connection.
Mining Pearl on the same card
A GPU serving a language model is idle most of the time on a normal workload, and Pearl can have those gaps. There are two arrangements and they are different work, not two settings of one knob.
Between requests — the default
With --llm_mine false (the default) Pearl mines its own work on the whole card
and stands down for the life of each request, taking the card back the moment it ends. The
LLM tells it when a request is in flight, so there is nothing to configure: run both and it
happens.
Measured on an RTX PRO 6000 Blackwell serving Llama-3.1-8B int7, against a matched control with no Pearl running at every load. That card mines 419.64 TH/s with no model at all:
| Load | Tokens kept | Median request | Hashrate | Of Pearl alone |
|---|---|---|---|---|
| a model loaded, nothing asked of it | – | – | 416.26 TH/s | 99.2% |
| 8 streams, 3 s between turns | 95.8% | +4.6% | 248.87 TH/s | 59.3% |
| 24 streams, 1.5 s between turns | 98.9% | +2.5% | 160.26 TH/s | 38.2% |
| saturated, 160 streams | 100.4% | −0.5% | 0.00 H/s | 0% |
Zero while saturated is the design, not a failure: there are no gaps to mine, and every mechanism tried for inventing them costs more than it earns. An interactive rig is the opposite case — it keeps nearly all of its tokens and still mines a third to three-fifths of the card.
During inference — --llm_mine true
The other mode mines the model’s own int7 matmuls: each linear layer is offered to Pearl before the backend computes it. It sounds like it should be free, and it is not — the product is multiplied once for the model and again, noised, for the search, so it is strictly extra work and costs tokens roughly in proportion to what it earns.
On the same rig it is worse at both ends. Saturated it buys 0.596 TH/s — 0.14% of that card — for 6.5% of the tokens. At 24 streams it mines nothing at all, because a 24-row batch is narrower than Pearl’s 105-row tile pattern and the service refuses every one. Leave it off unless you are measuring it.
Tuning the arrangement
| Variable | What it does |
|---|---|
BZ_PEARL_GEMM_NO_BUSY_GATE=1 | keep mining through requests. For a rig that wants the hashes whatever they cost the tokens |
BZ_PEARL_GEMM_BUSY_HOLDOFF_MS=<n> | how long Pearl keeps standing down after the last request. Follows the measured search launch by default |
BZ_LLM_SM_PARTITION=<n> | give the model n streaming multiprocessors and Pearl the rest, so both run continuously. Worth it only on a rig that is saturated and wants hashrate |
Problems
- “no gemm service - inference only” in the log. The Pearl plugin is not loaded. That is only a problem if you wanted the card mining; inference is unaffected.
- The model loads but the hashrate stays at zero. Check the load: a saturated server has no gaps, and zero is correct. If the card is idle and still not mining, the miner has no pool —
--dmonserves the model without mining anything. - Tokens collapse when Pearl is mining. Check that
BZ_PEARL_GEMM_NO_BUSY_GATEis not set. That variable exists to turn the protection off. - A bf16 checkpoint will not load on a GPU. It cannot — quantise it, or run it with
--llm_device cpu. - A download stops partway. It resumes: files are written as
<name>.partialuntil they are complete, so starting it again continues rather than restarts.