Curious Neurons
AI research

Small open models, tested in the open.

We fine-tune small open models for one narrow job, make them cheap to run, and look inside them to catch wrong answers. Our first line of work is FinOps: audit notes on GPU sizing and serving cost, built on Qwen3 4B. Every model and dataset here is free to download, from this site or from Hugging Face.

Models and data

Each release has a model card with the full numbers, the hardware it ran on and its limits. The models are under the Apache 2.0 licence, so you can use them in your own work.

Adapter · week 1

FinOps LoRA adapter

A small add-on (LoRA, rank 8) for Qwen3 4B that teaches the same FinOps audit format, trained on an Apple M4 laptop. The card also measures how much the adapter changes each weight matrix at layer 18, using singular value decomposition.

  • 66 MB adapter
  • Needs Qwen/Qwen3-4B
  • Training loss 3.84 → 2.08 in 15 steps
hf download imhlg/qwen3-4b-finops-lora-week1
Experiment · week 2

A check inside the model for made-up claims

A 20 KB file with two parts: a linear probe that reads the model's hidden state at layer 18 and flags a made-up FinOps claim, and a steering vector that nudges the answer back. On an L4 it added about 0.9 ms to a forward pass, compared with roughly half a second for asking a second model to judge.

It scored perfectly (F1 of 1.0) at every layer we tried, on 1,000 claims built from templates. A perfect score usually means true and false claims differ in easy, surface ways, so we treat this as a first step. The next test is on real, untemplated text.

Dataset

FinOps claims and model activations

1,000 short FinOps claims, each labelled true or made up, with the model's layer-18 activations saved beside them (1,000 × 2,560 values), so anyone can train or test a probe without running the model.

  • 1,000 claims
  • 10 MB of activations
  • Parquet and safetensors

How we run this research

The same three habits for every release.

Work with us

Questions about a model, a research idea or a collaboration are all welcome.

requests@cybec.io