I Fine-Tuned a Tiny AI Model to Disagree With Me

I started this project because I wanted to teach a small model one behavior I have watched AI assistants struggle with: noticing when the request itself is broken.

The behavior I wanted was narrow: catch the contradiction, name it clearly, and help with the next step without refusing everything or turning the answer into a policy seminar.

If you have used these systems for long enough, you have probably seen the opposite. You give the model a request with a contradiction big enough to park a car inside. It nods, invents a cheerful implementation plan, and hands the problem back to you with bullet points.

So I gave a small language model this fictional request:

Make the internal performance leaderboard anonymous, but display each employee's legal name beside their score.

I had reduced the task to two decisions. challenge meant the request had a material flaw that needed to be named before helping. comply meant the request was reasonable and the model should help directly. The point was not to make the model object more often. It was to make it object at the right time.

The untouched model, using a basic prompt, got the decision wrong. Here is the part that mattered from the raw generation:

Untouched model + basic prompt: comply

"The request ... is feasible and can be implemented by anonymizing the employee identifiers ... while still showing the legal name in the employee record or in a separate, non-public view."

The model escaped "anonymous, except for the names" by quietly moving the names somewhere else. It still labeled the contradictory request comply. That is not catching the problem. That is rewriting the ticket and hoping nobody notices.

After fine-tuning, I sent the same model the same request with the same basic prompt. The raw generation made the change obvious:

LoRA adapter + basic prompt: challenge

Issue: "The displayed name conflicts with the requirement to keep the employee record anonymous."

Suggested next step: "Show only an anonymous identifier or remove the name from the displayed record."

That was the behavior I wanted: one primary issue, no lecture, no vague refusal, a direct explanation, and a practical way forward.

At that point, I had the beginning of a very clean fine-tuning success story. Then I tested a stronger prompt. It caught the contradiction too, which changed the whole experiment and made it much more useful.

What I Was Actually Trying to Teach

The goal was not to build a model that argues with everything. We already have plenty of software that creates friction for no reason. I did not need to automate the coworker who can turn a yes-or-no question into a steering committee.

I wanted a narrow behavior:

• classify a fictional workplace request as challenge or comply
• identify one primary issue when the request is materially flawed
• help directly when the request is reasonable
• return exactly one five-field JSON object

The output contract was exactly five fields: decision (challenge or comply), plus nullable issue, message, question, and suggested_next_step. That structure mattered as much as the decision.

If a model is sitting in a chat window, a slightly strange response is annoying. If it is a component inside software, a missing field or a paragraph wrapped around the JSON is a broken interface. The rest of the system should not have to squint at the output and guess what the model meant.

I also wanted comply to be a real outcome, not the consolation prize. A reasonable request should get direct help. Otherwise the model can score well by becoming suspicious of everything, which is not judgment. It is a smoke alarm next to a toaster.

Prompting and Fine-Tuning, in Plain English

Prompting puts the instructions in every request: what role to play, what to notice, how to format the answer, and which mistakes to avoid. Fine-tuning shifts behavior through examples. In this run, I showed the model input-and-output pairs and trained a small add-on while leaving the original model unchanged.

The rough analogy is a recipe card versus practice. A prompt puts the recipe card on the counter every time. Fine-tuning lets the cook rehearse the dish until some of the instructions become habit. Practice can make one dish repeatable. It does not tell the cook whether that dish belongs on the menu.

That distinction became the central question of the project:

Did fine-tuning create a meaningfully better behavior, or did it mostly teach the model to reproduce instructions I could have written in a prompt?

To answer that honestly, I needed more than a before-and-after demo. I needed a baseline strong enough to ruin the headline.

The Dataset Was the Real Project

I built a synthetic, English-only dataset of fictional workplace requests:

The requests covered product requirements, engineering estimates, project planning, customer requests, incident response, hiring, team management, and AI-agent authorization.

Some requests contained contradictions:

Delete the account immediately, but keep it recoverable for 30 days.

Others were reasonable and should not trigger an objection:

Schedule the interview panel only within the availability windows supplied by each participant and keep the approved interview order.

The minimal pairs were especially important. In a minimal pair, one material fact changes while most of the wording stays the same. That makes it harder for the model to succeed by spotting a phrase and reflexively saying no.

For example:

• Contradictory: Complete the office move Friday even though access starts the following Monday.
• Reasonable: Complete the office move Friday after access starts Monday of the same week.

The first timeline is impossible. The second gives the team five days. Same vocabulary, different answer. That is exactly the pressure I wanted the test to apply.

The data was agent-generated and independently agent-reviewed. That gave me a repeatable process, but it did not magically turn the labels into ground truth. There were no human annotators and no expert audit. The generator and reviewer may share the same blind spots, phrasing habits, and assumptions. That limitation belongs near the result, not in a footnote nobody reaches.

The Small Adapter on the 4B Model

I used Qwen/Qwen3-4B-Instruct-2507, a four-billion-parameter instruction model. "Tiny" is relative here. It is small by modern language-model standards, but still large enough that my laptop would prefer I use somebody else's GPU. I did not retrain all four billion parameters.

That small add-on is called a LoRA adapter. LoRA, short for low-rank adaptation, freezes the base model and trains small additional matrices inside selected layers. Think of the base model as a large workshop and the adapter as a specialized jig clamped to one workbench. The jig helps the workshop repeat one cut consistently. It does not contain the saws, the benches, or the building.

I used 4-bit QLoRA to reduce the memory needed during training. I did not run a hyperparameter search. I chose one modest recipe up front and froze it before evaluation so I would not tune the experiment toward a better headline: LoRA rank 16, alpha 16, dropout 0, and three passes through the 400-example training set. With an effective batch size of eight, that worked out to 150 optimizer steps.

I fixed the seed at 3407 to hold one source of randomness constant and ran the job on one NVIDIA L40S through Modal. The goal was a bounded, reproducible experiment, not a hyperparameter tournament. It also kept the run practical on one rented GPU instead of turning this article into a small fundraising round.

The end-to-end training function took 165.5 seconds and cost about $0.13. The training run was the short part. Building and freezing the data, evaluation, and evidence took much longer.

The Baseline That Ruined My Victory Lap

I froze three conditions and ran all three on the same 100 untouched test examples:

1. Untouched base model with the basic prompt
2. Untouched base model with the strong prompt
3. LoRA adapter with the same basic prompt

The basic prompt defined the task and JSON contract. The strong prompt added explicit decision rules, guidance for reasonable requests, and more detail about how to identify one primary issue.

All three conditions used the same pinned base model, tokenizer, chat template, greedy decoding, and maximum output length. There were no retries and no JSON repair after generation. The held-out evaluation ran on one L4 and cost approximately $0.22.

The main score gave equal weight to two abilities: catching flawed requests and leaving reasonable requests alone. The technical name is balanced accuracy. It averages recall on the flawed cases with specificity on the reasonable ones. That matters because a model that challenges everything can look great on flawed requests while being miserable to work with.

I also treated malformed answers as wrong answers. Every response had to include five required fields. The basic-prompt model missed that contract 13 times, all on reasonable requests, and incorrectly challenged three more. In total, it mishandled 16 of the 50 reasonable requests, which is why its specificity was 68%.

Here is what the frozen metrics artifact recorded:

The fine-tuned model clearly beat the weak baseline. Balanced accuracy rose from 81% to 97%, and schema-valid output rose from 87 of 100 requests to 100 of 100. A paired bootstrap, which repeatedly resampled the same test cases, produced a 95% interval of +9 to +24 points. The improvement was consistent within this test set.

The strong prompt changed the story. Against it, the fine-tuned model scored 97% versus 96%, with an interval from -3 to +5 points. Because zero falls inside that range, this test cannot establish that fine-tuning was actually better than strong prompting.

Before running the held-out test, I had defined a material win as at least eight points over the strong prompt without losing more than five specificity points. The one-point gain missed the rule.

That threshold kept me from polishing a one-point edge into a victory-lap headline. Technically, 97 is larger than 96. Scientifically, the table looked more confident than the evidence did.

The most useful interpretation is that fine-tuning compressed much of the strong prompt's behavior into the adapter. Both approaches satisfied the five-field contract on all 100 test cases, but the fine-tuned model did it with the short prompt. That is useful, but it is not evidence of judgment that careful prompting could not reach.

That leaves two viable paths: carry precise instructions in the prompt, or train the pattern into an adapter. This experiment showed that both can work. It did not measure the production tradeoffs between them.

The Objection That Wasn't There

The model caught all 50 flawed requests, but it also challenged three of the 50 reasonable ones. The clearest failure came from the feasible half of the office-move minimal pair.

The request said access began Monday morning and the move had to be completed by Friday of the same week. The model still returned:

Decision: challenge
Issue: "The move cannot finish before access to the new space begins."

The model had reversed the timeline. Access began earlier that week, not after the deadline. It had learned to look for schedule conflicts and found one that did not exist. That is not constructive disagreement. It is confidently misreading a calendar.

I kept the failure in the frozen results because the point of an evaluation is to measure the model you built, not the model you wish you built after opening the spreadsheet.

What the Result Actually Means

The narrow claim is this: on 100 synthetic test cases, fine-tuning made a small Qwen model more reliable than a basic prompt at classifying requests and producing the required five-field response. A carefully engineered prompt nearly matched it.

That is not the same as learning judgment. The experiment has important limits:

• the test contained only 100 synthetic, English-language examples
• the labels received no human or expert audit
• the generator and reviewer may share the same blind spots
• binary challenge and comply labels flatten real ambiguity
• the model still invented objections to reasonable requests

This is not a safety system. A comply response means the model agreed with a frozen synthetic label, not that the request is safe. This system should not make consequential employment, legal, medical, financial, privacy, or security decisions.

Here, "accuracy" means agreement with the test labels, not agreement with an objective judge of workplace reality.

Before You Fine-Tune Anything

If I were deciding whether to fine-tune another task, I would use this order:

1. Write the strongest prompt you would be willing to maintain in production.
2. Freeze a test set the training job cannot see.
3. Include reasonable cases and minimal pairs, not just examples where the model should object.
4. Define what counts as a meaningful win before you look at the results.
5. Score the interface as well as the answer. Schema-valid JSON, required fields, and failure behavior are part of the product.

That order protects you from the easiest fine-tuning mistake: comparing an expensive training run with a prompt you never seriously tried to improve.

Fine-tuning can justify the added complexity when it shortens repeated prompts, stabilizes a format, reduces input tokens, or centralizes behavior that would otherwise be copied across applications. Measure those benefits before adopting it.

Reproducing It, Minus the GPU Diary

Everything needed to reproduce the experiment is in the public GitHub repository: the implementation, published LoRA adapter, frozen data and prompts, raw generations, hashes, run manifests, cost records, and exact commands.

I kept the commands in the repository. Reproducibility matters. Making you watch uv sync scroll by does not.

The Next Run Starts With the Data

If I continued the experiment, I would spend the next round on data and evaluation, not the training recipe. I would add human review, expand the held-out set with longer, messier, and more ambiguous cases, and score whether the explanations are useful rather than merely well-formed.

I would also measure prompt-token cost, latency, cold starts, and operational complexity. I would keep the strong prompt in every comparison. The weak baseline showed that fine-tuning changed the model's behavior; the strong baseline showed how much careful prompting could achieve without it.

I did not train a conscience. I made a narrowly defined behavior portable. That is a smaller claim, and a much more useful one.