Does the Quantum Layer Actually Learn? A Controlled Benchmark for Hybrid QCNNs
A single-file, reproducible experimental pipeline that isolates what the quantum layer
actually contributes in hybrid quantum–classical convolutional networks.
Most hybrid QCNN papers compare one hybrid model against one classical baseline and report
accuracy. That design cannot tell you whether the performance comes from the quantum circuit
learning or from the circuit acting as a fixed random non-linear feature map. This repository
provides the control experiment that separates the two.
Headline result: across 330 training runs on two datasets, the trainable variational quantum
circuit (VQC) ranked last among six models. The quantum block provides no benefit even when
frozen, and training its parameters makes things measurably worse.
Key findings
All comparisons are paired (dataset × training size × seed), tested with the Wilcoxon signed-rank
test, with bootstrap 95% confidence intervals over 10,000 resamples.
Comparison (55 paired runs)
Δ macro-F1
95% CI
p
Cohen's d
Trainable VQC − Frozen VQC
−0.073
[−0.100, −0.046]
5.2×10⁻⁶
−0.71
Trainable VQC − Parameter-matched MLP
−0.099
[−0.130, −0.067]
1.0×10⁻⁶
−0.81
Trainable VQC − Width-preserving MLP
−0.114
[−0.148, −0.080]
2.7×10⁻⁷
−0.88
Trainable VQC − Block-free baseline
−0.109
[−0.138, −0.081]
1.4×10⁻⁸
−0.98
Frozen VQC − Block-free baseline
−0.036
[−0.060, −0.014]
0.018
—
The result is two-level:
The quantum block does not help even with frozen parameters — removing it entirely is better.
Training the block parameters degrades performance further.
Training does help below a certain circuit size. Ablation over qubit count (SLD, N = 100/class):
Qubits
Δ macro-F1 (trainable − frozen)
95% CI
Seeds won
2
+0.057
[+0.016, +0.127]
5/5
4
+0.115
[+0.016, +0.213]
4/5
6
−0.066
[−0.107, −0.034]
0/5
8
−0.051
[−0.100, −0.003]
2/5
Linear interpolation places the sign change at ≈ 5.3 qubits. This tracks the gradient variance,
which decays exponentially with qubit count:
Var[∂⟨Z₀⟩/∂ϑ] ≈ V₀·exp(−βq) with β = 0.613 / 0.656 / 0.664 for L = 2 / 4 / 6 and R² ≥ 0.9997.
Secondary findings
Calibration degrades. Mean ECE 0.218 (trainable) vs 0.159 (frozen), p = 5.4×10⁻⁸. The trainable
circuit is systematically overconfident on its errors.
Cost. The hybrid model trains ≈ 5.3× slower than its classical equivalents while scoring lower.
Not an encoding artefact. Angle, angle + data re-uploading, and IQP encodings all show the same
direction.
Depth is a separate effect. Circuit depth hurts performance (Δ = −0.114 at L = 6) but does not
change gradient variance — this remains unexplained and is stated as an open question.
What makes this a controlled comparison
Five models share the same convolutional backbone, the same bottleneck width, the same
optimiser budget, and the same augmentation. They differ only in one block:
Model
Block
Role
hybrid_vqc
Trainable VQC (54 params)
Main model
ctrl_vqc_frozen
Same circuit, frozen at random init
Control A — does training help?
ctrl_mlp_matched
Linear(d→4)–Tanh–Linear(4→d) (58 params)
Control B — same budget, classical
ctrl_mlp_wide
Linear(d→d)–Tanh–Linear(d→d) (84 params)
Control B2 — no inner compression
ctrl_bottleneck
Identity
Control C — is a block needed at all?
Plus two deliberately unmatched references: classical_full (a normal CNN head) and pure_vqc
(no convolutional backbone, to show the information-bottleneck cost).
Protocol decisions that matter
Each of these corresponds to a common failure mode in the literature:
The test set is never used for model selection. Validation is carved out of the training pool;
the test set is read exactly once, after training completes.
Fixed test set across all runs. Drawn with a fixed seed, identical for every training size and
seed — this is what makes paired statistics legitimate.
Fixed-size validation set (20/class) at every training size, so early stopping is equally
reliable in the low-data regime.
Equal data. All models train on the same N. (Training the classical model on 60k and the hybrid
on 10k invalidates the comparison.)
Equal bottleneck width. Squeezing the quantum model to 6 dimensions while leaving the classical
head at 256 is a structural handicap, not a fair test.
Constant optimiser budget (~1,200 steps) across training sizes.
Residual gate, z ← z + γ·B(z) with learnable γ initialised to 1, so a block can never hurt
merely by existing. The comparison measures what the block adds, not whether adding a block hurts.
5 seeds, paired non-parametric tests, effect sizes and CIs — not single-run point estimates.
Repository contents
Hybrid_QCNN_Controlled_Experiment.ipynb # the full pipeline (setup → experiments → figures)
The notebook is self-contained and runs end to end in Google Colab.
Quick start (Google Colab)
Open the notebook in Colab and select a T4 GPU runtime.
Run Cell 1 (installation), then Runtime → Restart session.
Run Cell 1b — it verifies package versions and executes a 2-qubit sanity circuit.
Run cells 2–13 in order (definitions only).
Run Cell 14 — a 3–5 minute smoke test. Do not skip this.
Run Cell 15 for a runtime estimate, then the experiment stages.
Point RESULTS_DIR at Google Drive. Every finished run is appended to runs.jsonl and
completed runs are skipped on restart, so a dropped Colab session costs you nothing — just re-run
the same cell.
Pinned dependencies
bash
1pip install"pennylane==0.41.1""pennylane-lightning==0.41.1"2pip install"autoray==0.7.2"# must come AFTER pennylane3pip install torch torchvision scikit-learn scipy matplotlib pandas pillow
4pip uninstall -y jax jaxlib # last step
Two pins are not optional:
autoray < 0.8 — PennyLane 0.41 uses autoray.autoray.NumpyMimic, removed in autoray 0.8.0.
Without the pin, import pennylane raises
AttributeError: module 'autoray.autoray' has no attribute 'NumpyMimic'.
Tested boundary: present up to 0.7.2, gone from 0.8.0.
Remove JAX. Colab preinstalls it; PennyLane 0.41 tries to import it and fails on JAX ≥ 0.6.0
(jax.core.Primitive was moved). This work uses the PyTorch interface, so JAX is simply removed
rather than downgraded. Do it last — earlier pip install steps can pull it back in.
Cell 1b checks both conditions and prints the fix if either is violated.
Primary — small enough that the low-data regime is natural, not simulated
MNIST
70,000 images, 10 classes
Calibration baseline
Both go through the identical preprocessing path: grayscale → bilinear resize to 28×28 → scale to [0, 1].
Two gotchas in the SLD loader, both handled in the notebook:
Images are not uniformly sized: 2,059 are 100×100 but three are 3,024×3,024. Each image is
resized before stacking; stacking first raises
ValueError: all input arrays must have the same shape.
The GitHub release has no X.npy/Y.npy (those are in the Kaggle version). Data lives in
Dataset/<digit>/*.JPG, so folder name = true digit label. The loader supports both layouts.
Read the headline result narrowly. What was measured is the behaviour of one architecture family at
one scale.
Noiseless state-vector simulation — no shot noise, no decoherence. Real hardware may differ.
One ansatz family: Rot gates with ring-topology CNOTs (hardware-efficient). Problem-adapted or
symmetry-preserving ansätze may behave differently.
≤ 10 qubits, ≤ 6 layers.
≤ 125 samples/class (SLD), ≤ 500 (MNIST).
No systematic hyperparameter search beyond the quantum learning rate.
This is not evidence that quantum machine learning does not work. It is evidence that, in this
regime, the frozen-circuit control is necessary before any claim about the quantum layer's
contribution can be made.
Related work
The direction is consistent with — and extends — Bowles, Ahmed & Schuld
(arXiv:2403.07059), who benchmarked 12 quantum models
across 160 datasets and found that removing entanglement usually did not degrade performance. Here,
removing the quantum block entirely is shown to be better, under matched conditions on two
independent datasets.
Mechanism follows McClean et al., Barren plateaus in quantum neural network training landscapes
(Nat. Commun. 9:4812).
Citation
bibtex
1@article{qcnn_frozen_control,
2 title = {Does the Quantum Layer Actually Learn in Hybrid Quantum--Classical
3 Convolutional Networks? A Comparative Evaluation Using a Frozen-Circuit Control},
4 author = {Author Name},
5 journal = {Journal of Engineering Sciences and Research},
6 year = {2026},
7 note = {Code and results: https://huggingface.co/<user>/<repo>}
8}
License
MIT for the code in this repository. The Sign Language Digits dataset and MNIST retain their own
original licences.
