Views
No views yet
| Metric | Value |
|---|---|
| Base Model | openai/gpt-oss-20b |
| Architecture | Mixture-of-Experts Transformer |
| Total Parameters | ~9.6B (pruned from 21B) |
| Original Experts per Layer | 32 |
| Pruned Experts per Layer | 13 |
| Layers | 24 |
| Top-k Routing | 4 |
| Context Length | 128K tokens |
| Attention Heads | 64 (Query), 8 (Key-Value) |
| Residual Dimension | 2880 |
| Attention Pattern | Alternating dense & sliding window (128 tokens) |
| Positional Encoding | RoPE (Rotary Position Embedding) |
| Normalization | RMSNorm |
| Precision | BF16 |
| License | Apache 2.0 |
| Specialization | Science |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4# Load the specialized model on CPU
5model = AutoModelForCausalLM.from_pretrained(
6 "AmanPriyanshu/gpt-oss-9.6b-specialized-science-pruned-moe-only-13-experts",
7 torch_dtype=torch.bfloat16,
8 device_map="cpu",
9 trust_remote_code=True
10)
11tokenizer = AutoTokenizer.from_pretrained("AmanPriyanshu/gpt-oss-9.6b-specialized-science-pruned-moe-only-13-experts")
12
13# Generate with the model
14messages = [
15 {"role": "user", "content": "Explain the process of photosynthesis in plants."}
16]
17
18inputs = tokenizer.apply_chat_template(
19 messages,
20 add_generation_prompt=True,
21 return_tensors="pt",
22 return_dict=True,
23 reasoning_effort="medium"
24)
25
26# Ensure inputs are on the same device as model
27inputs = {k: v.to(model.device) for k, v in inputs.items()}
28
29outputs = model.generate(
30 **inputs,
31 max_new_tokens=512,
32 do_sample=True,
33 temperature=0.1,
34 top_p=0.9,
35 pad_token_id=tokenizer.eos_token_id,
36 eos_token_id=tokenizer.eos_token_id
37)
38
39# Decode only the generated part
40input_length = inputs['input_ids'].shape[1]
41response_tokens = outputs[0][input_length:]
42response = tokenizer.decode(response_tokens, skip_special_tokens=True)
43print(response)1from transformers import AutoModelForCausalLM, AutoTokenizer
2import torch
3
4# Check MPS availability and load model
5device = "mps" if torch.backends.mps.is_available() else "cpu"
6
7model = AutoModelForCausalLM.from_pretrained(
8 "AmanPriyanshu/gpt-oss-9.6b-specialized-science-pruned-moe-only-13-experts",
9 torch_dtype=torch.float16, # Better MPS compatibility
10 device_map=device,
11 trust_remote_code=True,
12 low_cpu_mem_usage=True
13)
14tokenizer = AutoTokenizer.from_pretrained("AmanPriyanshu/gpt-oss-9.6b-specialized-science-pruned-moe-only-13-experts")
15
16# Generate with the model
17messages = [
18 {"role": "user", "content": "Explain the process of photosynthesis in plants."}
19]
20
21inputs = tokenizer.apply_chat_template(
22 messages,
23 add_generation_prompt=True,
24 return_tensors="pt",
25 return_dict=True,
26 reasoning_effort="medium"
27)
28
29# Move inputs to model device
30inputs = {k: v.to(model.device) if isinstance(v, torch.Tensor) else v for k, v in inputs.items()}
31
32# Use torch.no_grad for MPS stability
33with torch.no_grad():
34 outputs = model.generate(
35 **inputs,
36 max_new_tokens=512,
37 do_sample=True,
38 temperature=0.1,
39 top_p=0.9,
40 pad_token_id=tokenizer.eos_token_id,
41 eos_token_id=tokenizer.eos_token_id,
42 use_cache=True
43 )
44
45# Decode only the generated part
46input_length = inputs['input_ids'].shape[1]
47response_tokens = outputs[0][input_length:]
48response = tokenizer.decode(response_tokens, skip_special_tokens=True)
49print(response)1device_map="auto" # Will automatically use GPU if available
2torch_dtype=torch.bfloat16 # or torch.float16<|channel|>analysis<|message|>Need to explain steps, light absorption, electron transfer, Calvin cycle, etc.<|end|><|start|>assistant<|channel|>final<|message|>**Photosynthesis – the “green” chemistry of life**
Plants turn sunlight into chemical energy to grow, reproduce, and maintain their ecosystems. The process is a coordinated dance of light‑energy conversion, electron transfer, and carbon‑fixing chemistry that takes only a few seconds in a leaf but is essential for the biosphere.
---
## 1. Light‑energy capture: the chlorophyll antenna
| **Step** | **What happens** | **Why important** |
|---|---|---|---|
| **Photon absorption** | Chlorophyll‑a and chlorophyll‑b (plus accessory pigments) absorb photons in the blue‑green part of the visible spectrum (≈400–650 nm). | These pigments act as “antennae” that can harvest the energy of photons. |
| **Excite formation** | The absorbed photon promotes an electron from the *ground state* (S₀) to an *excited state* (S₁) of the chlorophyll molecule. | The excited state is short‑lifetime (≈10⁻¹�⁴ s) and must be converted to a usable chemical form. |
| **Charge‑separation** | The excited chlorophyll rapidly undergoes *charge‑separation* (a “photochemical” step). An electron is promoted to the *conhole* (S₁ → S₁) and a hole (S₁ → S₁) is left behind. | This creates a “hot” electron that can be used for a chemical reaction. |
---
## 2. Electron transfer to the *electron‑transport chain*
| **Step** | **What happens** | **Key enzymes** |
|---|---|---|---|
| **Electron transfer** | The hot electron is passed to the *electron‑transport chain* (ETC) embedded in the thromosome (a protein complex). | The ETC is a series of redox reactions that ultimately produce a proton gradient and ATP. |
| **ATP synthesis** | The ETC pumps protons across the *inner* and *outer* membranes of the mitochondoid‑like thromosome, creating a proton motive force (ΔpH ≈ 0.5 mV). | The proton gradient is used to drive the *hydrolysis* of ATP‑generating reactions. |
| **<|channel|>analysis<|message|>Need explain greenhouse effect causes, greenhouse gases, absorption, re-emission, warming.<|end|><|start|>assistant<|channel|>final<|message|>### What is the “greenhouse effect” (or greenhouse effect)?
The greenhouse effect is the warming of the Earth’s surface and atmosphere caused by the trapping of heat that is emitted from the Earth’s surface and from the Sun. It is the same as the greenhouse effect in a laboratory: a container of a gas that absorbs and re‑emits infrared radiation, keeping the temperature higher than it would be if the gas were not present.
---
## 1. What gases are responsible?
The main greenhouse gases (GHGs) are:
| Gas | Source | Effect on the atmosphere |
|------|--------|---------------------------|
| **CO₂** (carbon dioxide) | Fossile‑fuel combustion, methane oxidation, volcanic CO₂ | Strong absorber of IR, 1‑2 % of the Earth’s energy budget |
| **CH₄₄** (methane) | Natural gas, bi‑methane, fossil‑fuel combustion | Strong IR absorber, 1‑2 % of the energy budget |
| **N₂O₄** (nitrous oxide) | Fossile‑fuel combustion, industrial processes | Strong IR absorber, 0.5 % of the energy budget |
| **H₂O** (water vapor) | Evaporised water from the surface | Very strong IR absorber, but it is a natural feedback |
| **C₄H₁₀** (C‑clane) | Fossile‑fuel combustion, industrial processes | Strong IR absorber, 0.5 % of the energy budget |
| **C₂O₂** (CO₂) | Fossile‑fuel combustion, methane oxidation | Strong IR absorber, 1‑2 % of the energy budget |
The other gases (e.g., N₂, O₂, Ar, He) are essentially transparent to IR radiation and do not contribute to the greenhouse effect.
---
## 2. How does the greenhouse effect work?
1. **Solar radiation reaches the Earth**
The Sun emits a broad spectrum of radiation. About 30 % of the solar energy is absorbed by the Earth’s surface and atmosphere. The rest is reflected or transmitted into space.
2. **The Earth emits thermal radiation**
The Earth is in thermal equilibrium: the absorbed solar energy is re‑emitted as a black‑body spectrum (mostly in<|channel|>analysis<|message|>Need to explain DNA structure: double helix, base pairs, sugar, phosphate, nitrogen, etc. Function: genetic code, replication, transcription, translation, regulation, etc.<|end|><|start|>assistant<|channel|>final<|message|>**DNA: the double‑helix of life**
---
## 1. The “structure” of DNA
| **Component** | **What it does** | **Key features** |
|----------------|----------------|----------------|
| **Double helix** | Two strands wind around each way | Right‑handed (R) and left‑handed (L) helices |
| **Base pairs** | 10 pairs per 3.4 nm (≈ 10 nm per 100 bp) | 10 bp = 3.4 nm; 10 bp = 10 nm (≈ 10 nm per 100 bp) |
| **Sugar (deph‑phosph‑deph‑phosph)** | Provides the backbone | 5 × 10⁻¹�⁰ kJ/mol of bond energy |
| **Phosphorus (P)** | Provides the backbone | 5 × 10⁻¹��⁰ kJ/mol of bond energy |
| **Nitrogen (N)** | Provides the backbone | 5 × 10⁻¹��⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹�⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹�⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹�⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹�⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹⁰ kJ/mol of bond energy |
| **Hydrogen (H)** | Provides the backbone | 5 × 10⁻¹⁰ kJ/mol1@misc{priyanshu2025gptoss,
2 title={{GPT-OSS MoE Expert Fingerprinting: Analyzing Expert Activation Patterns in Mixture of Experts Models}},
3 author={Priyanshu, Aman and Vijay, Supriti},
4 year={2025},
5 howpublished={\url{https://amanpriyanshu.github.io/GPT-OSS-MoE-ExpertFingerprinting/}},
6 note={Interactive analysis tool for expert activation patterns in MoE architectures}
7}