You can deploy this model and start earning money today!

sheng22213/
genarm-schemea0p0008-eval-repo

Name: genarm-schemea0p0008-eval-repo
Availability: InStock

safetensors

Views

No views yet

SchemeA0.0008 Eval Assets

This repository packages the evaluation code and adapter checkpoints used for the old GenARM original and scheme_a_threshold_0p0008 experiments, so the workflow can be reproduced on another machine.

Included checkpoints

checkpoints/helpfulness/arm_beta_0p5/final_checkpoint
checkpoints/harmlessness/arm_beta_0p01/final_checkpoint
checkpoints/helpfulness/arm_beta_0p5_masked_round4_extreme/final_checkpoint
checkpoints/harmlessness/arm_beta_0p01_masked_round4_extreme/final_checkpoint

Included code

code/evaluation/PKU-SafeRLHF/generate_outputs_Beaver.py
- --force_temperature override support has been removed in this copy.
code/evaluation/PKU-SafeRLHF/gpt4_eval_Beaver.py
code/evaluation/PKU-SafeRLHF/gpt4_eval_Beaver_humanness_single.py
code/pku_safe_rlhf_repro/generate_old_genarm_custom_scheme.py
code/pku_safe_rlhf_repro/gpt4_eval_Beaver_local_shard.py
code/pku_safe_rlhf_repro/gpt4_eval_Beaver_humanness_single_local_shard.py
code/pku_safe_rlhf_repro/helpfulness_scheme_interface.py
code/pku_safe_rlhf_repro/prepare_eval728_without_train_overlap_500.py
code/pku_safe_rlhf_repro/merge_generation_shards.py
code/pku_safe_rlhf_repro/merge_beaver_eval_shards.py
code/pku_safe_rlhf_repro/merge_beaver_humanness_single_shards.py
code/pku_safe_rlhf_repro/run_eval728_without_train_overlap_500_old_genarm_balance.sh
code/language-model-arithmetic/src/model_arithmetic

Included data

data/evaluation_prompts-full-728_without_train_overlap_500.json

Notes on portability

The uploaded generate_old_genarm_custom_scheme.py has been made portable:
- the base model defaults to PKU-Alignment/alpaca-7b-reproduced
- the helpful / harmless adapters default to the checkpoints bundled in this repository
- all three can still be overridden by command-line arguments
The uploaded run_eval728_without_train_overlap_500_old_genarm_balance.sh has been rewritten to use repository-relative paths instead of the original HPC absolute paths.
generate_outputs_Beaver.py in this repository no longer accepts --force_temperature; original generation always follows the script's internal temperature rule.

Suggested setup

Install the Python dependencies you need, then make sure model_arithmetic is on PYTHONPATH.

Example:

bash

pip install -r requirements.txt
export PYTHONPATH="$PWD/code/evaluation/PKU-SafeRLHF:$PWD/code/language-model-arithmetic/src:$PYTHONPATH"

Example: run original generation

bash

cd code/evaluation/PKU-SafeRLHF
python generate_outputs_Beaver.py \
  --model_base_name_or_path PKU-Alignment/alpaca-7b-reproduced \
  --model_arm_helpfulness_name_or_path ../../../checkpoints/helpfulness/arm_beta_0p5/final_checkpoint \
  --model_arm_harmlessness_name_or_path ../../../checkpoints/harmlessness/arm_beta_0p01/final_checkpoint \
  --alpha_helpfulness 0 \
  --alpha_harmlessness 6 \
  --model_name_for_logging alpaca-7b-reproduced \
  --model_arm_helpfulness_name_for_logging arm_beta_0p5 \
  --model_arm_harmlessness_name_for_logging arm_beta_0p01 \
  --datasets ../../../data/evaluation_prompts-full-728_without_train_overlap_500.json \
  --output_dir ../../../runs/example_original \
  --resume False \
  --rewarded_soups False \
  --seed 0

Example: run scheme A generation

bash

cd code/pku_safe_rlhf_repro
python generate_old_genarm_custom_scheme.py \
  --prompt_file ../../data/evaluation_prompts-full-728_without_train_overlap_500.json \
  --output_dir ../../runs/example_scheme_a \
  --model_name_for_logging example_scheme_a \
  --alpha_helpfulness 0 \
  --alpha_harmlessness 6 \
  --base_prob_threshold 0.0008 \
  --seed 0

Example: run the full 500-example old-GenARM evaluation pipeline

This wrapper expects:

a base-model generation file at runs/figure6_eval728_without_train_overlap_500_seed0_fast/models/base/generation.json
a base-model humanness file at runs/figure6_eval728_without_train_overlap_500_seed0_fast/models/base/evaluations_humanness_single_llama3_70b/base.json
a local judge model path in JUDGE_MODEL

Example:

bash

export JUDGE_MODEL=/path/to/Meta-Llama-3-70B-Instruct
export BASE_MODEL_PATH=PKU-Alignment/alpaca-7b-reproduced
bash code/pku_safe_rlhf_repro/run_eval728_without_train_overlap_500_old_genarm_balance.sh 0

Set these before running:

bash

export DECODE_VARIANT=scheme_a_threshold_0p0008
export ALPHA_HELPFULNESS=0
export ALPHA_HARMLESSNESS=6

sheng22213/genarm-schemea0p0008-eval-repo

SchemeA0.0008 Eval Assets

Included checkpoints

Included code

Included data

Notes on portability

Suggested setup

Example: run original generation

Example: run scheme A generation

Example: run the full 500-example old-GenARM evaluation pipeline

sheng22213/
genarm-schemea0p0008-eval-repo