How to build richer post-training data from these sources
Suggested build paths for this project. The source records supply evidence; the environments, trajectories and scoring rules below would be constructed and validated separately.
Interactive environments
Best source material: Linked proposal versions, amendments, procedural rules, dated positions and final decisions. A sequence showing what participants could change is more useful for interaction than an isolated speech. Start with the joined UN cases and acquire their draft/amendment histories.
How to build: Define an initial situation, each participant’s visible information, permitted actions (propose, amend, support, oppose, abstain), a rule for updating the situation and an ending condition. Record observation → action → next observation → outcome. Begin with one decision or amendment round. Implement procedural transitions from documented rules; label simulated diplomatic reactions as assumptions.
Make it useful for training: Include both feasible and infeasible proposals, preserve dated constraints, and keep later outcomes out of earlier observations. Test rules against held-out records before collecting rollouts. Measure valid actions and evidence-consistent behavior; reaching agreement alone is not a sufficient reward, and a historical outcome does not establish a counterfactual transition.
Multi-agent trajectories
Best source material: Chronological, speaker-attributed negotiation or meeting records, with proposal changes and responses. UN meeting records and directed UPR recommendations/responses can anchor roles and tasks; independent country speeches do not themselves form an interactive trajectory.
How to build: Give each agent its own dated observation and role. Log the scenario, acting agent, visible messages, action, tool results, resulting state and outcome at each turn. Retain model/simulator versions and seeds. Distinguish historical observations from generated continuations and explicitly invented private objectives.
Make it useful for training: Review agent turns for supported claims and retained conditions, then use selected turns for supervised training or validated trajectory rewards for RL. Hold out whole scenarios and translation families, test against different partners, and check each participant’s outcomes separately. Preserve legitimate disagreement and prevent agents from seeing information their roles should not have. SOTOPIA provides an interactive social-evaluation reference, not a validated diplomatic simulator.
Reward verifiers and possible RLVR tasks
Best source material: Tasks with a checkable answer: verified final vote labels paired with exact proposals, UPR response labels paired with recommendations, or numeric extraction questions backed by WDI definitions and units. Open-ended claims about national motives have no comparable automatic answer key.
How to build: Define the question, allowed output, reference answer and a deterministic checking function. Validate parsing, label aliases, numeric tolerances, units and missing values. Keep the reference answer outside the prompt. Store the verifier version and per-criterion results alongside generated answers; an ambiguous source should trigger review rather than an invented label.
Make it useful for training: Test the checker on correct answers, plausible mistakes and attempts to exploit it before using rewards in training. Audit accuracy across countries and languages on held-out events. Matching a historical vote verifies that label only, not its rationale or a new scenario’s best action. Only then consider a reward-based trainer such as TRL GRPO; the library accepts reward functions but does not establish their validity.
Preference pairs
Best source material: Substantive authentic Q&A, policy applications and decision explanations with exact supporting passages and review criteria. The current authored examples already provide useful checks for scope, dates, attribution and unsupported commitments.
How to build: Generate two answers to the same country/date/question/evidence prompt. Have a reviewer select the better-supported answer and record the evidence, error type and confidence. Store prompt, chosen, rejected plus a separate review/provenance record. Prefer actual model errors; leave ties or uncertain comparisons unlabeled.
Make it useful for training: Prefer substantive corrections—such as retaining a policy condition or the correct emissions baseline—over superficial style differences. Control for length and fluency, accept multiple valid answers, and keep pairs with their parent event’s split. Test whether training reduces those errors on new cases. TRL DPO supports this paired format without requiring an interactive environment.
Efficient starting point: Review a small set of well-linked decision and policy tasks, measure the evidence-assisted baseline, and collect its actual failures. Those failures can guide preference pairs and narrow verifiers before investing in a larger interactive simulator.