Vacuum ultraviolet/hydrogen peroxide (VUV/H₂O₂) advanced oxidation is a promising polishing strategy for low-total-organic-carbon (TOC) high-purity water systems. However, its practical operation is constrained by coupled trade-offs among TOC removal, residual H₂O₂, and energy demand, often leading to inefficient energy use and excessive oxidant consumption. This study developed a surrogate-assisted reinforcement learning (RL) framework for energy- and oxidant-efficient single-pass VUV/H₂O₂ polishing under low-TOC conditions, using methanol as a neutral low-molecular-weight model organic. A high-fidelity COMSOL multiphysics model was used to generate simulation data, and a deep neural network (DNN) surrogate replaced direct simulations, reducing evaluation time from minutes-scale COMSOL calculations to second-scale prediction and enabling scalable policy learning. Among the evaluated RL algorithms, Soft Actor-Critic (SAC) achieved the best performance under a multi-objective formulation integrating TOC removal, residual H₂O₂, electrical energy per order (EEO), and apparent reaction kinetic performance. The optimized policy identified operating regimes characterized by low energy consumption (EEO ≤ 2.5 kWh m^-3 order^-1), sufficient hydraulic throughput (>79 mL min^-1), and high TOC removal (>71.8 %). Experimental validation confirmed the applicability of the optimized policy, with RL predictions agreeing with measurements within 1.5–3.4 % for TOC removal. Compared with an empirical baseline under identical influent conditions, the optimized operation increased TOC removal by 18.0 percentage points, from 58.3 % to 76.3 %, while reducing EEO by 55.8 %, from 4.39 to 1.94 kWh m^-3 order^-1. These results demonstrate that integrating physics-based modeling, surrogate learning, and RL can support energy-efficient and residual-oxidant-aware operation of VUV/H₂O₂ polishing, providing a scalable pathway toward resource-efficient water treatment.