Theoretical foundations
Past the empirical outcomes, our sleep paradigm is grounded in a idea that clarifies why its two levels (reminiscence consolidation and dreaming) can assist continuous studying with out merely buying and selling new information for previous capabilities.
For reminiscence consolidation, the evaluation exhibits that information seeding may be understood as a structured, low-rank compression drawback. The mannequin is just not requested to repeat each element of its short-term traces into its parameters. As a substitute, it identifies a compact set of instructions that greatest protect each the mannequin’s personal on-policy conduct and the semantic qualities rewarded by the training goal.
This angle explains the complementary roles of the 2 indicators: on-policy distillation aligns consolidation with the trajectories the mannequin will truly produce at inference time, decreasing the mismatch that may come up when coaching solely on teacher-generated distributions; in the meantime, the semantic reward modifications what’s price retaining, prioritizing instructions that protect that means and task-relevant conduct fairly than merely matching token possibilities. Parameter growth then supplies the capability wanted to write down these chosen reminiscences with out overwriting beforehand consolidated information.
For dreaming, the idea characterizes random knowledgeable activation and reward-based pattern choice as mechanisms for enhancing the geometry of studying. Injecting totally different MoEs broadens the set of gradient instructions explored throughout artificial era, exposing the mannequin to a richer assortment of updates than these induced by its abnormal wake-time knowledge alone. Gradient-based filtering then emphasizes the artificial experiences that carry probably the most helpful studying sign, strengthening informative instructions whereas preserving the construction of the underlying gradient area.
Collectively, these steps yield a extra balanced optimization panorama: studying is much less dominated by a small variety of instructions, gradients are higher conditioned, and the mannequin could make extra steady progress when refining its capabilities. On this view, dreaming is just not merely self-generated knowledge augmentation; it’s a deliberate course of for diversifying, deciding on, and consolidating the updates via which the mannequin improves itself.
