FiUni's approach of constructing FIM-derived frozen subspaces to guide low-rank adaptation while matching Fisher principal subspaces enables adaptive determination of whether to reuse knowledge, expand subspaces, or create new ones, dynamically balancing knowledge sharing and task isolation

FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA), while matching the Fisher principal subspace of each incoming batch window with historical subspaces. This enables FiUni to adaptively determine whether to reuse existing knowledge, expand a related subspace, or create a new subspace, dynamically balancing knowledge sharing and task isolation.

http://arxiv.org/abs/2608.27070v1