Abstract: We propose a mural restoration algorithm based on semantic inference and dynamic joint learning to address the issues of semantic inconsistency and disordered structural textures caused by the lack of semantic constraints and isolated restoration of texture structure in existing deep learning methods for mural restoration. Firstly, a mural restoration framework based on joint learning is constructed, and a joint hierarchical network is designed to divide the mural into high-level semantics and low-level semantics, enabling hierarchical restoration of different semantics. Then, a joint global generation module is designed to model the global semantics of the mural through autoregressive modeling and infer the repaired global semantic information. Next, a joint local generation module is constructed, which introduces a context aggregation block to learn the contextual information of the mural and generate local information for the mural. Finally, a joint attention mechanism is introduced to enable collaborative training between the global semantic restoration module and the local restoration module, overcoming the issues of error accumulation and semantic inconsistency caused by isolated restoration. Experimental results on real Dunhuang murals demonstrate that the proposed method achieves better structural and texture consistency in the restoration results compared to the baseline methods, and objective evaluation metrics outperform the comparison algorithms.

Key words: Mural inpainting, Semantic reasoning, Joint learning, Context aggregation, Joint attention