Abstract: Addressing the issues of false positives and low detection accuracy in generative face detection tasks, this paper proposes the RFCE-YOLOv8 generative face detection algorithm. Firstly, the RCS-OSA (RCS-Based One-Shot Aggregation) module is introduced into the backbone network to replace the original C2f module. Leveraging the repetitive stacking characteristics of the RCS module ensures feature reuse and enhances the flow of information between adjacent layer features, thereby strengthening the model's feature extraction capability. Secondly, the Focal Modulation module is introduced, employing a focal modulation mechanism to focus on key areas of the image, enhancing the capture ability of long-distance dependencies and contextual information, thereby improving the detection accuracy of targets. Next, the CBAM (Convolutional Block Attention Module) attention mechanism is introduced to capture facial detail features from both channel and spatial dimensions, enabling the model to adaptively focus on key channels and spatial positions in the input features, thereby improving the model's performance. Finally, the bounding box regression loss function is optimized, replacing CIoU with Focal-EIoU, dynamically adjusting sample weights to focus the model more on high-quality samples, reduce interference from low-quality samples, and improve the regression accuracy of the model. Experimental results demonstrate that the improved algorithm achieves an mAP of 93.87%, a 2.28% improvement over the baseline algorithm, effectively addressing false positives and improving detection accuracy in generative face detection tasks.

Key words: generative face detection, YOLOv8, feature enhancement, feature extraction, attention mechanism, loss function optimization