The lack of inherent characteristics of infrared small targets, with issues such as high time consumption, prevents applying existing detection methods in complex environments. Herein, we propose an infrared small-target detection algorithm suitable for embedded edge-computing devices which transforms the problem of small-target detection into the problem of semantic segmentation. A lightweight backbone network is used to extract the features, a cross-layer feature fusion method based on context modulation is designed to exchange high-level semantics and low-level details, and dual attention mechanism based on channels and locations is introduced to highlight the small targets in the feature map. Experiments prove that this algorithm outperforms existing state-of-the-art methods in terms of detection effects, false alarms, and time consumption in complex backgrounds. The model is only 100 KB in size and can detect video streams at 20 frame/s in real-time, making it ideal for embedded deployment and practical applications.