Herein, we propose a Transformer multiattention network (TransMANet), a network structure based on Transformer and attention mechanisms, to address the issues of low segmentation accuracy, inadequate global feature extraction, and insufficient association between the multiattention network (MANet) algorithm and image semantic information. This network structure features a dual-branch decoder that combines local and global contexts and enhances the semantic information of shallow networks. First, we introduce a local attention embedding mechanism that enhances the embedding of context information and semantic information of high-level features into low-level features. Then, we design a dual-branch decoder that combines Transformer and convolutional neural networks, which extracts global context information and detailed information with different scales, thereby modeling global and local information. Finally, we improve the original loss function and use a joint loss function that combines cross-entropy loss and Dice loss to address the class imbalance problem often encountered in remote sensing datasets and thus improve segmentation accuracy. Our experimental results demonstrate the superiority of TransMANet over MANet and other advanced methods in terms of intersection over union on UAVid, LoveDA, Potsdam, and Vaihingen datasets. This indicates the strong generalization capability of TransMANet and its effectiveness in achieving accurate segmentation results.