Accurate segmentation of colorectal cancer (CRC) from endoscopic images is crucial for computer-aided diagnosis. Visual intelligence enhances detection precision, supporting clinical decision-making. However, current segmentation methods often struggle with accurately delineating fine-grained lesion boundaries due to limited context comprehension and inadequate attention to optimal features. Additionally, the poor fusion of multi-scale semantic cues hinders performance, especially in complex endoscopic scenarios. To address these issues, we introduce ColoSegNet, a Visual Intelligence-Driven Triple Attention Feature Fusion Network designed for high-precision CRC segmentation. Our approach begins with an analysis of backbone networks to identify optimal intermediate-level feature extraction. The architecture includes a non-local attention module to capture long-range dependencies, enhanced by channel and spatial attention mechanisms for better feature selectivity across semantic hierarchies. Multi-scale fusion strategies are used to preserve both low-level details and high-level context, enabling accurate localization under challenging conditions. Extensive evaluations across four publicly available endoscopic datasets, along with cross-dataset assessments, demonstrate the effectiveness and generalizability of ColoSegNet, outperforming state-of-the-art segmentation methods.