Planar cameras with high-performance and wide field-of-view (FOV) are critical in various fields, requiring highly compact and integrated technology. Existing wide FOV metalenses show great potential for ultra-thin optical components, but there are a set of tricky challenges like chromatic aberrations correction, central bright speckle removal, and image quality improvement of wide FOVs. In this paper, we design a neural meta-camera by introducing a knowledge-fused data-driven (KD) paradigm equipped with transformer-based network. Such paradigm enables the network to sequentially assimilate the physical prior and experimental data of the metalens, and thus can effectively mitigate the aforementioned challenges. An ultra-wide FOV meta-camera, integrating an off-axis monochromatic aberration-corrected metalens with a neural CMOS image sensor without any relay lenses, is employed to demonstrate the availability. High-quality reconstructed results of color images and real scene images at different distance validate that the proposed meta-camera can achieve ultra-wide FOV (> 100-degree) and full-color image with the correction of chromatic aberration, distortion and central bright speckle, and the contrast increase up to 13.5 times. Notably, coupled with its compact size (<0.13 cm3), portability, and full-color imaging capabilities, the neural meta-camera emerges as a compelling alternative for applications such as micro-navigation, micro-endoscopes, and various on-chip devices.