To identify the phytoplankton community by discrete three-dimensional fluorescence spectra, we studied a method based on a self-weighted alternating trilinear decomposition (SWTATLD) algorithm for five common alga species (Microcystis aeruginosa, Scenedesmus obliquus, Nitzschia sp., Peridinium umbonatum var. inaequale, and Cryptomonas obovata.). Then, the results were compared with those of the parallel factor (PARAFAC) algorithm. The recovery of the PARAFAC algorithm is 92.73%±13.99% for Microcystis aeruginosa, 105.51%±11.58% for Scenedesmus obliquus, 89.25%±13.68% for Nitzschia sp., 109.48%±13.47% for Peridinium umbonatum var. inaequale and 88.76%±13.60% for Cryptomonas obovata. The recovery of the SWTATLD algorithm is 96.70%±3.94% for Microcystis aeruginosa, 98.07%±4.48% for Scenedesmus obliquus, 101.71%±3.97% for Nitzschia sp., 97.26%±4.11% for Peridinium umbonatum var. inaequale, and 103.57%±4.34% for Cryptomonas obovata. The recovery results based on the SWTATLD algorithm were closer to the real concentration and have smaller deviations than those based on the PARAFAC algorithm. Our results provide a method for the effective identification and quantitative analysis of phytoplankton communities.