Label-free cell classification from microscopy images has emerged as a valuable alternative to traditional staining techniques, enabling high-throughput analysis while preserving cellular integrity. Despite promising advances through deep learning, progress has been hindered by the absence of large, standardized datasets to support model development and evaluation. In this study, LIVECell-CLS is introduced as the largest benchmark dataset for label-free cell classification to date, comprising over 1.6 million cropped images derived from the LIVECell segmentation dataset and encompassing eight morphologically distinct cell lines. A diverse set of deep learning architectures - including ResNets, Vision Transformers (ViTs), and MLP-Mixers—along with their Tensor Network-based variants, have been systematically evaluated. These models incorporate a C. elegans connectome-inspired module designed to enhance latent feature representation prior to classification. To gain insights into model behavior and feature representations, multiple Explainable AI (XAI) methods have been applied in conjunction with UMAP-based visualizations. The results indicate that improvements in classification accuracy are closely associated with increased feature separability and interpretability, particularly in challenging cases involving closely related cell morphologies. This work provides a comprehensive benchmark for future developments in label-free microscopy and underscores the importance of model explainability in biomedical AI.