Leveraging natural language processing, machine learning, and a knowledge-grounded language model to detect linguistic markers of deception
چکیده
Introduction Deception elicits measurable cognitive and linguistic changes, yet accurately identifying deceptive language in high-stakes environments remains a persistent challenge. Although natural language processing (NLP) and machine learning (ML) techniques have shown promise in detecting misinformation and low-stakes deception, their performance on spontaneous, stress-induced speech remains poorly understood. This study investigates linguistic markers of deception and evaluates the performance of supervised ML models and a knowledge-grounded language model, FranklinAsk™, across both synthetic and real-world datasets. Methods Linguistic analyses were conducted on responses from a National Center for Credibility Assessment (NCCA) military training exercise involving truthful and deceptive participants. A synthetic dataset was generated using MOSTLYAI to provide a balanced corpus for ML training. Linguistic features were extracted using pretrained NLP models, and supervised classifiers, including logistic regression, Random Forest, Naïve Bayes, and Support Vector Machine (SVM), were trained using LIWC-22 features with 10-fold cross-validation. The FranklinAsk™ RAG system was evaluated using the same participant responses to compare knowledge-grounded reasoning with statistical learning. Results Two linguistic features, cardinal numbers (nominal p = 0.041) and particles (nominal p = 0.0486), reached the unadjusted significance threshold, each with a moderate effect size ( d = 0.55); neither remained statistically significant after Benjamini-Hochberg FDR correction (adjusted p = 0.668 for both). No significant sentiment differences were observed. On the synthetic dataset, logistic regression performed best (F1 = 0.930; ROC AUC = 0.978), but all supervised models showed substantial performance declines when applied to the real-world dataset. In contrast, FranklinAsk™ achieved the highest F1-score (0.519), highest accuracy (0.487), and second-highest recall (0.700) on the military exercise data. Discussion These findings highlight the limited generalizability of synthetic-trained deception models to real-world, high-stakes communication and motivate further investigation of knowledge-grounded language models as an alternative approach when labeled real-world training data are limited. Future research should prioritize expanding ecologically valid datasets and exploring hybrid approaches that combine statistical modeling with knowledge-grounded reasoning.