AI-Enabled Training Micro-Agents Longitudinal Effects on Adoption, Learning Efficiency, and Human Oversight

Abstract

This study reports a longitudinal assessment of micro-agents facilitated by AI in a front-line hospitality environment. It compares a supervised micro-agent deployment in 2024 with a scaled organizational deployment in 2025. Deployment maturity is the independent variable, while objective learning platform trace metrics -   completion rate, assessment performance, and time-on-task - that comprise the dependent variables of this study. An exposure-adjusted active employee model is utilized to reduce potential biases due to a high employee turnover rate. A decrease in completion rate is found from 100% (656/656) in the supervised micro-agent deployment to 86.82% (8,505/9,796) in the scaled micro-agent deployment. A two-proportion z-test shows a significant difference (z = 11.52, p < .001) with a decrease in completion rate by 13.18% (95% CI [12.51, 13.85]). This decline is consistent with normalization effects commonly observed when controlled pilot interventions transition to scaled operational environments. Assessment performance increases from M = 80.10 (SD = 21.55) to M = 83.38 (SD = 23.14) with a small effect size (Welch's t (1108.70) = 4.29, p < .001; Cohen's d = 0.14; 95% CI [1.78, 4.79]). A decrease in mean time-on-task is found from 10.30 minutes (SD = 11.53) to 5.98 minutes (SD = 7.82) with a moderate efficiency effect (Welch's t(971.64) = -10.88, p < .001; Cohen's d = -0.52; 95% CI [-5.09, -3.53]). The findings provide empirical evidence regarding the effectiveness of AI-enabled micro-agent frameworks within frontline organizational learning environments. It links longitudinal behavioral traces with micro-agent frameworks, providing a replicable model for assessing the effectiveness of AI-enhanced organizational learning systems.